[2025-10-20T17:00:02.334531] [INFO ] [MAIN ] Using last completed trading day: 2025-10-20 [2025-10-20T17:00:02.518040] [DEBUG] [R-STATUS ] Registered 'tdm' as a dependent service. [2025-10-20T17:00:02.518206] [INFO ] [STATE ] Waiting for all dependent services to be clear... [2025-10-20T17:00:02.704747] [WARN ] [tdm-status] Service 'tdm' is not running. [2025-10-20T17:00:02.705159] [WARN ] [STATE ] State set to: RUNNING [2025-10-20T17:00:02.705223] [INFO ] [MAIN ] Refreshing symbols list before update... [2025-10-20T17:00:02.705261] [INFO ] [MAIN ] Starting TDM symbol refresh process. [2025-10-20T17:00:02.705340] [INFO ] [MAIN ] Fetching latest NYSE symbols list. [2025-10-20T17:00:02.705375] [DEBUG] [SYMBOL ] Starting fetch of NYSE Symbols. [2025-10-20T17:00:04.348978] [DEBUG] [SYMBOL ] Found 2419 valid NYSE symbols. [2025-10-20T17:00:04.349149] [INFO ] [SYMBOL ] Completed fetch of NYSE Symbols. [2025-10-20T17:00:04.349348] [INFO ] [MAIN ] Filtered out 54 blacklisted symbols. [2025-10-20T17:00:04.349895] [INFO ] [LOCAL ] Symbols cache updated with 2367 symbols. [2025-10-20T17:00:04.349951] [INFO ] [MAIN ] Successfully refreshed symbols list. [2025-10-20T17:00:04.349982] [INFO ] [MAIN ] Finished TDM symbol refresh process. [2025-10-20T17:00:04.350036] [INFO ] [MAIN ] Pruning stale symbols... [2025-10-20T17:00:04.350068] [INFO ] [MAIN ] Starting TDM stale symbol pruning process. [2025-10-20T17:00:04.350097] [INFO ] [LOCAL ] Reading cached symbols list. [2025-10-20T17:00:04.350475] [DEBUG] [LOCAL ] Found 2367 symbols in cached list. [2025-10-20T17:00:04.350526] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-10-20T17:00:04.350702] [INFO ] [MAIN ] Including 54 blacklisted symbols for pruning. [2025-10-20T17:00:04.350757] [INFO ] [MAIN ] Pruning data for symbols not in current list of 2367 valid symbols. [2025-10-20T17:00:04.350789] [INFO ] [LOCAL ] Starting pruning of local symbol history data for inactive/invalid symbols. [2025-10-20T17:00:04.350819] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/primary_storage' [2025-10-20T17:00:04.370323] [DEBUG] [LOCAL ] Deleted /opt/tdm/output/data/primary_storage/BTCM.csv (symbol BTCM no longer active). [2025-10-20T17:00:04.376525] [DEBUG] [LOCAL ] Deleted /opt/tdm/output/data/primary_storage/SAND.csv (symbol SAND no longer active). [2025-10-20T17:00:04.377825] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/derivative_storage' [2025-10-20T17:00:04.397126] [DEBUG] [LOCAL ] Deleted /opt/tdm/output/data/derivative_storage/BTCM.csv (symbol BTCM no longer active). [2025-10-20T17:00:04.403000] [DEBUG] [LOCAL ] Deleted /opt/tdm/output/data/derivative_storage/SAND.csv (symbol SAND no longer active). [2025-10-20T17:00:04.404326] [INFO ] [LOCAL ] Deleted 4 obsolete data files. [2025-10-20T17:00:04.404380] [INFO ] [LOCAL ] Completed pruning of unlisted local symbol history data. [2025-10-20T17:00:04.404413] [INFO ] [MAIN ] Successfully pruned stale symbol data. [2025-10-20T17:00:04.404441] [INFO ] [MAIN ] Finished TDM stale symbol pruning process. [2025-10-20T17:00:04.404496] [INFO ] [LOCAL ] Reading cached symbols list. [2025-10-20T17:00:04.404873] [DEBUG] [LOCAL ] Found 2367 symbols in cached list. [2025-10-20T17:00:04.404924] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-10-20T17:00:04.404957] [INFO ] [MAIN ] Starting incremental update for 2367 symbols. [2025-10-20T17:00:04.404984] [INFO ] [MAIN ] Running in parallel mode [2025-10-20T17:00:04.405151] [INFO ] [MAIN ] Starting parallel primary update with 4 worker threads for 2367 symbols [2025-10-20T17:00:05.038992] [INFO ] [MAIN ] AAMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:05.038131] [INFO ] [MAIN ] A: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:05.068031] [INFO ] [MAIN ] AA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:05.066449] [DEBUG] [LOCAL ] AAMI: No existing data file found in primary storage. [2025-10-20T17:00:05.103732] [WARN ] [MAIN ] AAMI: No existing data found, performing full refresh instead. [2025-10-20T17:00:05.107633] [INFO ] [MAIN ] AAMI: Wiping existing primary data before full refresh. [2025-10-20T17:00:05.134255] [INFO ] [LOCAL ] AAMI: Starting deletion of primary data. [2025-10-20T17:00:05.132897] [INFO ] [MAIN ] AAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:05.134381] [DEBUG] [LOCAL ] AAMI: No primary data file found to delete. [2025-10-20T17:00:05.134483] [INFO ] [MAIN ] AAMI: Starting full primary refresh. [2025-10-20T17:00:05.134534] [DEBUG] [LOCAL ] AAM: No existing data file found in primary storage. [2025-10-20T17:00:05.140386] [WARN ] [MAIN ] AAM: No existing data found, performing full refresh instead. [2025-10-20T17:00:05.208869] [INFO ] [MAIN ] AAM: Wiping existing primary data before full refresh. [2025-10-20T17:00:05.225485] [INFO ] [LOCAL ] AAM: Starting deletion of primary data. [2025-10-20T17:00:05.239586] [DEBUG] [LOCAL ] AAM: No primary data file found to delete. [2025-10-20T17:00:05.240002] [INFO ] [MAIN ] AAM: Starting full primary refresh. [2025-10-20T17:00:05.763366] [DEBUG] [LOCAL ] A: Loaded 51523 existing records from primary storage. [2025-10-20T17:00:05.818078] [INFO ] [REMOTE ] AAMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:05.845594] [DEBUG] [LOCAL ] AA: Loaded 54401 existing records from primary storage. [2025-10-20T17:00:05.851241] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-01. [2025-10-20T17:00:05.851444] [INFO ] [MAIN ] A: Loaded 51523 existing records. [2025-10-20T17:00:05.863040] [INFO ] [MAIN ] AA: Loaded 54401 existing records. [2025-10-20T17:00:06.121022] [INFO ] [REMOTE ] AAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:06.147968] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-01. [2025-10-20T17:00:06.264550] [WARN ] [MAIN ] A: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:06.275035] [INFO ] [MAIN ] A: Wiping existing primary data before full refresh. [2025-10-20T17:00:06.280236] [INFO ] [LOCAL ] A: Starting deletion of primary data. [2025-10-20T17:00:06.294375] [DEBUG] [LOCAL ] A: Deleted primary data file. [2025-10-20T17:00:06.299849] [INFO ] [MAIN ] A: Starting full primary refresh. [2025-10-20T17:00:06.413418] [WARN ] [MAIN ] AA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:06.437835] [INFO ] [MAIN ] AA: Wiping existing primary data before full refresh. [2025-10-20T17:00:06.440327] [INFO ] [LOCAL ] AA: Starting deletion of primary data. [2025-10-20T17:00:06.440880] [DEBUG] [LOCAL ] AA: Deleted primary data file. [2025-10-20T17:00:06.446261] [INFO ] [MAIN ] AA: Starting full primary refresh. [2025-10-20T17:00:07.030268] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T17:00:07.032681] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-02. [2025-10-20T17:00:07.036815] [INFO ] [REMOTE ] A: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:07.037805] [INFO ] [REMOTE ] A: Fetching primary data for 2025-01. [2025-10-20T17:00:07.122411] [INFO ] [REMOTE ] AA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:07.122893] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-01. [2025-10-20T17:00:07.292735] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:00:07.296932] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-02. [2025-10-20T17:00:07.542599] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T17:00:07.543554] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-03. [2025-10-20T17:00:08.162073] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T17:00:08.163040] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-04. [2025-10-20T17:00:08.514797] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T17:00:08.515778] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-05. [2025-10-20T17:00:08.713940] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:00:08.717946] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-03. [2025-10-20T17:00:08.755257] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-10-20T17:00:08.756083] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-06. [2025-10-20T17:00:09.015543] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-06 (0.26 seconds). [2025-10-20T17:00:09.016583] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-07. [2025-10-20T17:00:09.073728] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T17:00:09.084189] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-02. [2025-10-20T17:00:09.338312] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-10-20T17:00:09.339351] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-08. [2025-10-20T17:00:09.504898] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-01 (2.47 seconds). [2025-10-20T17:00:09.514552] [INFO ] [REMOTE ] A: Fetching primary data for 2025-02. [2025-10-20T17:00:09.627735] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-10-20T17:00:09.628804] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-09. [2025-10-20T17:00:09.873819] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-09 (0.25 seconds). [2025-10-20T17:00:09.874728] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-10. [2025-10-20T17:00:09.982202] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:00:09.987135] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-04. [2025-10-20T17:00:10.244887] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-10-20T17:00:10.245520] [INFO ] [REMOTE ] AAM: Full fetch complete (918 records). [2025-10-20T17:00:10.245590] [DEBUG] [VALIDATOR ] AAM: Validating primary data. [2025-10-20T17:00:10.419815] [WARN ] [VALIDATOR ] AAM: Validation failed - insufficient historical data. Have 184 unique days, need 200 days [2025-10-20T17:00:10.419975] [FATAL] [MAIN ] AAM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:00:10.420021] [WARN ] [MAIN ] AAM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:00:10.420061] [INFO ] [LOCAL ] AAM: Starting deletion of primary data. [2025-10-20T17:00:10.420114] [DEBUG] [LOCAL ] AAM: No primary data file found to delete. [2025-10-20T17:00:10.420156] [INFO ] [LOCAL ] AAM: Starting deletion of derivative data. [2025-10-20T17:00:10.420195] [DEBUG] [LOCAL ] AAM: No derivative data file found to delete. [2025-10-20T17:00:10.420246] [INFO ] [MAIN ] Update Progress: 1/2367 symbols completed. AAM took 6.0s. Estimated remaining: 3h 57m 12s. Rate: 10.0 symbols/min [2025-10-20T17:00:10.585806] [INFO ] [MAIN ] AAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:10.638242] [DEBUG] [LOCAL ] AAP: Loaded 52578 existing records from primary storage. [2025-10-20T17:00:10.638579] [INFO ] [MAIN ] AAP: Loaded 52578 existing records. [2025-10-20T17:00:10.815126] [WARN ] [MAIN ] AAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:10.815343] [INFO ] [MAIN ] AAP: Wiping existing primary data before full refresh. [2025-10-20T17:00:10.815389] [INFO ] [LOCAL ] AAP: Starting deletion of primary data. [2025-10-20T17:00:10.815971] [DEBUG] [LOCAL ] AAP: Deleted primary data file. [2025-10-20T17:00:10.816030] [INFO ] [MAIN ] AAP: Starting full primary refresh. [2025-10-20T17:00:11.181452] [INFO ] [REMOTE ] AAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:11.182429] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-01. [2025-10-20T17:00:11.239052] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-02 (2.15 seconds). [2025-10-20T17:00:11.238078] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:00:11.258760] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-03. [2025-10-20T17:00:11.259002] [INFO ] [REMOTE ] A: Fetching primary data for 2025-03. [2025-10-20T17:00:11.574505] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:00:11.581040] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-05. [2025-10-20T17:00:12.835299] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:00:12.845854] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-02. [2025-10-20T17:00:12.866347] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T17:00:12.871980] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-06. [2025-10-20T17:00:13.040833] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T17:00:13.051341] [INFO ] [REMOTE ] A: Fetching primary data for 2025-04. [2025-10-20T17:00:13.181104] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T17:00:13.191961] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-04. [2025-10-20T17:00:13.857329] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T17:00:13.862327] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-07. [2025-10-20T17:00:14.568464] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:00:14.578563] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-03. [2025-10-20T17:00:14.741533] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:00:14.752475] [INFO ] [REMOTE ] A: Fetching primary data for 2025-05. [2025-10-20T17:00:14.885030] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T17:00:14.890834] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-08. [2025-10-20T17:00:15.439044] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-04 (2.25 seconds). [2025-10-20T17:00:15.451741] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-05. [2025-10-20T17:00:15.782214] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T17:00:15.787854] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-09. [2025-10-20T17:00:16.145221] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T17:00:16.157178] [INFO ] [REMOTE ] A: Fetching primary data for 2025-06. [2025-10-20T17:00:16.493789] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T17:00:16.505263] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-04. [2025-10-20T17:00:17.092151] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T17:00:17.098392] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-10. [2025-10-20T17:00:17.341107] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T17:00:17.353377] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-06. [2025-10-20T17:00:17.393014] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T17:00:17.404421] [INFO ] [REMOTE ] A: Fetching primary data for 2025-07. [2025-10-20T17:00:18.355357] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:00:18.368727] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-05. [2025-10-20T17:00:18.389695] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T17:00:18.394178] [INFO ] [REMOTE ] AAMI: Full fetch complete (33226 records). [2025-10-20T17:00:18.394354] [DEBUG] [VALIDATOR ] AAMI: Validating primary data. [2025-10-20T17:00:18.828935] [DEBUG] [VALIDATOR ] AAMI: All expected trading days are present. [2025-10-20T17:00:18.829121] [DEBUG] [VALIDATOR ] AAMI: Primary data validation passed. [2025-10-20T17:00:18.829547] [INFO ] [MAIN ] AAMI: Fetched primary data is valid. [2025-10-20T17:00:18.829615] [DEBUG] [LOCAL ] AAMI: Saving primary data. [2025-10-20T17:00:18.830903] [DEBUG] [LOCAL ] AAMI: Trimmed 33226 records to 24762 (keeping 200 days of data) [2025-10-20T17:00:18.995126] [DEBUG] [LOCAL ] AAMI: Primary data saved (24762 records). [2025-10-20T17:00:18.995337] [INFO ] [MAIN ] AAMI: Full history saved. [2025-10-20T17:00:18.995496] [INFO ] [MAIN ] Update Progress: 2/2367 symbols completed. AAMI took 14.5s. Estimated remaining: 4h 47m 33s. Rate: 8.2 symbols/min [2025-10-20T17:00:19.186982] [INFO ] [MAIN ] AAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:19.246334] [DEBUG] [LOCAL ] AAT: Loaded 30862 existing records from primary storage. [2025-10-20T17:00:19.247392] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T17:00:19.258605] [INFO ] [MAIN ] AAT: Loaded 30862 existing records. [2025-10-20T17:00:19.292432] [INFO ] [REMOTE ] A: Fetching primary data for 2025-08. [2025-10-20T17:00:19.450067] [WARN ] [MAIN ] AAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:19.450358] [INFO ] [MAIN ] AAT: Wiping existing primary data before full refresh. [2025-10-20T17:00:19.450532] [INFO ] [LOCAL ] AAT: Starting deletion of primary data. [2025-10-20T17:00:19.450762] [DEBUG] [LOCAL ] AAT: Deleted primary data file. [2025-10-20T17:00:19.450817] [INFO ] [MAIN ] AAT: Starting full primary refresh. [2025-10-20T17:00:19.773737] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-06 (2.42 seconds). [2025-10-20T17:00:19.807012] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-07. [2025-10-20T17:00:19.874730] [INFO ] [REMOTE ] AAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:19.875502] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-01. [2025-10-20T17:00:20.276482] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T17:00:20.289627] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-06. [2025-10-20T17:00:20.880994] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T17:00:20.894427] [INFO ] [REMOTE ] A: Fetching primary data for 2025-09. [2025-10-20T17:00:20.915535] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:00:20.921192] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-02. [2025-10-20T17:00:21.640175] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T17:00:21.652980] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-08. [2025-10-20T17:00:21.822733] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T17:00:21.828797] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-03. [2025-10-20T17:00:22.019551] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:00:22.032432] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-07. [2025-10-20T17:00:22.375083] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T17:00:22.387209] [INFO ] [REMOTE ] A: Fetching primary data for 2025-10. [2025-10-20T17:00:23.218166] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:00:23.226217] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-04. [2025-10-20T17:00:23.585040] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T17:00:23.597379] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-09. [2025-10-20T17:00:23.689029] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T17:00:23.700729] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-08. [2025-10-20T17:00:24.407283] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T17:00:24.415054] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-05. [2025-10-20T17:00:24.483707] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:00:24.491444] [INFO ] [REMOTE ] A: Full fetch complete (73577 records). [2025-10-20T17:00:24.491693] [DEBUG] [VALIDATOR ] A: Validating primary data. [2025-10-20T17:00:24.926245] [DEBUG] [VALIDATOR ] A: All expected trading days are present. [2025-10-20T17:00:24.926348] [DEBUG] [VALIDATOR ] A: Primary data validation passed. [2025-10-20T17:00:24.927033] [INFO ] [MAIN ] A: Fetched primary data is valid. [2025-10-20T17:00:24.927102] [DEBUG] [LOCAL ] A: Saving primary data. [2025-10-20T17:00:24.929088] [DEBUG] [LOCAL ] A: Trimmed 73577 records to 50738 (keeping 200 days of data) [2025-10-20T17:00:25.250828] [DEBUG] [LOCAL ] A: Primary data saved (50738 records). [2025-10-20T17:00:25.250993] [INFO ] [MAIN ] A: Full history saved. [2025-10-20T17:00:25.251306] [INFO ] [MAIN ] Update Progress: 3/2367 symbols completed. A took 20.8s. Estimated remaining: 4h 33m 47s. Rate: 8.6 symbols/min [2025-10-20T17:00:25.466509] [INFO ] [MAIN ] AAUC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:25.466880] [DEBUG] [LOCAL ] AAUC: No existing data file found in primary storage. [2025-10-20T17:00:25.467117] [WARN ] [MAIN ] AAUC: No existing data found, performing full refresh instead. [2025-10-20T17:00:25.467429] [INFO ] [MAIN ] AAUC: Wiping existing primary data before full refresh. [2025-10-20T17:00:25.467884] [INFO ] [LOCAL ] AAUC: Starting deletion of primary data. [2025-10-20T17:00:25.468274] [DEBUG] [LOCAL ] AAUC: No primary data file found to delete. [2025-10-20T17:00:25.468355] [INFO ] [MAIN ] AAUC: Starting full primary refresh. [2025-10-20T17:00:25.785025] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T17:00:25.887951] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:00:25.896724] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-09. [2025-10-20T17:00:25.942727] [INFO ] [REMOTE ] AAUC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:25.944395] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-06. [2025-10-20T17:00:25.945309] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-01. [2025-10-20T17:00:25.946879] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-09 (2.35 seconds). [2025-10-20T17:00:25.959953] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-10. [2025-10-20T17:00:26.157623] [DEBUG] [REMOTE ] AAUC: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:00:26.158069] [WARN ] [REMOTE ] AAUC: No data returned for month 2025-01 [2025-10-20T17:00:26.158160] [FATAL] [REMOTE ] AAUC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:00:26.158265] [FATAL] [MAIN ] AAUC: Failed to fetch primary data. [2025-10-20T17:00:26.158338] [WARN ] [MAIN ] AAUC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:00:26.158405] [INFO ] [LOCAL ] AAUC: Starting deletion of primary data. [2025-10-20T17:00:26.158492] [DEBUG] [LOCAL ] AAUC: No primary data file found to delete. [2025-10-20T17:00:26.158604] [INFO ] [LOCAL ] AAUC: Starting deletion of derivative data. [2025-10-20T17:00:26.158734] [DEBUG] [LOCAL ] AAUC: No derivative data file found to delete. [2025-10-20T17:00:26.158835] [INFO ] [MAIN ] Update Progress: 4/2367 symbols completed. AAUC took 0.9s. Estimated remaining: 3h 34m 11s. Rate: 11.0 symbols/min [2025-10-20T17:00:26.327235] [INFO ] [MAIN ] AB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:26.327405] [DEBUG] [LOCAL ] AB: No existing data file found in primary storage. [2025-10-20T17:00:26.327442] [WARN ] [MAIN ] AB: No existing data found, performing full refresh instead. [2025-10-20T17:00:26.327552] [INFO ] [MAIN ] AB: Wiping existing primary data before full refresh. [2025-10-20T17:00:26.327594] [INFO ] [LOCAL ] AB: Starting deletion of primary data. [2025-10-20T17:00:26.327679] [DEBUG] [LOCAL ] AB: No primary data file found to delete. [2025-10-20T17:00:26.327750] [INFO ] [MAIN ] AB: Starting full primary refresh. [2025-10-20T17:00:26.674038] [INFO ] [REMOTE ] AB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:26.674367] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-01. [2025-10-20T17:00:27.110479] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T17:00:27.116819] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-07. [2025-10-20T17:00:27.276125] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:00:27.286906] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-10. [2025-10-20T17:00:27.895169] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T17:00:27.902475] [INFO ] [REMOTE ] AA: Full fetch complete (77343 records). [2025-10-20T17:00:27.902613] [DEBUG] [VALIDATOR ] AA: Validating primary data. [2025-10-20T17:00:27.903744] [FATAL] [VALIDATOR ] AA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:00:27.903832] [FATAL] [MAIN ] AA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:00:27.903874] [WARN ] [MAIN ] AA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:00:27.903937] [INFO ] [LOCAL ] AA: Starting deletion of primary data. [2025-10-20T17:00:27.904000] [DEBUG] [LOCAL ] AA: No primary data file found to delete. [2025-10-20T17:00:27.904035] [INFO ] [LOCAL ] AA: Starting deletion of derivative data. [2025-10-20T17:00:27.904161] [DEBUG] [LOCAL ] AA: Deleted derivative data file. [2025-10-20T17:00:27.904477] [INFO ] [MAIN ] Update Progress: 5/2367 symbols completed. AA took 23.5s. Estimated remaining: 3h 5m 1s. Rate: 12.8 symbols/min [2025-10-20T17:00:28.073489] [INFO ] [MAIN ] ABBV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:28.126527] [DEBUG] [LOCAL ] ABBV: Loaded 54465 existing records from primary storage. [2025-10-20T17:00:28.126889] [INFO ] [MAIN ] ABBV: Loaded 54465 existing records. [2025-10-20T17:00:28.328940] [WARN ] [MAIN ] ABBV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:28.329341] [INFO ] [MAIN ] ABBV: Wiping existing primary data before full refresh. [2025-10-20T17:00:28.331794] [INFO ] [LOCAL ] ABBV: Starting deletion of primary data. [2025-10-20T17:00:28.333382] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:00:28.341348] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-02. [2025-10-20T17:00:28.340575] [DEBUG] [LOCAL ] ABBV: Deleted primary data file. [2025-10-20T17:00:28.341673] [INFO ] [MAIN ] ABBV: Starting full primary refresh. [2025-10-20T17:00:28.519287] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T17:00:28.624979] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-08. [2025-10-20T17:00:28.734987] [INFO ] [REMOTE ] ABBV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:28.735336] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-01. [2025-10-20T17:00:29.206152] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:00:29.214290] [INFO ] [REMOTE ] AAP: Full fetch complete (75154 records). [2025-10-20T17:00:29.214465] [DEBUG] [VALIDATOR ] AAP: Validating primary data. [2025-10-20T17:00:29.615441] [DEBUG] [VALIDATOR ] AAP: All expected trading days are present. [2025-10-20T17:00:29.615545] [DEBUG] [VALIDATOR ] AAP: Primary data validation passed. [2025-10-20T17:00:29.616272] [INFO ] [MAIN ] AAP: Fetched primary data is valid. [2025-10-20T17:00:29.616339] [DEBUG] [LOCAL ] AAP: Saving primary data. [2025-10-20T17:00:29.619108] [DEBUG] [LOCAL ] AAP: Trimmed 75154 records to 51783 (keeping 200 days of data) [2025-10-20T17:00:29.935184] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:00:29.947282] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-09. [2025-10-20T17:00:29.985111] [DEBUG] [LOCAL ] AAP: Primary data saved (51783 records). [2025-10-20T17:00:29.987152] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:00:29.987495] [INFO ] [MAIN ] AAP: Full history saved. [2025-10-20T17:00:29.996488] [INFO ] [MAIN ] Update Progress: 6/2367 symbols completed. AAP took 19.6s. Estimated remaining: 2h 47m 50s. Rate: 14.1 symbols/min [2025-10-20T17:00:29.996885] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-03. [2025-10-20T17:00:30.174270] [INFO ] [MAIN ] ABCB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:30.202496] [DEBUG] [LOCAL ] ABCB: Loaded 31020 existing records from primary storage. [2025-10-20T17:00:30.202794] [INFO ] [MAIN ] ABCB: Loaded 31020 existing records. [2025-10-20T17:00:30.381448] [WARN ] [MAIN ] ABCB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:30.381790] [INFO ] [MAIN ] ABCB: Wiping existing primary data before full refresh. [2025-10-20T17:00:30.381894] [INFO ] [LOCAL ] ABCB: Starting deletion of primary data. [2025-10-20T17:00:30.382155] [DEBUG] [LOCAL ] ABCB: Deleted primary data file. [2025-10-20T17:00:30.382207] [INFO ] [MAIN ] ABCB: Starting full primary refresh. [2025-10-20T17:00:30.742926] [INFO ] [REMOTE ] ABCB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:30.743299] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-01. [2025-10-20T17:00:30.783288] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T17:00:30.793556] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-02. [2025-10-20T17:00:30.848996] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T17:00:30.855269] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-10. [2025-10-20T17:00:31.148272] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T17:00:31.157842] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-04. [2025-10-20T17:00:32.134153] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:00:32.140978] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-02. [2025-10-20T17:00:32.252851] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T17:00:32.261070] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-05. [2025-10-20T17:00:32.349079] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T17:00:32.364974] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T17:00:32.371323] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-03. [2025-10-20T17:00:32.372756] [INFO ] [REMOTE ] AAT: Full fetch complete (43029 records). [2025-10-20T17:00:32.372844] [DEBUG] [VALIDATOR ] AAT: Validating primary data. [2025-10-20T17:00:32.765721] [DEBUG] [VALIDATOR ] AAT: All expected trading days are present. [2025-10-20T17:00:32.765812] [DEBUG] [VALIDATOR ] AAT: Primary data validation passed. [2025-10-20T17:00:32.766252] [INFO ] [MAIN ] AAT: Fetched primary data is valid. [2025-10-20T17:00:32.766293] [DEBUG] [LOCAL ] AAT: Saving primary data. [2025-10-20T17:00:32.767304] [DEBUG] [LOCAL ] AAT: Trimmed 43029 records to 30441 (keeping 200 days of data) [2025-10-20T17:00:32.954035] [DEBUG] [LOCAL ] AAT: Primary data saved (30441 records). [2025-10-20T17:00:32.954173] [INFO ] [MAIN ] AAT: Full history saved. [2025-10-20T17:00:32.956434] [INFO ] [MAIN ] Update Progress: 7/2367 symbols completed. AAT took 14.0s. Estimated remaining: 2h 40m 26s. Rate: 14.7 symbols/min [2025-10-20T17:00:33.125078] [INFO ] [MAIN ] ABEV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:33.177005] [DEBUG] [LOCAL ] ABEV: Loaded 54055 existing records from primary storage. [2025-10-20T17:00:33.177364] [INFO ] [MAIN ] ABEV: Loaded 54055 existing records. [2025-10-20T17:00:33.366919] [WARN ] [MAIN ] ABEV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:33.367361] [INFO ] [MAIN ] ABEV: Wiping existing primary data before full refresh. [2025-10-20T17:00:33.367830] [INFO ] [LOCAL ] ABEV: Starting deletion of primary data. [2025-10-20T17:00:33.368494] [DEBUG] [LOCAL ] ABEV: Deleted primary data file. [2025-10-20T17:00:33.368635] [INFO ] [MAIN ] ABEV: Starting full primary refresh. [2025-10-20T17:00:33.484867] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T17:00:33.515377] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T17:00:33.537222] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-06. [2025-10-20T17:00:33.573559] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-03. [2025-10-20T17:00:33.813966] [INFO ] [REMOTE ] ABEV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:33.814295] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-01. [2025-10-20T17:00:34.463234] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T17:00:34.474893] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-04. [2025-10-20T17:00:34.488556] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T17:00:34.493312] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-07. [2025-10-20T17:00:35.044476] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T17:00:35.051671] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-04. [2025-10-20T17:00:35.484459] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T17:00:35.496093] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-02. [2025-10-20T17:00:35.636424] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T17:00:35.642103] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-08. [2025-10-20T17:00:36.252521] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:00:36.264297] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-05. [2025-10-20T17:00:36.391301] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:00:36.399913] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-05. [2025-10-20T17:00:36.416156] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T17:00:36.424258] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-09. [2025-10-20T17:00:37.059003] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T17:00:37.069188] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-03. [2025-10-20T17:00:37.241351] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T17:00:37.246757] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-10. [2025-10-20T17:00:37.485489] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T17:00:37.491851] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-06. [2025-10-20T17:00:38.117338] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T17:00:38.128342] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-06. [2025-10-20T17:00:38.211557] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T17:00:38.215089] [INFO ] [REMOTE ] AB: Full fetch complete (44285 records). [2025-10-20T17:00:38.215174] [DEBUG] [VALIDATOR ] AB: Validating primary data. [2025-10-20T17:00:38.546347] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T17:00:38.630402] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-07. [2025-10-20T17:00:38.649118] [DEBUG] [VALIDATOR ] AB: All expected trading days are present. [2025-10-20T17:00:38.649255] [DEBUG] [VALIDATOR ] AB: Primary data validation passed. [2025-10-20T17:00:38.649979] [INFO ] [MAIN ] AB: Fetched primary data is valid. [2025-10-20T17:00:38.665706] [DEBUG] [LOCAL ] AB: Saving primary data. [2025-10-20T17:00:38.682227] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T17:00:38.694208] [DEBUG] [LOCAL ] AB: Trimmed 44285 records to 25030 (keeping 200 days of data) [2025-10-20T17:00:38.694785] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-04. [2025-10-20T17:00:38.850535] [DEBUG] [LOCAL ] AB: Primary data saved (25030 records). [2025-10-20T17:00:38.850718] [INFO ] [MAIN ] AB: Full history saved. [2025-10-20T17:00:38.850804] [INFO ] [MAIN ] Update Progress: 8/2367 symbols completed. AB took 12.7s. Estimated remaining: 2h 49m 17s. Rate: 13.9 symbols/min [2025-10-20T17:00:39.025914] [INFO ] [MAIN ] ABG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:39.044685] [DEBUG] [LOCAL ] ABG: Loaded 18603 existing records from primary storage. [2025-10-20T17:00:39.044986] [INFO ] [MAIN ] ABG: Loaded 18603 existing records. [2025-10-20T17:00:39.229507] [WARN ] [MAIN ] ABG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:39.229819] [INFO ] [MAIN ] ABG: Wiping existing primary data before full refresh. [2025-10-20T17:00:39.229879] [INFO ] [LOCAL ] ABG: Starting deletion of primary data. [2025-10-20T17:00:39.230069] [DEBUG] [LOCAL ] ABG: Deleted primary data file. [2025-10-20T17:00:39.230126] [INFO ] [MAIN ] ABG: Starting full primary refresh. [2025-10-20T17:00:39.591712] [INFO ] [REMOTE ] ABG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:39.592463] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-01. [2025-10-20T17:00:40.125165] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:00:40.144581] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T17:00:40.153112] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-08. [2025-10-20T17:00:40.152824] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-07. [2025-10-20T17:00:40.537535] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T17:00:40.539953] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:00:40.556957] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-02. [2025-10-20T17:00:40.558245] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-05. [2025-10-20T17:00:41.251265] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T17:00:41.256755] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-03. [2025-10-20T17:00:41.402577] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T17:00:41.409445] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-09. [2025-10-20T17:00:42.065916] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:00:42.070830] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-04. [2025-10-20T17:00:42.442000] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-07 (2.29 seconds). [2025-10-20T17:00:42.453722] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-08. [2025-10-20T17:00:42.683936] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T17:00:42.696782] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-06. [2025-10-20T17:00:42.943343] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:00:42.950501] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-10. [2025-10-20T17:00:43.258383] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T17:00:43.264912] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-05. [2025-10-20T17:00:43.979118] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T17:00:43.982596] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-06. [2025-10-20T17:00:44.284770] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:00:44.289988] [INFO ] [REMOTE ] ABCB: Full fetch complete (44683 records). [2025-10-20T17:00:44.290190] [DEBUG] [VALIDATOR ] ABCB: Validating primary data. [2025-10-20T17:00:44.290967] [FATAL] [VALIDATOR ] ABCB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:00:44.291065] [FATAL] [MAIN ] ABCB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:00:44.291140] [WARN ] [MAIN ] ABCB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:00:44.291229] [INFO ] [LOCAL ] ABCB: Starting deletion of primary data. [2025-10-20T17:00:44.291320] [DEBUG] [LOCAL ] ABCB: No primary data file found to delete. [2025-10-20T17:00:44.291362] [INFO ] [LOCAL ] ABCB: Starting deletion of derivative data. [2025-10-20T17:00:44.291524] [DEBUG] [LOCAL ] ABCB: Deleted derivative data file. [2025-10-20T17:00:44.293143] [INFO ] [MAIN ] Update Progress: 9/2367 symbols completed. ABCB took 14.3s. Estimated remaining: 2h 54m 11s. Rate: 13.5 symbols/min [2025-10-20T17:00:44.468874] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-10-20T17:00:44.533189] [INFO ] [MAIN ] ABM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:44.535058] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-09. [2025-10-20T17:00:44.565124] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T17:00:44.577113] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-07. [2025-10-20T17:00:44.606444] [DEBUG] [LOCAL ] ABM: Loaded 32640 existing records from primary storage. [2025-10-20T17:00:44.606744] [INFO ] [MAIN ] ABM: Loaded 32640 existing records. [2025-10-20T17:00:44.792818] [WARN ] [MAIN ] ABM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:44.793068] [INFO ] [MAIN ] ABM: Wiping existing primary data before full refresh. [2025-10-20T17:00:44.793137] [INFO ] [LOCAL ] ABM: Starting deletion of primary data. [2025-10-20T17:00:44.793349] [DEBUG] [LOCAL ] ABM: Deleted primary data file. [2025-10-20T17:00:44.793410] [INFO ] [MAIN ] ABM: Starting full primary refresh. [2025-10-20T17:00:45.165884] [INFO ] [REMOTE ] ABM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:45.166926] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T17:00:45.167202] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-01. [2025-10-20T17:00:45.172074] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-07. [2025-10-20T17:00:46.297465] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T17:00:46.302475] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-08. [2025-10-20T17:00:46.767725] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-10-20T17:00:46.780629] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-10. [2025-10-20T17:00:46.824157] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-10-20T17:00:46.836522] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-08. [2025-10-20T17:00:46.970991] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:00:46.980224] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-02. [2025-10-20T17:00:47.374698] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T17:00:47.379762] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-09. [2025-10-20T17:00:47.623091] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T17:00:47.631778] [INFO ] [REMOTE ] ABBV: Full fetch complete (77458 records). [2025-10-20T17:00:47.631979] [DEBUG] [VALIDATOR ] ABBV: Validating primary data. [2025-10-20T17:00:47.633156] [FATAL] [VALIDATOR ] ABBV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:00:47.633282] [FATAL] [MAIN ] ABBV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:00:47.633326] [WARN ] [MAIN ] ABBV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:00:47.633366] [INFO ] [LOCAL ] ABBV: Starting deletion of primary data. [2025-10-20T17:00:47.633426] [DEBUG] [LOCAL ] ABBV: No primary data file found to delete. [2025-10-20T17:00:47.633461] [INFO ] [LOCAL ] ABBV: Starting deletion of derivative data. [2025-10-20T17:00:47.633600] [DEBUG] [LOCAL ] ABBV: Deleted derivative data file. [2025-10-20T17:00:47.636195] [INFO ] [MAIN ] Update Progress: 10/2367 symbols completed. ABBV took 19.7s. Estimated remaining: 2h 49m 50s. Rate: 13.9 symbols/min [2025-10-20T17:00:47.823574] [INFO ] [MAIN ] ABR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:47.878021] [DEBUG] [LOCAL ] ABR: Loaded 52735 existing records from primary storage. [2025-10-20T17:00:47.878422] [INFO ] [MAIN ] ABR: Loaded 52735 existing records. [2025-10-20T17:00:48.065605] [WARN ] [MAIN ] ABR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:48.066376] [INFO ] [MAIN ] ABR: Wiping existing primary data before full refresh. [2025-10-20T17:00:48.066481] [INFO ] [LOCAL ] ABR: Starting deletion of primary data. [2025-10-20T17:00:48.072254] [DEBUG] [LOCAL ] ABR: Deleted primary data file. [2025-10-20T17:00:48.081699] [INFO ] [MAIN ] ABR: Starting full primary refresh. [2025-10-20T17:00:48.115991] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T17:00:48.155980] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-03. [2025-10-20T17:00:48.268498] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T17:00:48.299737] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-10. [2025-10-20T17:00:48.486544] [INFO ] [REMOTE ] ABR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:48.486981] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-01. [2025-10-20T17:00:48.768869] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T17:00:48.780683] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-09. [2025-10-20T17:00:49.447323] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:00:49.450436] [INFO ] [REMOTE ] ABG: Full fetch complete (26719 records). [2025-10-20T17:00:49.450587] [DEBUG] [VALIDATOR ] ABG: Validating primary data. [2025-10-20T17:00:49.846789] [DEBUG] [VALIDATOR ] ABG: All expected trading days are present. [2025-10-20T17:00:49.846991] [DEBUG] [VALIDATOR ] ABG: Primary data validation passed. [2025-10-20T17:00:49.847414] [INFO ] [MAIN ] ABG: Fetched primary data is valid. [2025-10-20T17:00:49.847550] [DEBUG] [LOCAL ] ABG: Saving primary data. [2025-10-20T17:00:49.868633] [DEBUG] [LOCAL ] ABG: Trimmed 26719 records to 18128 (keeping 200 days of data) [2025-10-20T17:00:49.869694] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:00:49.878149] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-04. [2025-10-20T17:00:50.001840] [DEBUG] [LOCAL ] ABG: Primary data saved (18128 records). [2025-10-20T17:00:50.002191] [INFO ] [MAIN ] ABG: Full history saved. [2025-10-20T17:00:50.002277] [INFO ] [MAIN ] Update Progress: 11/2367 symbols completed. ABG took 11.2s. Estimated remaining: 2h 42m 46s. Rate: 14.5 symbols/min [2025-10-20T17:00:50.182179] [INFO ] [MAIN ] ABT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:50.182478] [DEBUG] [LOCAL ] ABT: No existing data file found in primary storage. [2025-10-20T17:00:50.182524] [WARN ] [MAIN ] ABT: No existing data found, performing full refresh instead. [2025-10-20T17:00:50.182634] [INFO ] [MAIN ] ABT: Wiping existing primary data before full refresh. [2025-10-20T17:00:50.182718] [INFO ] [LOCAL ] ABT: Starting deletion of primary data. [2025-10-20T17:00:50.182767] [DEBUG] [LOCAL ] ABT: No primary data file found to delete. [2025-10-20T17:00:50.182800] [INFO ] [MAIN ] ABT: Starting full primary refresh. [2025-10-20T17:00:50.546179] [INFO ] [REMOTE ] ABT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:50.546847] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-01. [2025-10-20T17:00:50.583757] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:00:50.594766] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-02. [2025-10-20T17:00:50.759804] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T17:00:50.772326] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-10. [2025-10-20T17:00:51.539808] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T17:00:51.547415] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-05. [2025-10-20T17:00:52.439032] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T17:00:52.450060] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-03. [2025-10-20T17:00:52.481762] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:00:52.492401] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-02. [2025-10-20T17:00:52.758890] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T17:00:52.764780] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-06. [2025-10-20T17:00:52.912989] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-10-20T17:00:52.921913] [INFO ] [REMOTE ] ABEV: Full fetch complete (77346 records). [2025-10-20T17:00:52.922110] [DEBUG] [VALIDATOR ] ABEV: Validating primary data. [2025-10-20T17:00:53.319593] [DEBUG] [VALIDATOR ] ABEV: All expected trading days are present. [2025-10-20T17:00:53.319750] [DEBUG] [VALIDATOR ] ABEV: Primary data validation passed. [2025-10-20T17:00:53.320492] [INFO ] [MAIN ] ABEV: Fetched primary data is valid. [2025-10-20T17:00:53.320547] [DEBUG] [LOCAL ] ABEV: Saving primary data. [2025-10-20T17:00:53.322236] [DEBUG] [LOCAL ] ABEV: Trimmed 77346 records to 53291 (keeping 200 days of data) [2025-10-20T17:00:53.629220] [DEBUG] [LOCAL ] ABEV: Primary data saved (53291 records). [2025-10-20T17:00:53.629377] [INFO ] [MAIN ] ABEV: Full history saved. [2025-10-20T17:00:53.630805] [INFO ] [MAIN ] Update Progress: 12/2367 symbols completed. ABEV took 20.7s. Estimated remaining: 2h 41m 1s. Rate: 14.6 symbols/min [2025-10-20T17:00:53.805744] [INFO ] [MAIN ] ACA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:53.832095] [DEBUG] [LOCAL ] ACA: Loaded 23255 existing records from primary storage. [2025-10-20T17:00:53.832414] [INFO ] [MAIN ] ACA: Loaded 23255 existing records. [2025-10-20T17:00:54.019766] [WARN ] [MAIN ] ACA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:54.021025] [INFO ] [MAIN ] ACA: Wiping existing primary data before full refresh. [2025-10-20T17:00:54.021135] [INFO ] [LOCAL ] ACA: Starting deletion of primary data. [2025-10-20T17:00:54.027958] [DEBUG] [LOCAL ] ACA: Deleted primary data file. [2025-10-20T17:00:54.038723] [INFO ] [MAIN ] ACA: Starting full primary refresh. [2025-10-20T17:00:54.049360] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T17:00:54.117512] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-07. [2025-10-20T17:00:54.488213] [INFO ] [REMOTE ] ACA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:54.489020] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-01. [2025-10-20T17:00:54.518680] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-10-20T17:00:54.558297] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-03. [2025-10-20T17:00:54.561432] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T17:00:54.572983] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-04. [2025-10-20T17:00:55.671287] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:00:55.676671] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-02. [2025-10-20T17:00:55.767118] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:00:55.775258] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-08. [2025-10-20T17:00:56.359020] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:00:56.370605] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-05. [2025-10-20T17:00:56.626432] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T17:00:56.638089] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-04. [2025-10-20T17:00:56.755548] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:00:56.761913] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-03. [2025-10-20T17:00:56.788750] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:00:56.797943] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-09. [2025-10-20T17:00:57.815609] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T17:00:57.822479] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-04. [2025-10-20T17:00:58.087306] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T17:00:58.096613] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-10. [2025-10-20T17:00:58.343104] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T17:00:58.354899] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-06. [2025-10-20T17:00:58.938917] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T17:00:58.945039] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-05. [2025-10-20T17:00:59.226752] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-04 (2.59 seconds). [2025-10-20T17:00:59.238586] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-05. [2025-10-20T17:00:59.378978] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T17:00:59.384217] [INFO ] [REMOTE ] ABM: Full fetch complete (45009 records). [2025-10-20T17:00:59.384327] [DEBUG] [VALIDATOR ] ABM: Validating primary data. [2025-10-20T17:00:59.818744] [DEBUG] [VALIDATOR ] ABM: All expected trading days are present. [2025-10-20T17:00:59.818858] [DEBUG] [VALIDATOR ] ABM: Primary data validation passed. [2025-10-20T17:00:59.819380] [INFO ] [MAIN ] ABM: Fetched primary data is valid. [2025-10-20T17:00:59.819425] [DEBUG] [LOCAL ] ABM: Saving primary data. [2025-10-20T17:00:59.820629] [DEBUG] [LOCAL ] ABM: Trimmed 45009 records to 32177 (keeping 200 days of data) [2025-10-20T17:01:00.047882] [DEBUG] [LOCAL ] ABM: Primary data saved (32177 records). [2025-10-20T17:01:00.048097] [INFO ] [MAIN ] ABM: Full history saved. [2025-10-20T17:01:00.049313] [INFO ] [MAIN ] Update Progress: 13/2367 symbols completed. ABM took 15.8s. Estimated remaining: 2h 47m 56s. Rate: 14.0 symbols/min [2025-10-20T17:01:00.084226] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:01:00.181155] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-07. [2025-10-20T17:01:00.249269] [INFO ] [MAIN ] ACCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:00.264481] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T17:01:00.270440] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-06. [2025-10-20T17:01:00.305431] [DEBUG] [LOCAL ] ACCO: Loaded 37509 existing records from primary storage. [2025-10-20T17:01:00.305762] [INFO ] [MAIN ] ACCO: Loaded 37509 existing records. [2025-10-20T17:01:00.478401] [WARN ] [MAIN ] ACCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:00.478598] [INFO ] [MAIN ] ACCO: Wiping existing primary data before full refresh. [2025-10-20T17:01:00.478676] [INFO ] [LOCAL ] ACCO: Starting deletion of primary data. [2025-10-20T17:01:00.478994] [DEBUG] [LOCAL ] ACCO: Deleted primary data file. [2025-10-20T17:01:00.479055] [INFO ] [MAIN ] ACCO: Starting full primary refresh. [2025-10-20T17:01:00.826903] [INFO ] [REMOTE ] ACCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:00.827234] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-01. [2025-10-20T17:01:01.167189] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T17:01:01.178194] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-06. [2025-10-20T17:01:01.215161] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T17:01:01.222011] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-07. [2025-10-20T17:01:01.998031] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T17:01:02.003773] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-08. [2025-10-20T17:01:02.119520] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T17:01:02.131173] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-08. [2025-10-20T17:01:02.816281] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T17:01:02.823929] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-02. [2025-10-20T17:01:02.902937] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T17:01:02.919294] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-07. [2025-10-20T17:01:03.393032] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:01:03.451503] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-09. [2025-10-20T17:01:04.312670] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-10-20T17:01:04.323985] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-09. [2025-10-20T17:01:04.363573] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:01:04.437826] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T17:01:05.051453] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-03. [2025-10-20T17:01:05.174769] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T17:01:05.851350] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-08. [2025-10-20T17:01:05.936413] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T17:01:06.588241] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:01:06.651340] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-04. [2025-10-20T17:01:07.451365] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-10. [2025-10-20T17:01:07.909777] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T17:01:08.251610] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-10. [2025-10-20T17:01:08.295188] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:01:08.504349] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:01:08.509277] [INFO ] [REMOTE ] ACA: Full fetch complete (34169 records). [2025-10-20T17:01:08.509420] [DEBUG] [VALIDATOR ] ACA: Validating primary data. [2025-10-20T17:01:08.510899] [FATAL] [VALIDATOR ] ACA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:01:08.511021] [FATAL] [MAIN ] ACA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:01:08.511099] [WARN ] [MAIN ] ACA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:01:08.511179] [INFO ] [LOCAL ] ACA: Starting deletion of primary data. [2025-10-20T17:01:08.511281] [DEBUG] [LOCAL ] ACA: No primary data file found to delete. [2025-10-20T17:01:08.511345] [INFO ] [LOCAL ] ACA: Starting deletion of derivative data. [2025-10-20T17:01:08.511614] [DEBUG] [LOCAL ] ACA: Deleted derivative data file. [2025-10-20T17:01:08.511746] [INFO ] [MAIN ] Update Progress: 14/2367 symbols completed. ACA took 14.9s. Estimated remaining: 2h 59m 35s. Rate: 13.1 symbols/min [2025-10-20T17:01:08.735256] [INFO ] [MAIN ] ACEL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:08.766708] [DEBUG] [LOCAL ] ACEL: Loaded 27028 existing records from primary storage. [2025-10-20T17:01:08.767145] [INFO ] [MAIN ] ACEL: Loaded 27028 existing records. [2025-10-20T17:01:08.970443] [WARN ] [MAIN ] ACEL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:08.970733] [INFO ] [MAIN ] ACEL: Wiping existing primary data before full refresh. [2025-10-20T17:01:08.970792] [INFO ] [LOCAL ] ACEL: Starting deletion of primary data. [2025-10-20T17:01:08.971001] [DEBUG] [LOCAL ] ACEL: Deleted primary data file. [2025-10-20T17:01:08.971044] [INFO ] [MAIN ] ACEL: Starting full primary refresh. [2025-10-20T17:01:09.056792] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-05. [2025-10-20T17:01:09.388599] [INFO ] [REMOTE ] ACEL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:09.851433] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-09. [2025-10-20T17:01:09.965309] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T17:01:09.973703] [INFO ] [REMOTE ] ABR: Full fetch complete (75420 records). [2025-10-20T17:01:09.973885] [DEBUG] [VALIDATOR ] ABR: Validating primary data. [2025-10-20T17:01:10.434832] [DEBUG] [VALIDATOR ] ABR: All expected trading days are present. [2025-10-20T17:01:10.434990] [DEBUG] [VALIDATOR ] ABR: Primary data validation passed. [2025-10-20T17:01:10.436130] [INFO ] [MAIN ] ABR: Fetched primary data is valid. [2025-10-20T17:01:10.436222] [DEBUG] [LOCAL ] ABR: Saving primary data. [2025-10-20T17:01:10.463798] [DEBUG] [LOCAL ] ABR: Trimmed 75420 records to 51949 (keeping 200 days of data) [2025-10-20T17:01:10.465471] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T17:01:10.672031] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-01. [2025-10-20T17:01:10.825366] [DEBUG] [LOCAL ] ABR: Primary data saved (51949 records). [2025-10-20T17:01:10.825566] [INFO ] [MAIN ] ABR: Full history saved. [2025-10-20T17:01:10.826174] [INFO ] [MAIN ] Update Progress: 15/2367 symbols completed. ABR took 23.2s. Estimated remaining: 2h 53m 35s. Rate: 13.5 symbols/min [2025-10-20T17:01:11.014860] [INFO ] [MAIN ] ACHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:11.073809] [DEBUG] [LOCAL ] ACHR: Loaded 54598 existing records from primary storage. [2025-10-20T17:01:11.074272] [INFO ] [MAIN ] ACHR: Loaded 54598 existing records. [2025-10-20T17:01:11.286981] [WARN ] [MAIN ] ACHR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:11.287272] [INFO ] [MAIN ] ACHR: Wiping existing primary data before full refresh. [2025-10-20T17:01:11.287322] [INFO ] [LOCAL ] ACHR: Starting deletion of primary data. [2025-10-20T17:01:11.287790] [DEBUG] [LOCAL ] ACHR: Deleted primary data file. [2025-10-20T17:01:11.287848] [INFO ] [MAIN ] ACHR: Starting full primary refresh. [2025-10-20T17:01:11.455414] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-06. [2025-10-20T17:01:11.712848] [INFO ] [REMOTE ] ACHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:11.811905] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T17:01:11.948194] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T17:01:12.251422] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-02. [2025-10-20T17:01:13.051425] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-10. [2025-10-20T17:01:13.190248] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:01:13.336701] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T17:01:13.533916] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T17:01:13.545005] [INFO ] [REMOTE ] ABT: Full fetch complete (77769 records). [2025-10-20T17:01:13.545208] [DEBUG] [VALIDATOR ] ABT: Validating primary data. [2025-10-20T17:01:13.859396] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-03. [2025-10-20T17:01:13.990939] [DEBUG] [VALIDATOR ] ABT: All expected trading days are present. [2025-10-20T17:01:13.991031] [DEBUG] [VALIDATOR ] ABT: Primary data validation passed. [2025-10-20T17:01:13.991752] [INFO ] [MAIN ] ABT: Fetched primary data is valid. [2025-10-20T17:01:13.991818] [DEBUG] [LOCAL ] ABT: Saving primary data. [2025-10-20T17:01:13.993587] [DEBUG] [LOCAL ] ABT: Trimmed 77769 records to 53622 (keeping 200 days of data) [2025-10-20T17:01:14.359356] [DEBUG] [LOCAL ] ABT: Primary data saved (53622 records). [2025-10-20T17:01:14.359513] [INFO ] [MAIN ] ABT: Full history saved. [2025-10-20T17:01:14.359587] [INFO ] [MAIN ] Update Progress: 16/2367 symbols completed. ABT took 24.4s. Estimated remaining: 2h 51m 19s. Rate: 13.7 symbols/min [2025-10-20T17:01:14.555107] [INFO ] [MAIN ] ACI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:14.613768] [DEBUG] [LOCAL ] ACI: Loaded 54062 existing records from primary storage. [2025-10-20T17:01:14.614167] [INFO ] [MAIN ] ACI: Loaded 54062 existing records. [2025-10-20T17:01:14.659396] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-07. [2025-10-20T17:01:14.822425] [WARN ] [MAIN ] ACI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:14.822708] [INFO ] [MAIN ] ACI: Wiping existing primary data before full refresh. [2025-10-20T17:01:14.822789] [INFO ] [LOCAL ] ACI: Starting deletion of primary data. [2025-10-20T17:01:14.823264] [DEBUG] [LOCAL ] ACI: Deleted primary data file. [2025-10-20T17:01:14.823321] [INFO ] [MAIN ] ACI: Starting full primary refresh. [2025-10-20T17:01:15.219975] [INFO ] [REMOTE ] ACI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:15.236984] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T17:01:15.451403] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-04. [2025-10-20T17:01:15.909283] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:01:16.251403] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-08. [2025-10-20T17:01:16.838000] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:01:17.051384] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-05. [2025-10-20T17:01:17.659577] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T17:01:17.851364] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-01. [2025-10-20T17:01:18.230612] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T17:01:18.651391] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-06. [2025-10-20T17:01:19.451424] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-09. [2025-10-20T17:01:19.623870] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T17:01:19.780231] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T17:01:20.251352] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-07. [2025-10-20T17:01:20.723710] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T17:01:21.051425] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-02. [2025-10-20T17:01:21.436040] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T17:01:21.851374] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-01. [2025-10-20T17:01:22.650132] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:01:22.659822] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-08. [2025-10-20T17:01:23.451375] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-10. [2025-10-20T17:01:24.010453] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:01:24.251351] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-03. [2025-10-20T17:01:24.560375] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-01 (2.71 seconds). [2025-10-20T17:01:24.799994] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T17:01:24.805457] [INFO ] [REMOTE ] ACCO: Full fetch complete (54045 records). [2025-10-20T17:01:24.805591] [DEBUG] [VALIDATOR ] ACCO: Validating primary data. [2025-10-20T17:01:25.056603] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-09. [2025-10-20T17:01:25.194381] [DEBUG] [VALIDATOR ] ACCO: All expected trading days are present. [2025-10-20T17:01:25.194471] [DEBUG] [VALIDATOR ] ACCO: Primary data validation passed. [2025-10-20T17:01:25.195018] [INFO ] [MAIN ] ACCO: Fetched primary data is valid. [2025-10-20T17:01:25.195081] [DEBUG] [LOCAL ] ACCO: Saving primary data. [2025-10-20T17:01:25.196941] [DEBUG] [LOCAL ] ACCO: Trimmed 54045 records to 36838 (keeping 200 days of data) [2025-10-20T17:01:25.410380] [DEBUG] [LOCAL ] ACCO: Primary data saved (36838 records). [2025-10-20T17:01:25.410561] [INFO ] [MAIN ] ACCO: Full history saved. [2025-10-20T17:01:25.410679] [INFO ] [MAIN ] Update Progress: 17/2367 symbols completed. ACCO took 25.4s. Estimated remaining: 3h 6m 38s. Rate: 12.6 symbols/min [2025-10-20T17:01:25.589811] [INFO ] [MAIN ] ACM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:25.632410] [DEBUG] [LOCAL ] ACM: Loaded 43560 existing records from primary storage. [2025-10-20T17:01:25.632771] [INFO ] [MAIN ] ACM: Loaded 43560 existing records. [2025-10-20T17:01:25.817895] [WARN ] [MAIN ] ACM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:25.818226] [INFO ] [MAIN ] ACM: Wiping existing primary data before full refresh. [2025-10-20T17:01:25.818301] [INFO ] [LOCAL ] ACM: Starting deletion of primary data. [2025-10-20T17:01:25.818751] [DEBUG] [LOCAL ] ACM: Deleted primary data file. [2025-10-20T17:01:25.818809] [INFO ] [MAIN ] ACM: Starting full primary refresh. [2025-10-20T17:01:25.862567] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-02. [2025-10-20T17:01:26.226795] [INFO ] [REMOTE ] ACM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:26.260553] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T17:01:26.380759] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T17:01:26.651372] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-01. [2025-10-20T17:01:27.451389] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-10. [2025-10-20T17:01:28.251406] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-04. [2025-10-20T17:01:28.447022] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-02 (2.58 seconds). [2025-10-20T17:01:28.487150] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T17:01:28.629799] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T17:01:28.635116] [INFO ] [REMOTE ] ACEL: Full fetch complete (38833 records). [2025-10-20T17:01:28.635304] [DEBUG] [VALIDATOR ] ACEL: Validating primary data. [2025-10-20T17:01:29.050751] [DEBUG] [VALIDATOR ] ACEL: All expected trading days are present. [2025-10-20T17:01:29.051076] [DEBUG] [VALIDATOR ] ACEL: Primary data validation passed. [2025-10-20T17:01:29.051534] [INFO ] [MAIN ] ACEL: Fetched primary data is valid. [2025-10-20T17:01:29.051599] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-02. [2025-10-20T17:01:29.052796] [DEBUG] [LOCAL ] ACEL: Saving primary data. [2025-10-20T17:01:29.053776] [DEBUG] [LOCAL ] ACEL: Trimmed 38833 records to 26577 (keeping 200 days of data) [2025-10-20T17:01:29.220626] [DEBUG] [LOCAL ] ACEL: Primary data saved (26577 records). [2025-10-20T17:01:29.220814] [INFO ] [MAIN ] ACEL: Full history saved. [2025-10-20T17:01:29.220898] [INFO ] [MAIN ] Update Progress: 18/2367 symbols completed. ACEL took 20.7s. Estimated remaining: 3h 4m 28s. Rate: 12.7 symbols/min [2025-10-20T17:01:29.407239] [INFO ] [MAIN ] ACN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:29.460497] [DEBUG] [LOCAL ] ACN: Loaded 54216 existing records from primary storage. [2025-10-20T17:01:29.460942] [INFO ] [MAIN ] ACN: Loaded 54216 existing records. [2025-10-20T17:01:29.679128] [WARN ] [MAIN ] ACN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:29.679372] [INFO ] [MAIN ] ACN: Wiping existing primary data before full refresh. [2025-10-20T17:01:29.679421] [INFO ] [LOCAL ] ACN: Starting deletion of primary data. [2025-10-20T17:01:29.679967] [DEBUG] [LOCAL ] ACN: Deleted primary data file. [2025-10-20T17:01:29.680028] [INFO ] [MAIN ] ACN: Starting full primary refresh. [2025-10-20T17:01:29.859808] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-03. [2025-10-20T17:01:30.096588] [INFO ] [REMOTE ] ACN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:30.132901] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T17:01:30.307213] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T17:01:30.651502] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-01. [2025-10-20T17:01:31.451379] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-03. [2025-10-20T17:01:32.251354] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-05. [2025-10-20T17:01:32.440497] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:01:32.665613] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-03 (2.81 seconds). [2025-10-20T17:01:33.051353] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-02. [2025-10-20T17:01:33.087592] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T17:01:33.851359] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-04. [2025-10-20T17:01:34.061693] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:01:34.651368] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-04. [2025-10-20T17:01:34.824228] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T17:01:35.227510] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:01:35.451374] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-03. [2025-10-20T17:01:36.251424] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-06. [2025-10-20T17:01:37.051369] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-05. [2025-10-20T17:01:37.402686] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-04 (2.75 seconds). [2025-10-20T17:01:37.492291] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-03 (2.04 seconds). [2025-10-20T17:01:37.851358] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-05. [2025-10-20T17:01:37.993816] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T17:01:38.661725] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-04. [2025-10-20T17:01:38.665010] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:01:39.451396] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-07. [2025-10-20T17:01:40.251419] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-06. [2025-10-20T17:01:40.436615] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-05 (2.59 seconds). [2025-10-20T17:01:40.743757] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T17:01:41.051416] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-06. [2025-10-20T17:01:41.429618] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T17:01:41.743500] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T17:01:41.851438] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-05. [2025-10-20T17:01:42.651398] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-07. [2025-10-20T17:01:43.439834] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-06 (2.39 seconds). [2025-10-20T17:01:43.451336] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-07. [2025-10-20T17:01:43.758922] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T17:01:44.251348] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-06. [2025-10-20T17:01:44.348756] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T17:01:45.051466] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-08. [2025-10-20T17:01:45.851432] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-08. [2025-10-20T17:01:45.933040] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T17:01:46.326571] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-07 (2.88 seconds). [2025-10-20T17:01:46.581926] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:01:46.651597] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-09. [2025-10-20T17:01:47.383939] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:01:47.451436] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-08. [2025-10-20T17:01:48.116936] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T17:01:48.251414] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-07. [2025-10-20T17:01:49.051377] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-09. [2025-10-20T17:01:49.851400] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-10. [2025-10-20T17:01:49.932225] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-08 (2.48 seconds). [2025-10-20T17:01:50.496226] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-07 (2.24 seconds). [2025-10-20T17:01:50.651352] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-08. [2025-10-20T17:01:50.793859] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:01:51.451389] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-10. [2025-10-20T17:01:51.958555] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-10 (2.11 seconds). [2025-10-20T17:01:51.966510] [INFO ] [REMOTE ] ACI: Full fetch complete (76843 records). [2025-10-20T17:01:51.966711] [DEBUG] [VALIDATOR ] ACI: Validating primary data. [2025-10-20T17:01:51.967586] [FATAL] [VALIDATOR ] ACI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:01:51.967702] [FATAL] [MAIN ] ACI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:01:51.967752] [WARN ] [MAIN ] ACI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:01:51.967820] [INFO ] [LOCAL ] ACI: Starting deletion of primary data. [2025-10-20T17:01:51.967916] [DEBUG] [LOCAL ] ACI: No primary data file found to delete. [2025-10-20T17:01:51.967979] [INFO ] [LOCAL ] ACI: Starting deletion of derivative data. [2025-10-20T17:01:51.968159] [DEBUG] [LOCAL ] ACI: Deleted derivative data file. [2025-10-20T17:01:51.970074] [INFO ] [MAIN ] Update Progress: 19/2367 symbols completed. ACI took 37.6s. Estimated remaining: 3h 41m 33s. Rate: 10.6 symbols/min [2025-10-20T17:01:52.165402] [INFO ] [MAIN ] ACP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:52.166534] [DEBUG] [LOCAL ] ACP: No existing data file found in primary storage. [2025-10-20T17:01:52.166587] [WARN ] [MAIN ] ACP: No existing data found, performing full refresh instead. [2025-10-20T17:01:52.166736] [INFO ] [MAIN ] ACP: Wiping existing primary data before full refresh. [2025-10-20T17:01:52.166783] [INFO ] [LOCAL ] ACP: Starting deletion of primary data. [2025-10-20T17:01:52.166827] [DEBUG] [LOCAL ] ACP: No primary data file found to delete. [2025-10-20T17:01:52.166860] [INFO ] [MAIN ] ACP: Starting full primary refresh. [2025-10-20T17:01:52.258763] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-09. [2025-10-20T17:01:52.546065] [INFO ] [REMOTE ] ACP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:52.613987] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T17:01:52.856917] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:01:52.863500] [INFO ] [REMOTE ] ACM: Full fetch complete (63289 records). [2025-10-20T17:01:52.863665] [DEBUG] [VALIDATOR ] ACM: Validating primary data. [2025-10-20T17:01:53.052883] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-09. [2025-10-20T17:01:53.271862] [DEBUG] [VALIDATOR ] ACM: All expected trading days are present. [2025-10-20T17:01:53.271949] [DEBUG] [VALIDATOR ] ACM: Primary data validation passed. [2025-10-20T17:01:53.272578] [INFO ] [MAIN ] ACM: Fetched primary data is valid. [2025-10-20T17:01:53.272622] [DEBUG] [LOCAL ] ACM: Saving primary data. [2025-10-20T17:01:53.273961] [DEBUG] [LOCAL ] ACM: Trimmed 63289 records to 42890 (keeping 200 days of data) [2025-10-20T17:01:53.548779] [DEBUG] [LOCAL ] ACM: Primary data saved (42890 records). [2025-10-20T17:01:53.548919] [INFO ] [MAIN ] ACM: Full history saved. [2025-10-20T17:01:53.550278] [INFO ] [MAIN ] Update Progress: 20/2367 symbols completed. ACM took 28.1s. Estimated remaining: 3h 33m 28s. Rate: 11.0 symbols/min [2025-10-20T17:01:53.727462] [INFO ] [MAIN ] ACR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:53.736496] [DEBUG] [LOCAL ] ACR: Loaded 5512 existing records from primary storage. [2025-10-20T17:01:53.736754] [INFO ] [MAIN ] ACR: Loaded 5512 existing records. [2025-10-20T17:01:53.856513] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-01. [2025-10-20T17:01:53.929308] [WARN ] [MAIN ] ACR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:53.929511] [INFO ] [MAIN ] ACR: Wiping existing primary data before full refresh. [2025-10-20T17:01:53.929561] [INFO ] [LOCAL ] ACR: Starting deletion of primary data. [2025-10-20T17:01:53.929727] [DEBUG] [LOCAL ] ACR: Deleted primary data file. [2025-10-20T17:01:53.929785] [INFO ] [MAIN ] ACR: Starting full primary refresh. [2025-10-20T17:01:54.311698] [INFO ] [REMOTE ] ACR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:54.651332] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-01. [2025-10-20T17:01:55.054838] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-09 (2.80 seconds). [2025-10-20T17:01:55.141144] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-10-20T17:01:55.451414] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-10. [2025-10-20T17:01:55.504992] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T17:01:55.504848] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:01:56.251438] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-10. [2025-10-20T17:01:57.051444] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-02. [2025-10-20T17:01:57.851380] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-02. [2025-10-20T17:01:58.269484] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-10 (2.82 seconds). [2025-10-20T17:01:58.278112] [INFO ] [REMOTE ] ACHR: Full fetch complete (77998 records). [2025-10-20T17:01:58.278273] [DEBUG] [VALIDATOR ] ACHR: Validating primary data. [2025-10-20T17:01:58.577355] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T17:01:58.661186] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-03. [2025-10-20T17:01:58.647940] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:01:58.732123] [DEBUG] [VALIDATOR ] ACHR: All expected trading days are present. [2025-10-20T17:01:58.732473] [DEBUG] [VALIDATOR ] ACHR: Primary data validation passed. [2025-10-20T17:01:58.733334] [INFO ] [MAIN ] ACHR: Fetched primary data is valid. [2025-10-20T17:01:58.733414] [DEBUG] [LOCAL ] ACHR: Saving primary data. [2025-10-20T17:01:58.734992] [DEBUG] [LOCAL ] ACHR: Trimmed 77998 records to 53818 (keeping 200 days of data) [2025-10-20T17:01:58.842544] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-10 (2.59 seconds). [2025-10-20T17:01:58.863036] [INFO ] [REMOTE ] ACN: Full fetch complete (77401 records). [2025-10-20T17:01:58.864693] [DEBUG] [VALIDATOR ] ACN: Validating primary data. [2025-10-20T17:01:59.463709] [DEBUG] [VALIDATOR ] ACN: All expected trading days are present. [2025-10-20T17:01:59.458944] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-03. [2025-10-20T17:01:59.463834] [DEBUG] [VALIDATOR ] ACN: Primary data validation passed. [2025-10-20T17:01:59.475105] [INFO ] [MAIN ] ACN: Fetched primary data is valid. [2025-10-20T17:01:59.479310] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T17:01:59.483716] [DEBUG] [LOCAL ] ACN: Saving primary data. [2025-10-20T17:01:59.491549] [DEBUG] [LOCAL ] ACN: Trimmed 77401 records to 53440 (keeping 200 days of data) [2025-10-20T17:01:59.563831] [DEBUG] [LOCAL ] ACHR: Primary data saved (53818 records). [2025-10-20T17:01:59.594094] [INFO ] [MAIN ] ACHR: Full history saved. [2025-10-20T17:01:59.603034] [INFO ] [MAIN ] Update Progress: 21/2367 symbols completed. ACHR took 48.8s. Estimated remaining: 3h 34m 29s. Rate: 10.9 symbols/min [2025-10-20T17:01:59.874812] [INFO ] [MAIN ] ACRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:00.042087] [DEBUG] [LOCAL ] ACRE: Loaded 34100 existing records from primary storage. [2025-10-20T17:02:00.047743] [INFO ] [MAIN ] ACRE: Loaded 34100 existing records. [2025-10-20T17:02:00.159608] [DEBUG] [LOCAL ] ACN: Primary data saved (53440 records). [2025-10-20T17:02:00.162234] [INFO ] [MAIN ] ACN: Full history saved. [2025-10-20T17:02:00.163053] [INFO ] [MAIN ] Update Progress: 22/2367 symbols completed. ACN took 30.9s. Estimated remaining: 3h 25m 39s. Rate: 11.4 symbols/min [2025-10-20T17:02:00.265842] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-04. [2025-10-20T17:02:00.320405] [WARN ] [MAIN ] ACRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:00.350479] [INFO ] [MAIN ] ACRE: Wiping existing primary data before full refresh. [2025-10-20T17:02:00.366956] [INFO ] [LOCAL ] ACRE: Starting deletion of primary data. [2025-10-20T17:02:00.425691] [DEBUG] [LOCAL ] ACRE: Deleted primary data file. [2025-10-20T17:02:00.425615] [INFO ] [MAIN ] ACV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:00.425940] [INFO ] [MAIN ] ACRE: Starting full primary refresh. [2025-10-20T17:02:00.553282] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:02:00.563938] [DEBUG] [LOCAL ] ACV: Loaded 7718 existing records from primary storage. [2025-10-20T17:02:00.575810] [INFO ] [MAIN ] ACV: Loaded 7718 existing records. [2025-10-20T17:02:00.890831] [WARN ] [MAIN ] ACV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:00.908453] [INFO ] [MAIN ] ACV: Wiping existing primary data before full refresh. [2025-10-20T17:02:00.928855] [INFO ] [LOCAL ] ACV: Starting deletion of primary data. [2025-10-20T17:02:00.953379] [DEBUG] [LOCAL ] ACV: Deleted primary data file. [2025-10-20T17:02:00.986581] [INFO ] [MAIN ] ACV: Starting full primary refresh. [2025-10-20T17:02:01.057701] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-04. [2025-10-20T17:02:01.073714] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T17:02:01.245418] [INFO ] [REMOTE ] ACRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:01.519619] [INFO ] [REMOTE ] ACV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:01.851356] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-01. [2025-10-20T17:02:02.636395] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:02:02.651402] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-05. [2025-10-20T17:02:03.007876] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T17:02:03.351261] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:02:03.451404] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-06. [2025-10-20T17:02:03.872501] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T17:02:04.251409] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-07. [2025-10-20T17:02:04.720878] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T17:02:05.051369] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-08. [2025-10-20T17:02:05.564021] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-10-20T17:02:05.851365] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-09. [2025-10-20T17:02:06.233270] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-10-20T17:02:06.651385] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-05. [2025-10-20T17:02:07.451449] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-10. [2025-10-20T17:02:07.899792] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T17:02:08.018491] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T17:02:08.020122] [INFO ] [REMOTE ] ACR: Full fetch complete (8534 records). [2025-10-20T17:02:08.020220] [DEBUG] [VALIDATOR ] ACR: Validating primary data. [2025-10-20T17:02:08.020811] [FATAL] [VALIDATOR ] ACR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:02:08.021064] [FATAL] [MAIN ] ACR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:02:08.021120] [WARN ] [MAIN ] ACR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:02:08.021202] [INFO ] [LOCAL ] ACR: Starting deletion of primary data. [2025-10-20T17:02:08.021283] [DEBUG] [LOCAL ] ACR: No primary data file found to delete. [2025-10-20T17:02:08.021332] [INFO ] [LOCAL ] ACR: Starting deletion of derivative data. [2025-10-20T17:02:08.021498] [DEBUG] [LOCAL ] ACR: Deleted derivative data file. [2025-10-20T17:02:08.021611] [INFO ] [MAIN ] Update Progress: 23/2367 symbols completed. ACR took 14.5s. Estimated remaining: 3h 29m 58s. Rate: 11.2 symbols/min [2025-10-20T17:02:08.197998] [INFO ] [MAIN ] ACVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:08.198176] [DEBUG] [LOCAL ] ACVA: No existing data file found in primary storage. [2025-10-20T17:02:08.198213] [WARN ] [MAIN ] ACVA: No existing data found, performing full refresh instead. [2025-10-20T17:02:08.198319] [INFO ] [MAIN ] ACVA: Wiping existing primary data before full refresh. [2025-10-20T17:02:08.198363] [INFO ] [LOCAL ] ACVA: Starting deletion of primary data. [2025-10-20T17:02:08.198450] [DEBUG] [LOCAL ] ACVA: No primary data file found to delete. [2025-10-20T17:02:08.198490] [INFO ] [MAIN ] ACVA: Starting full primary refresh. [2025-10-20T17:02:08.259244] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-06. [2025-10-20T17:02:08.558750] [INFO ] [REMOTE ] ACVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:09.051415] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-01. [2025-10-20T17:02:09.270207] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:02:09.500370] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T17:02:09.851381] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-02. [2025-10-20T17:02:10.352011] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T17:02:10.651384] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-07. [2025-10-20T17:02:11.451404] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-01. [2025-10-20T17:02:11.599091] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:02:12.251382] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-08. [2025-10-20T17:02:12.837311] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:02:13.051358] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-02. [2025-10-20T17:02:13.508216] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:02:13.851384] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-02. [2025-10-20T17:02:14.495487] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T17:02:14.651383] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-09. [2025-10-20T17:02:15.031273] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T17:02:15.451404] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-03. [2025-10-20T17:02:15.771494] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T17:02:16.251375] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-03. [2025-10-20T17:02:17.002567] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:02:17.051378] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-04. [2025-10-20T17:02:17.671816] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:02:17.851387] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-04. [2025-10-20T17:02:18.651355] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-03. [2025-10-20T17:02:18.843078] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:02:19.319519] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:02:19.404861] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T17:02:19.451338] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-04. [2025-10-20T17:02:20.251417] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-10. [2025-10-20T17:02:20.355277] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T17:02:21.051381] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-05. [2025-10-20T17:02:21.665125] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:02:21.670400] [INFO ] [REMOTE ] ACP: Full fetch complete (46459 records). [2025-10-20T17:02:21.670587] [DEBUG] [VALIDATOR ] ACP: Validating primary data. [2025-10-20T17:02:21.857117] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-05. [2025-10-20T17:02:22.065230] [DEBUG] [VALIDATOR ] ACP: All expected trading days are present. [2025-10-20T17:02:22.065318] [DEBUG] [VALIDATOR ] ACP: Primary data validation passed. [2025-10-20T17:02:22.065812] [INFO ] [MAIN ] ACP: Fetched primary data is valid. [2025-10-20T17:02:22.065873] [DEBUG] [LOCAL ] ACP: Saving primary data. [2025-10-20T17:02:22.066887] [DEBUG] [LOCAL ] ACP: Trimmed 46459 records to 31109 (keeping 200 days of data) [2025-10-20T17:02:22.237721] [DEBUG] [LOCAL ] ACP: Primary data saved (31109 records). [2025-10-20T17:02:22.237859] [INFO ] [MAIN ] ACP: Full history saved. [2025-10-20T17:02:22.237932] [INFO ] [MAIN ] Update Progress: 24/2367 symbols completed. ACP took 30.3s. Estimated remaining: 3h 44m 16s. Rate: 10.4 symbols/min [2025-10-20T17:02:22.415198] [INFO ] [MAIN ] AD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:22.415408] [DEBUG] [LOCAL ] AD: No existing data file found in primary storage. [2025-10-20T17:02:22.415451] [WARN ] [MAIN ] AD: No existing data found, performing full refresh instead. [2025-10-20T17:02:22.415563] [INFO ] [MAIN ] AD: Wiping existing primary data before full refresh. [2025-10-20T17:02:22.415604] [INFO ] [LOCAL ] AD: Starting deletion of primary data. [2025-10-20T17:02:22.415669] [DEBUG] [LOCAL ] AD: No primary data file found to delete. [2025-10-20T17:02:22.415724] [INFO ] [MAIN ] AD: Starting full primary refresh. [2025-10-20T17:02:22.673422] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-05. [2025-10-20T17:02:22.690493] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T17:02:22.823313] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:02:22.821314] [INFO ] [REMOTE ] AD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:23.451408] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-06. [2025-10-20T17:02:24.079866] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T17:02:24.235928] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:02:24.251366] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-06. [2025-10-20T17:02:25.051387] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-06. [2025-10-20T17:02:25.690830] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T17:02:25.851360] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-07. [2025-10-20T17:02:26.493859] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T17:02:26.651408] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-01. [2025-10-20T17:02:26.862071] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:02:26.862387] [WARN ] [REMOTE ] AD: No data returned for month 2025-01 [2025-10-20T17:02:26.862473] [FATAL] [REMOTE ] AD: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:02:26.862548] [FATAL] [MAIN ] AD: Failed to fetch primary data. [2025-10-20T17:02:26.862632] [WARN ] [MAIN ] AD: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:02:26.862729] [INFO ] [LOCAL ] AD: Starting deletion of primary data. [2025-10-20T17:02:26.862813] [DEBUG] [LOCAL ] AD: No primary data file found to delete. [2025-10-20T17:02:26.862851] [INFO ] [LOCAL ] AD: Starting deletion of derivative data. [2025-10-20T17:02:26.862891] [DEBUG] [LOCAL ] AD: No derivative data file found to delete. [2025-10-20T17:02:26.862937] [INFO ] [MAIN ] Update Progress: 25/2367 symbols completed. AD took 4.6s. Estimated remaining: 3h 42m 25s. Rate: 10.5 symbols/min [2025-10-20T17:02:27.039542] [INFO ] [MAIN ] ADC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:27.081620] [DEBUG] [LOCAL ] ADC: Loaded 47056 existing records from primary storage. [2025-10-20T17:02:27.081951] [INFO ] [MAIN ] ADC: Loaded 47056 existing records. [2025-10-20T17:02:27.274434] [WARN ] [MAIN ] ADC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:27.274899] [INFO ] [MAIN ] ADC: Wiping existing primary data before full refresh. [2025-10-20T17:02:27.275001] [INFO ] [LOCAL ] ADC: Starting deletion of primary data. [2025-10-20T17:02:27.275280] [DEBUG] [LOCAL ] ADC: Deleted primary data file. [2025-10-20T17:02:27.275338] [INFO ] [MAIN ] ADC: Starting full primary refresh. [2025-10-20T17:02:27.454010] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-07. [2025-10-20T17:02:27.661667] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T17:02:27.693528] [INFO ] [REMOTE ] ADC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:28.239636] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T17:02:28.251322] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-08. [2025-10-20T17:02:29.027470] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T17:02:29.051436] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-08. [2025-10-20T17:02:29.851390] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-09. [2025-10-20T17:02:30.493768] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T17:02:30.651405] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-09. [2025-10-20T17:02:30.675947] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T17:02:31.451377] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-10. [2025-10-20T17:02:32.251406] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-07. [2025-10-20T17:02:32.359256] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T17:02:32.361700] [INFO ] [REMOTE ] ACV: Full fetch complete (10342 records). [2025-10-20T17:02:32.361799] [DEBUG] [VALIDATOR ] ACV: Validating primary data. [2025-10-20T17:02:32.362224] [FATAL] [VALIDATOR ] ACV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:02:32.362314] [FATAL] [MAIN ] ACV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:02:32.362425] [WARN ] [MAIN ] ACV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:02:32.362723] [INFO ] [LOCAL ] ACV: Starting deletion of primary data. [2025-10-20T17:02:32.362813] [DEBUG] [LOCAL ] ACV: No primary data file found to delete. [2025-10-20T17:02:32.362859] [INFO ] [LOCAL ] ACV: Starting deletion of derivative data. [2025-10-20T17:02:32.362996] [DEBUG] [LOCAL ] ACV: Deleted derivative data file. [2025-10-20T17:02:32.363070] [INFO ] [MAIN ] Update Progress: 26/2367 symbols completed. ACV took 32.2s. Estimated remaining: 3h 42m 2s. Rate: 10.5 symbols/min [2025-10-20T17:02:32.598925] [INFO ] [MAIN ] ADCT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:32.599116] [DEBUG] [LOCAL ] ADCT: No existing data file found in primary storage. [2025-10-20T17:02:32.599339] [WARN ] [MAIN ] ADCT: No existing data found, performing full refresh instead. [2025-10-20T17:02:32.600900] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T17:02:32.600993] [INFO ] [MAIN ] ADCT: Wiping existing primary data before full refresh. [2025-10-20T17:02:32.610938] [INFO ] [LOCAL ] ADCT: Starting deletion of primary data. [2025-10-20T17:02:32.611100] [DEBUG] [LOCAL ] ADCT: No primary data file found to delete. [2025-10-20T17:02:32.611153] [INFO ] [MAIN ] ADCT: Starting full primary refresh. [2025-10-20T17:02:32.982165] [INFO ] [REMOTE ] ADCT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:33.051586] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-10. [2025-10-20T17:02:33.505792] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:02:33.851410] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-01. [2025-10-20T17:02:34.651388] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-01. [2025-10-20T17:02:34.995542] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T17:02:35.002957] [INFO ] [REMOTE ] ACVA: Full fetch complete (70994 records). [2025-10-20T17:02:35.003073] [DEBUG] [VALIDATOR ] ACVA: Validating primary data. [2025-10-20T17:02:35.363096] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T17:02:35.425161] [DEBUG] [VALIDATOR ] ACVA: All expected trading days are present. [2025-10-20T17:02:35.425231] [DEBUG] [VALIDATOR ] ACVA: Primary data validation passed. [2025-10-20T17:02:35.425904] [INFO ] [MAIN ] ACVA: Fetched primary data is valid. [2025-10-20T17:02:35.425967] [DEBUG] [LOCAL ] ACVA: Saving primary data. [2025-10-20T17:02:35.427599] [DEBUG] [LOCAL ] ACVA: Trimmed 70994 records to 49252 (keeping 200 days of data) [2025-10-20T17:02:35.474353] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-02. [2025-10-20T17:02:35.714464] [DEBUG] [LOCAL ] ACVA: Primary data saved (49252 records). [2025-10-20T17:02:35.714618] [INFO ] [MAIN ] ACVA: Full history saved. [2025-10-20T17:02:35.714726] [INFO ] [MAIN ] Update Progress: 27/2367 symbols completed. ACVA took 27.7s. Estimated remaining: 3h 38m 34s. Rate: 10.7 symbols/min [2025-10-20T17:02:35.893778] [INFO ] [MAIN ] ADM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:35.946279] [DEBUG] [LOCAL ] ADM: Loaded 54178 existing records from primary storage. [2025-10-20T17:02:35.946833] [INFO ] [MAIN ] ADM: Loaded 54178 existing records. [2025-10-20T17:02:36.133839] [WARN ] [MAIN ] ADM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:36.134614] [INFO ] [MAIN ] ADM: Wiping existing primary data before full refresh. [2025-10-20T17:02:36.134730] [INFO ] [LOCAL ] ADM: Starting deletion of primary data. [2025-10-20T17:02:36.135006] [DEBUG] [LOCAL ] ADM: Deleted primary data file. [2025-10-20T17:02:36.135067] [INFO ] [MAIN ] ADM: Starting full primary refresh. [2025-10-20T17:02:36.254582] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-08. [2025-10-20T17:02:36.497637] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T17:02:36.538076] [INFO ] [REMOTE ] ADM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:36.589783] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:02:37.051445] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-01. [2025-10-20T17:02:37.637081] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T17:02:37.851404] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-02. [2025-10-20T17:02:38.651403] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-03. [2025-10-20T17:02:38.997295] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T17:02:39.248207] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T17:02:39.451389] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-03. [2025-10-20T17:02:39.530670] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T17:02:40.251440] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-09. [2025-10-20T17:02:41.051386] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-04. [2025-10-20T17:02:41.091698] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T17:02:41.182383] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T17:02:41.851899] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T17:02:41.851971] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-10. [2025-10-20T17:02:42.651407] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-05. [2025-10-20T17:02:43.103412] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T17:02:43.108205] [INFO ] [REMOTE ] ACRE: Full fetch complete (49468 records). [2025-10-20T17:02:43.108350] [DEBUG] [VALIDATOR ] ACRE: Validating primary data. [2025-10-20T17:02:43.109023] [FATAL] [VALIDATOR ] ACRE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:02:43.109108] [FATAL] [MAIN ] ACRE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:02:43.109153] [WARN ] [MAIN ] ACRE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:02:43.109233] [INFO ] [LOCAL ] ACRE: Starting deletion of primary data. [2025-10-20T17:02:43.109312] [DEBUG] [LOCAL ] ACRE: No primary data file found to delete. [2025-10-20T17:02:43.109360] [INFO ] [LOCAL ] ACRE: Starting deletion of derivative data. [2025-10-20T17:02:43.109576] [DEBUG] [LOCAL ] ACRE: Deleted derivative data file. [2025-10-20T17:02:43.110227] [INFO ] [MAIN ] Update Progress: 28/2367 symbols completed. ACRE took 43.5s. Estimated remaining: 3h 40m 58s. Rate: 10.6 symbols/min [2025-10-20T17:02:43.284805] [INFO ] [MAIN ] ADNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:43.329255] [DEBUG] [LOCAL ] ADNT: Loaded 47379 existing records from primary storage. [2025-10-20T17:02:43.329580] [INFO ] [MAIN ] ADNT: Loaded 47379 existing records. [2025-10-20T17:02:43.456558] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-02. [2025-10-20T17:02:43.522754] [WARN ] [MAIN ] ADNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:43.523340] [INFO ] [MAIN ] ADNT: Wiping existing primary data before full refresh. [2025-10-20T17:02:43.523404] [INFO ] [LOCAL ] ADNT: Starting deletion of primary data. [2025-10-20T17:02:43.523833] [DEBUG] [LOCAL ] ADNT: Deleted primary data file. [2025-10-20T17:02:43.523892] [INFO ] [MAIN ] ADNT: Starting full primary refresh. [2025-10-20T17:02:43.876590] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T17:02:43.952605] [INFO ] [REMOTE ] ADNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:44.251464] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-01. [2025-10-20T17:02:44.707611] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:02:45.051393] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-06. [2025-10-20T17:02:45.851395] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-03. [2025-10-20T17:02:45.989855] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T17:02:46.651393] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-02. [2025-10-20T17:02:46.877888] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T17:02:47.356216] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T17:02:47.451372] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-07. [2025-10-20T17:02:47.810937] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:02:48.251379] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-03. [2025-10-20T17:02:48.837508] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:02:49.051461] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-08. [2025-10-20T17:02:49.788389] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:02:49.851401] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-04. [2025-10-20T17:02:50.311784] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:02:50.651437] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-04. [2025-10-20T17:02:51.451366] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-04. [2025-10-20T17:02:51.783949] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T17:02:51.916908] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T17:02:52.251377] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-05. [2025-10-20T17:02:53.033617] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:02:53.051333] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-05. [2025-10-20T17:02:53.851372] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-09. [2025-10-20T17:02:54.651356] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-05. [2025-10-20T17:02:54.823980] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-05 (2.57 seconds). [2025-10-20T17:02:55.009529] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T17:02:55.451379] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-06. [2025-10-20T17:02:55.917141] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T17:02:56.047637] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:02:56.251397] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-10. [2025-10-20T17:02:56.880198] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T17:02:57.051454] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-07. [2025-10-20T17:02:57.842730] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T17:02:57.849314] [INFO ] [REMOTE ] ADCT: Full fetch complete (51212 records). [2025-10-20T17:02:57.849456] [DEBUG] [VALIDATOR ] ADCT: Validating primary data. [2025-10-20T17:02:57.850214] [FATAL] [VALIDATOR ] ADCT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:02:57.850311] [FATAL] [MAIN ] ADCT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:02:57.850363] [WARN ] [MAIN ] ADCT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:02:57.850442] [INFO ] [LOCAL ] ADCT: Starting deletion of primary data. [2025-10-20T17:02:57.850761] [DEBUG] [LOCAL ] ADCT: No primary data file found to delete. [2025-10-20T17:02:57.851017] [INFO ] [LOCAL ] ADCT: Starting deletion of derivative data. [2025-10-20T17:02:57.851084] [DEBUG] [LOCAL ] ADCT: No derivative data file found to delete. [2025-10-20T17:02:57.851192] [INFO ] [MAIN ] Update Progress: 29/2367 symbols completed. ADCT took 25.5s. Estimated remaining: 3h 53m 3s. Rate: 10.0 symbols/min [2025-10-20T17:02:57.851390] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-06. [2025-10-20T17:02:58.024261] [INFO ] [MAIN ] ADT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:58.073040] [DEBUG] [LOCAL ] ADT: Loaded 54217 existing records from primary storage. [2025-10-20T17:02:58.073383] [INFO ] [MAIN ] ADT: Loaded 54217 existing records. [2025-10-20T17:02:58.248251] [WARN ] [MAIN ] ADT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:58.248459] [INFO ] [MAIN ] ADT: Wiping existing primary data before full refresh. [2025-10-20T17:02:58.248547] [INFO ] [LOCAL ] ADT: Starting deletion of primary data. [2025-10-20T17:02:58.249413] [DEBUG] [LOCAL ] ADT: Deleted primary data file. [2025-10-20T17:02:58.249470] [INFO ] [MAIN ] ADT: Starting full primary refresh. [2025-10-20T17:02:58.646159] [INFO ] [REMOTE ] ADT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:58.658899] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-06. [2025-10-20T17:02:58.681302] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T17:02:59.044033] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:02:59.451468] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-07. [2025-10-20T17:03:00.251359] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-01. [2025-10-20T17:03:00.434979] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:03:00.805779] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T17:03:01.051356] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-08. [2025-10-20T17:03:01.670079] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:03:01.851360] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-07. [2025-10-20T17:03:02.651426] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-08. [2025-10-20T17:03:02.696400] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:03:03.451385] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-09. [2025-10-20T17:03:04.056017] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:03:04.102215] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-10-20T17:03:04.251403] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-09. [2025-10-20T17:03:04.826917] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:03:05.051433] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-08. [2025-10-20T17:03:05.771559] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T17:03:05.851393] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-10. [2025-10-20T17:03:06.651417] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-02. [2025-10-20T17:03:07.451477] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-10. [2025-10-20T17:03:07.517705] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-08 (2.47 seconds). [2025-10-20T17:03:07.706433] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T17:03:07.712936] [INFO ] [REMOTE ] ADNT: Full fetch complete (67975 records). [2025-10-20T17:03:07.713119] [DEBUG] [VALIDATOR ] ADNT: Validating primary data. [2025-10-20T17:03:08.169069] [DEBUG] [VALIDATOR ] ADNT: All expected trading days are present. [2025-10-20T17:03:08.169191] [DEBUG] [VALIDATOR ] ADNT: Primary data validation passed. [2025-10-20T17:03:08.169987] [INFO ] [MAIN ] ADNT: Fetched primary data is valid. [2025-10-20T17:03:08.170070] [DEBUG] [LOCAL ] ADNT: Saving primary data. [2025-10-20T17:03:08.171752] [DEBUG] [LOCAL ] ADNT: Trimmed 67975 records to 46530 (keeping 200 days of data) [2025-10-20T17:03:08.252908] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-09. [2025-10-20T17:03:08.295559] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T17:03:08.520509] [DEBUG] [LOCAL ] ADNT: Primary data saved (46530 records). [2025-10-20T17:03:08.520691] [INFO ] [MAIN ] ADNT: Full history saved. [2025-10-20T17:03:08.521784] [INFO ] [MAIN ] Update Progress: 30/2367 symbols completed. ADNT took 25.4s. Estimated remaining: 3h 59m 3s. Rate: 9.8 symbols/min [2025-10-20T17:03:08.728515] [INFO ] [MAIN ] ADX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:08.763756] [DEBUG] [LOCAL ] ADX: Loaded 28160 existing records from primary storage. [2025-10-20T17:03:08.764217] [INFO ] [MAIN ] ADX: Loaded 28160 existing records. [2025-10-20T17:03:08.980032] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T17:03:09.014479] [WARN ] [MAIN ] ADX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:09.015148] [INFO ] [MAIN ] ADX: Wiping existing primary data before full refresh. [2025-10-20T17:03:09.015216] [INFO ] [LOCAL ] ADX: Starting deletion of primary data. [2025-10-20T17:03:09.015707] [DEBUG] [LOCAL ] ADX: Deleted primary data file. [2025-10-20T17:03:09.015785] [INFO ] [MAIN ] ADX: Starting full primary refresh. [2025-10-20T17:03:09.015667] [INFO ] [REMOTE ] ADC: Full fetch complete (66615 records). [2025-10-20T17:03:09.015886] [DEBUG] [VALIDATOR ] ADC: Validating primary data. [2025-10-20T17:03:09.054120] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-03. [2025-10-20T17:03:09.865202] [INFO ] [REMOTE ] ADX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:09.886950] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-01. [2025-10-20T17:03:09.908588] [DEBUG] [VALIDATOR ] ADC: All expected trading days are present. [2025-10-20T17:03:09.909007] [DEBUG] [VALIDATOR ] ADC: Primary data validation passed. [2025-10-20T17:03:09.909828] [INFO ] [MAIN ] ADC: Fetched primary data is valid. [2025-10-20T17:03:09.910109] [DEBUG] [LOCAL ] ADC: Saving primary data. [2025-10-20T17:03:09.911995] [DEBUG] [LOCAL ] ADC: Trimmed 66615 records to 46431 (keeping 200 days of data) [2025-10-20T17:03:10.070545] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T17:03:10.250563] [DEBUG] [LOCAL ] ADC: Primary data saved (46431 records). [2025-10-20T17:03:10.250752] [INFO ] [MAIN ] ADC: Full history saved. [2025-10-20T17:03:10.251470] [INFO ] [MAIN ] Update Progress: 31/2367 symbols completed. ADC took 43.4s. Estimated remaining: 3h 53m 24s. Rate: 10.0 symbols/min [2025-10-20T17:03:10.423692] [INFO ] [MAIN ] AEE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:10.423946] [DEBUG] [LOCAL ] AEE: No existing data file found in primary storage. [2025-10-20T17:03:10.424009] [WARN ] [MAIN ] AEE: No existing data found, performing full refresh instead. [2025-10-20T17:03:10.424510] [INFO ] [MAIN ] AEE: Wiping existing primary data before full refresh. [2025-10-20T17:03:10.424611] [INFO ] [LOCAL ] AEE: Starting deletion of primary data. [2025-10-20T17:03:10.424721] [DEBUG] [LOCAL ] AEE: No primary data file found to delete. [2025-10-20T17:03:10.424835] [INFO ] [MAIN ] AEE: Starting full primary refresh. [2025-10-20T17:03:10.665885] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-10. [2025-10-20T17:03:10.821442] [INFO ] [REMOTE ] AEE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:10.826896] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:03:11.177133] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T17:03:11.185293] [INFO ] [REMOTE ] ADM: Full fetch complete (77067 records). [2025-10-20T17:03:11.185472] [DEBUG] [VALIDATOR ] ADM: Validating primary data. [2025-10-20T17:03:11.186573] [FATAL] [VALIDATOR ] ADM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:03:11.186690] [FATAL] [MAIN ] ADM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:03:11.186743] [WARN ] [MAIN ] ADM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:03:11.186813] [INFO ] [LOCAL ] ADM: Starting deletion of primary data. [2025-10-20T17:03:11.186882] [DEBUG] [LOCAL ] ADM: No primary data file found to delete. [2025-10-20T17:03:11.186919] [INFO ] [LOCAL ] ADM: Starting deletion of derivative data. [2025-10-20T17:03:11.187054] [DEBUG] [LOCAL ] ADM: Deleted derivative data file. [2025-10-20T17:03:11.187734] [INFO ] [MAIN ] Update Progress: 32/2367 symbols completed. ADM took 35.5s. Estimated remaining: 3h 47m 9s. Rate: 10.3 symbols/min [2025-10-20T17:03:11.380627] [INFO ] [MAIN ] AEFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:11.393213] [DEBUG] [LOCAL ] AEFC: Loaded 8348 existing records from primary storage. [2025-10-20T17:03:11.393459] [INFO ] [MAIN ] AEFC: Loaded 8348 existing records. [2025-10-20T17:03:11.458027] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-04. [2025-10-20T17:03:11.584764] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T17:03:11.611581] [WARN ] [MAIN ] AEFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:11.612724] [INFO ] [MAIN ] AEFC: Wiping existing primary data before full refresh. [2025-10-20T17:03:11.612780] [INFO ] [LOCAL ] AEFC: Starting deletion of primary data. [2025-10-20T17:03:11.612948] [DEBUG] [LOCAL ] AEFC: Deleted primary data file. [2025-10-20T17:03:11.612985] [INFO ] [MAIN ] AEFC: Starting full primary refresh. [2025-10-20T17:03:12.007630] [INFO ] [REMOTE ] AEFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:12.251375] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-01. [2025-10-20T17:03:12.978933] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T17:03:13.025124] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T17:03:13.051362] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-02. [2025-10-20T17:03:13.759834] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T17:03:13.851379] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-01. [2025-10-20T17:03:14.651452] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-05. [2025-10-20T17:03:15.173105] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:03:15.451399] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-02. [2025-10-20T17:03:15.968234] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T17:03:16.163846] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:03:16.251390] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-03. [2025-10-20T17:03:16.777853] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-10-20T17:03:17.051368] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-04. [2025-10-20T17:03:17.848165] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T17:03:17.851325] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-05. [2025-10-20T17:03:18.506067] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T17:03:18.651353] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-06. [2025-10-20T17:03:19.421787] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T17:03:19.451353] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-07. [2025-10-20T17:03:20.183688] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T17:03:20.251389] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-08. [2025-10-20T17:03:20.963386] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T17:03:21.051375] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-06. [2025-10-20T17:03:21.851335] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-09. [2025-10-20T17:03:22.597907] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:03:22.628867] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T17:03:22.651378] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-07. [2025-10-20T17:03:23.451395] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-02. [2025-10-20T17:03:24.251351] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-03. [2025-10-20T17:03:24.299242] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:03:25.010389] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T17:03:25.051387] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-03. [2025-10-20T17:03:25.851438] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-08. [2025-10-20T17:03:26.265535] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T17:03:26.651351] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-10. [2025-10-20T17:03:26.879858] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T17:03:27.280417] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T17:03:27.352561] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T17:03:27.354408] [INFO ] [REMOTE ] AEFC: Full fetch complete (11640 records). [2025-10-20T17:03:27.354542] [DEBUG] [VALIDATOR ] AEFC: Validating primary data. [2025-10-20T17:03:27.456716] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-04. [2025-10-20T17:03:27.781027] [DEBUG] [VALIDATOR ] AEFC: All expected trading days are present. [2025-10-20T17:03:27.781116] [DEBUG] [VALIDATOR ] AEFC: Primary data validation passed. [2025-10-20T17:03:27.781392] [INFO ] [MAIN ] AEFC: Fetched primary data is valid. [2025-10-20T17:03:27.781459] [DEBUG] [LOCAL ] AEFC: Saving primary data. [2025-10-20T17:03:27.782133] [DEBUG] [LOCAL ] AEFC: Trimmed 11640 records to 8025 (keeping 200 days of data) [2025-10-20T17:03:27.840106] [DEBUG] [LOCAL ] AEFC: Primary data saved (8025 records). [2025-10-20T17:03:27.840254] [INFO ] [MAIN ] AEFC: Full history saved. [2025-10-20T17:03:27.840339] [INFO ] [MAIN ] Update Progress: 33/2367 symbols completed. AEFC took 16.7s. Estimated remaining: 3h 59m 48s. Rate: 9.7 symbols/min [2025-10-20T17:03:28.066030] [INFO ] [MAIN ] AEG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:28.118187] [DEBUG] [LOCAL ] AEG: Loaded 50300 existing records from primary storage. [2025-10-20T17:03:28.118555] [INFO ] [MAIN ] AEG: Loaded 50300 existing records. [2025-10-20T17:03:28.256576] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-04. [2025-10-20T17:03:28.313461] [WARN ] [MAIN ] AEG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:28.313752] [INFO ] [MAIN ] AEG: Wiping existing primary data before full refresh. [2025-10-20T17:03:28.313823] [INFO ] [LOCAL ] AEG: Starting deletion of primary data. [2025-10-20T17:03:28.314311] [DEBUG] [LOCAL ] AEG: Deleted primary data file. [2025-10-20T17:03:28.314368] [INFO ] [MAIN ] AEG: Starting full primary refresh. [2025-10-20T17:03:28.712573] [INFO ] [REMOTE ] AEG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:28.725566] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T17:03:29.051392] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-09. [2025-10-20T17:03:29.512313] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T17:03:29.851385] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-01. [2025-10-20T17:03:30.651353] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-05. [2025-10-20T17:03:30.884504] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T17:03:31.451356] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-10. [2025-10-20T17:03:31.495722] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T17:03:31.522697] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T17:03:32.251381] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-05. [2025-10-20T17:03:32.861674] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:03:32.869226] [INFO ] [REMOTE ] ADT: Full fetch complete (77363 records). [2025-10-20T17:03:32.869324] [DEBUG] [VALIDATOR ] ADT: Validating primary data. [2025-10-20T17:03:33.058975] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-06. [2025-10-20T17:03:33.269533] [DEBUG] [VALIDATOR ] ADT: All expected trading days are present. [2025-10-20T17:03:33.269688] [DEBUG] [VALIDATOR ] ADT: Primary data validation passed. [2025-10-20T17:03:33.271097] [INFO ] [MAIN ] ADT: Fetched primary data is valid. [2025-10-20T17:03:33.271886] [DEBUG] [LOCAL ] ADT: Saving primary data. [2025-10-20T17:03:33.273708] [DEBUG] [LOCAL ] ADT: Trimmed 77363 records to 53438 (keeping 200 days of data) [2025-10-20T17:03:33.601480] [DEBUG] [LOCAL ] ADT: Primary data saved (53438 records). [2025-10-20T17:03:33.601678] [INFO ] [MAIN ] ADT: Full history saved. [2025-10-20T17:03:33.602751] [INFO ] [MAIN ] Update Progress: 34/2367 symbols completed. ADT took 35.8s. Estimated remaining: 3h 59m 15s. Rate: 9.8 symbols/min [2025-10-20T17:03:33.819202] [INFO ] [MAIN ] AEM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:33.819531] [DEBUG] [LOCAL ] AEM: No existing data file found in primary storage. [2025-10-20T17:03:33.819603] [WARN ] [MAIN ] AEM: No existing data found, performing full refresh instead. [2025-10-20T17:03:33.819877] [INFO ] [MAIN ] AEM: Wiping existing primary data before full refresh. [2025-10-20T17:03:33.819964] [INFO ] [LOCAL ] AEM: Starting deletion of primary data. [2025-10-20T17:03:33.820052] [DEBUG] [LOCAL ] AEM: No primary data file found to delete. [2025-10-20T17:03:33.820213] [INFO ] [MAIN ] AEM: Starting full primary refresh. [2025-10-20T17:03:33.862397] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-02. [2025-10-20T17:03:33.967995] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:03:34.235266] [INFO ] [REMOTE ] AEM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:34.252764] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:03:34.651386] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-06. [2025-10-20T17:03:35.351714] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:03:35.451374] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-07. [2025-10-20T17:03:36.048370] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T17:03:36.251366] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-07. [2025-10-20T17:03:36.543472] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T17:03:37.051380] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-03. [2025-10-20T17:03:37.851370] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-08. [2025-10-20T17:03:37.923294] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T17:03:38.338202] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T17:03:38.651422] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-01. [2025-10-20T17:03:39.043797] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T17:03:39.451467] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-08. [2025-10-20T17:03:40.251362] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-09. [2025-10-20T17:03:40.368541] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T17:03:41.051354] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-02. [2025-10-20T17:03:41.117552] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T17:03:41.343108] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T17:03:41.851423] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-10. [2025-10-20T17:03:42.651440] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-09. [2025-10-20T17:03:42.693517] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T17:03:42.786994] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T17:03:42.791984] [INFO ] [REMOTE ] ADX: Full fetch complete (41970 records). [2025-10-20T17:03:42.792168] [DEBUG] [VALIDATOR ] ADX: Validating primary data. [2025-10-20T17:03:43.208880] [DEBUG] [VALIDATOR ] ADX: All expected trading days are present. [2025-10-20T17:03:43.209174] [DEBUG] [VALIDATOR ] ADX: Primary data validation passed. [2025-10-20T17:03:43.209724] [INFO ] [MAIN ] ADX: Fetched primary data is valid. [2025-10-20T17:03:43.209796] [DEBUG] [LOCAL ] ADX: Saving primary data. [2025-10-20T17:03:43.211342] [DEBUG] [LOCAL ] ADX: Trimmed 41970 records to 27841 (keeping 200 days of data) [2025-10-20T17:03:43.382081] [DEBUG] [LOCAL ] ADX: Primary data saved (27841 records). [2025-10-20T17:03:43.382238] [INFO ] [MAIN ] ADX: Full history saved. [2025-10-20T17:03:43.382769] [INFO ] [MAIN ] Update Progress: 35/2367 symbols completed. ADX took 34.9s. Estimated remaining: 4h 3m 10s. Rate: 9.6 symbols/min [2025-10-20T17:03:43.456500] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-04. [2025-10-20T17:03:43.574932] [INFO ] [MAIN ] AEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:43.642878] [DEBUG] [LOCAL ] AEO: Loaded 54346 existing records from primary storage. [2025-10-20T17:03:43.643252] [INFO ] [MAIN ] AEO: Loaded 54346 existing records. [2025-10-20T17:03:43.833414] [WARN ] [MAIN ] AEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:43.833670] [INFO ] [MAIN ] AEO: Wiping existing primary data before full refresh. [2025-10-20T17:03:43.833734] [INFO ] [LOCAL ] AEO: Starting deletion of primary data. [2025-10-20T17:03:43.833997] [DEBUG] [LOCAL ] AEO: Deleted primary data file. [2025-10-20T17:03:43.834056] [INFO ] [MAIN ] AEO: Starting full primary refresh. [2025-10-20T17:03:44.238461] [INFO ] [REMOTE ] AEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:44.263112] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-01. [2025-10-20T17:03:44.285443] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T17:03:45.051446] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-10. [2025-10-20T17:03:45.302731] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:03:45.851403] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-05. [2025-10-20T17:03:46.071019] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:03:46.651441] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-03. [2025-10-20T17:03:46.940715] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T17:03:46.948930] [INFO ] [REMOTE ] AEE: Full fetch complete (73026 records). [2025-10-20T17:03:46.949072] [DEBUG] [VALIDATOR ] AEE: Validating primary data. [2025-10-20T17:03:47.354638] [DEBUG] [VALIDATOR ] AEE: All expected trading days are present. [2025-10-20T17:03:47.354771] [DEBUG] [VALIDATOR ] AEE: Primary data validation passed. [2025-10-20T17:03:47.355985] [INFO ] [MAIN ] AEE: Fetched primary data is valid. [2025-10-20T17:03:47.356065] [DEBUG] [LOCAL ] AEE: Saving primary data. [2025-10-20T17:03:47.358342] [DEBUG] [LOCAL ] AEE: Trimmed 73026 records to 50155 (keeping 200 days of data) [2025-10-20T17:03:47.458760] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-02. [2025-10-20T17:03:47.490403] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T17:03:47.718890] [DEBUG] [LOCAL ] AEE: Primary data saved (50155 records). [2025-10-20T17:03:47.719031] [INFO ] [MAIN ] AEE: Full history saved. [2025-10-20T17:03:47.719101] [INFO ] [MAIN ] Update Progress: 36/2367 symbols completed. AEE took 37.5s. Estimated remaining: 4h 0m 60s. Rate: 9.7 symbols/min [2025-10-20T17:03:47.910161] [INFO ] [MAIN ] AER: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:47.965111] [DEBUG] [LOCAL ] AER: Loaded 48795 existing records from primary storage. [2025-10-20T17:03:47.965595] [INFO ] [MAIN ] AER: Loaded 48795 existing records. [2025-10-20T17:03:48.144913] [WARN ] [MAIN ] AER: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:48.145124] [INFO ] [MAIN ] AER: Wiping existing primary data before full refresh. [2025-10-20T17:03:48.145171] [INFO ] [LOCAL ] AER: Starting deletion of primary data. [2025-10-20T17:03:48.145384] [DEBUG] [LOCAL ] AER: Deleted primary data file. [2025-10-20T17:03:48.145439] [INFO ] [MAIN ] AER: Starting full primary refresh. [2025-10-20T17:03:48.252267] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-06. [2025-10-20T17:03:48.546955] [INFO ] [REMOTE ] AER: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:48.611758] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T17:03:49.051389] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-04. [2025-10-20T17:03:49.091857] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T17:03:49.753744] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T17:03:49.851380] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-07. [2025-10-20T17:03:50.651415] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-03. [2025-10-20T17:03:51.036936] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:03:51.242609] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T17:03:51.451371] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-05. [2025-10-20T17:03:52.039993] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:03:52.251378] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-08. [2025-10-20T17:03:53.057952] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-04. [2025-10-20T17:03:53.064958] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:03:53.861287] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-06. [2025-10-20T17:03:53.884617] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T17:03:54.651468] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-09. [2025-10-20T17:03:54.839811] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:03:55.392070] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T17:03:55.451448] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-01. [2025-10-20T17:03:56.215503] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T17:03:56.251368] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-10. [2025-10-20T17:03:56.906690] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T17:03:57.051460] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-05. [2025-10-20T17:03:57.851386] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-07. [2025-10-20T17:03:57.959094] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T17:03:57.967617] [INFO ] [REMOTE ] AEG: Full fetch complete (70650 records). [2025-10-20T17:03:57.967891] [DEBUG] [VALIDATOR ] AEG: Validating primary data. [2025-10-20T17:03:58.430259] [DEBUG] [VALIDATOR ] AEG: All expected trading days are present. [2025-10-20T17:03:58.430389] [DEBUG] [VALIDATOR ] AEG: Primary data validation passed. [2025-10-20T17:03:58.431259] [INFO ] [MAIN ] AEG: Fetched primary data is valid. [2025-10-20T17:03:58.431334] [DEBUG] [LOCAL ] AEG: Saving primary data. [2025-10-20T17:03:58.433743] [DEBUG] [LOCAL ] AEG: Trimmed 70650 records to 49602 (keeping 200 days of data) [2025-10-20T17:03:58.671882] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-02. [2025-10-20T17:03:58.742954] [DEBUG] [LOCAL ] AEG: Primary data saved (49602 records). [2025-10-20T17:03:58.743548] [INFO ] [MAIN ] AEG: Full history saved. [2025-10-20T17:03:58.744922] [INFO ] [MAIN ] Update Progress: 37/2367 symbols completed. AEG took 30.9s. Estimated remaining: 4h 5m 57s. Rate: 9.5 symbols/min [2025-10-20T17:03:58.980843] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T17:03:59.451388] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-06. [2025-10-20T17:03:59.757356] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T17:03:59.942508] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T17:04:00.251389] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-03. [2025-10-20T17:04:01.056566] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-08. [2025-10-20T17:04:01.083129] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T17:04:01.803168] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:04:01.851380] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-04. [2025-10-20T17:04:02.651396] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-07. [2025-10-20T17:04:02.732440] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:04:03.451432] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-09. [2025-10-20T17:04:03.553681] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:04:04.263564] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-05. [2025-10-20T17:04:04.270840] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T17:04:05.051482] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-08. [2025-10-20T17:04:05.186559] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:04:05.737581] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:04:05.851368] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-06. [2025-10-20T17:04:06.651426] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-10. [2025-10-20T17:04:07.192478] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T17:04:07.451408] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-09. [2025-10-20T17:04:07.709959] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T17:04:08.251406] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-07. [2025-10-20T17:04:08.574606] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:04:08.582604] [INFO ] [REMOTE ] AEM: Full fetch complete (77457 records). [2025-10-20T17:04:08.582836] [DEBUG] [VALIDATOR ] AEM: Validating primary data. [2025-10-20T17:04:09.005931] [DEBUG] [VALIDATOR ] AEM: All expected trading days are present. [2025-10-20T17:04:09.006055] [DEBUG] [VALIDATOR ] AEM: Primary data validation passed. [2025-10-20T17:04:09.006978] [INFO ] [MAIN ] AEM: Fetched primary data is valid. [2025-10-20T17:04:09.007048] [DEBUG] [LOCAL ] AEM: Saving primary data. [2025-10-20T17:04:09.009822] [DEBUG] [LOCAL ] AEM: Trimmed 77457 records to 53477 (keeping 200 days of data) [2025-10-20T17:04:09.379013] [DEBUG] [LOCAL ] AEM: Primary data saved (53477 records). [2025-10-20T17:04:09.379211] [INFO ] [MAIN ] AEM: Full history saved. [2025-10-20T17:04:09.379302] [INFO ] [MAIN ] Update Progress: 38/2367 symbols completed. AEM took 35.8s. Estimated remaining: 4h 10m 14s. Rate: 9.3 symbols/min [2025-10-20T17:04:09.846999] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-09 (2.40 seconds). [2025-10-20T17:04:09.860682] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-10. [2025-10-20T17:04:10.258842] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T17:04:10.270922] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-08. [2025-10-20T17:04:11.707520] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T17:04:11.717506] [INFO ] [REMOTE ] AEO: Full fetch complete (77648 records). [2025-10-20T17:04:11.717982] [DEBUG] [VALIDATOR ] AEO: Validating primary data. [2025-10-20T17:04:12.172720] [DEBUG] [VALIDATOR ] AEO: All expected trading days are present. [2025-10-20T17:04:12.172879] [DEBUG] [VALIDATOR ] AEO: Primary data validation passed. [2025-10-20T17:04:12.173875] [INFO ] [MAIN ] AEO: Fetched primary data is valid. [2025-10-20T17:04:12.173979] [DEBUG] [LOCAL ] AEO: Saving primary data. [2025-10-20T17:04:12.176238] [DEBUG] [LOCAL ] AEO: Trimmed 77648 records to 53565 (keeping 200 days of data) [2025-10-20T17:04:12.307073] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-10-20T17:04:12.347871] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-09. [2025-10-20T17:04:12.564079] [DEBUG] [LOCAL ] AEO: Primary data saved (53565 records). [2025-10-20T17:04:12.564287] [INFO ] [MAIN ] AEO: Full history saved. [2025-10-20T17:04:12.565481] [INFO ] [MAIN ] Update Progress: 39/2367 symbols completed. AEO took 29.2s. Estimated remaining: 4h 6m 53s. Rate: 9.4 symbols/min [2025-10-20T17:04:14.109104] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T17:04:14.120263] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-10. [2025-10-20T17:04:15.803254] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T17:04:15.811044] [INFO ] [REMOTE ] AER: Full fetch complete (69618 records). [2025-10-20T17:04:15.811280] [DEBUG] [VALIDATOR ] AER: Validating primary data. [2025-10-20T17:04:15.812485] [FATAL] [VALIDATOR ] AER: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:04:15.812667] [FATAL] [MAIN ] AER: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:04:15.812730] [WARN ] [MAIN ] AER: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:04:15.813073] [INFO ] [LOCAL ] AER: Starting deletion of primary data. [2025-10-20T17:04:15.813151] [DEBUG] [LOCAL ] AER: No primary data file found to delete. [2025-10-20T17:04:15.813192] [INFO ] [LOCAL ] AER: Starting deletion of derivative data. [2025-10-20T17:04:15.813426] [DEBUG] [LOCAL ] AER: Deleted derivative data file. [2025-10-20T17:04:15.814811] [INFO ] [MAIN ] Update Progress: 40/2367 symbols completed. AER took 28.1s. Estimated remaining: 4h 3m 46s. Rate: 9.5 symbols/min [2025-10-20T17:04:16.481102] [INFO ] [MAIN ] AFB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:16.618315] [INFO ] [MAIN ] AES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:16.606838] [INFO ] [MAIN ] AEXA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:16.638389] [INFO ] [MAIN ] AESI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:16.638778] [DEBUG] [LOCAL ] AEXA: No existing data file found in primary storage. [2025-10-20T17:04:16.639578] [WARN ] [MAIN ] AEXA: No existing data found, performing full refresh instead. [2025-10-20T17:04:16.644844] [INFO ] [MAIN ] AEXA: Wiping existing primary data before full refresh. [2025-10-20T17:04:16.645258] [INFO ] [LOCAL ] AEXA: Starting deletion of primary data. [2025-10-20T17:04:16.645417] [DEBUG] [LOCAL ] AEXA: No primary data file found to delete. [2025-10-20T17:04:16.645463] [INFO ] [MAIN ] AEXA: Starting full primary refresh. [2025-10-20T17:04:16.734278] [DEBUG] [LOCAL ] AFB: Loaded 11315 existing records from primary storage. [2025-10-20T17:04:16.772952] [INFO ] [MAIN ] AFB: Loaded 11315 existing records. [2025-10-20T17:04:17.212828] [DEBUG] [LOCAL ] AES: Loaded 54448 existing records from primary storage. [2025-10-20T17:04:17.244455] [INFO ] [MAIN ] AES: Loaded 54448 existing records. [2025-10-20T17:04:17.297274] [DEBUG] [LOCAL ] AESI: Loaded 50515 existing records from primary storage. [2025-10-20T17:04:17.396131] [INFO ] [MAIN ] AESI: Loaded 50515 existing records. [2025-10-20T17:04:17.414832] [WARN ] [MAIN ] AFB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:17.576919] [INFO ] [MAIN ] AFB: Wiping existing primary data before full refresh. [2025-10-20T17:04:17.588426] [INFO ] [REMOTE ] AEXA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:17.658537] [INFO ] [REMOTE ] AEXA: Fetching primary data for 2025-01. [2025-10-20T17:04:17.647985] [INFO ] [LOCAL ] AFB: Starting deletion of primary data. [2025-10-20T17:04:17.757932] [DEBUG] [LOCAL ] AFB: Deleted primary data file. [2025-10-20T17:04:17.784296] [INFO ] [MAIN ] AFB: Starting full primary refresh. [2025-10-20T17:04:17.838303] [WARN ] [MAIN ] AES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:17.843941] [INFO ] [MAIN ] AES: Wiping existing primary data before full refresh. [2025-10-20T17:04:17.874777] [INFO ] [LOCAL ] AES: Starting deletion of primary data. [2025-10-20T17:04:17.936427] [DEBUG] [LOCAL ] AES: Deleted primary data file. [2025-10-20T17:04:17.936332] [DEBUG] [REMOTE ] AEXA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:04:17.969384] [WARN ] [MAIN ] AESI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:17.980920] [INFO ] [MAIN ] AESI: Wiping existing primary data before full refresh. [2025-10-20T17:04:17.975323] [INFO ] [MAIN ] AES: Starting full primary refresh. [2025-10-20T17:04:17.975143] [WARN ] [REMOTE ] AEXA: No data returned for month 2025-01 [2025-10-20T17:04:17.996675] [INFO ] [LOCAL ] AESI: Starting deletion of primary data. [2025-10-20T17:04:18.007395] [FATAL] [REMOTE ] AEXA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:04:18.067089] [FATAL] [MAIN ] AEXA: Failed to fetch primary data. [2025-10-20T17:04:18.077787] [WARN ] [MAIN ] AEXA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:04:18.113259] [INFO ] [LOCAL ] AEXA: Starting deletion of primary data. [2025-10-20T17:04:18.107863] [DEBUG] [LOCAL ] AESI: Deleted primary data file. [2025-10-20T17:04:18.126690] [INFO ] [MAIN ] AESI: Starting full primary refresh. [2025-10-20T17:04:18.137431] [DEBUG] [LOCAL ] AEXA: No primary data file found to delete. [2025-10-20T17:04:18.192853] [INFO ] [LOCAL ] AEXA: Starting deletion of derivative data. [2025-10-20T17:04:18.262607] [DEBUG] [LOCAL ] AEXA: No derivative data file found to delete. [2025-10-20T17:04:18.301844] [INFO ] [MAIN ] Update Progress: 41/2367 symbols completed. AEXA took 2.4s. Estimated remaining: 4h 0m 4s. Rate: 9.7 symbols/min [2025-10-20T17:04:19.192670] [INFO ] [REMOTE ] AES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:19.189411] [INFO ] [MAIN ] AFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:19.184946] [INFO ] [REMOTE ] AFB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:19.233470] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-01. [2025-10-20T17:04:19.243918] [DEBUG] [LOCAL ] AFG: No existing data file found in primary storage. [2025-10-20T17:04:19.250173] [WARN ] [MAIN ] AFG: No existing data found, performing full refresh instead. [2025-10-20T17:04:19.244598] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-01. [2025-10-20T17:04:19.262058] [INFO ] [MAIN ] AFG: Wiping existing primary data before full refresh. [2025-10-20T17:04:19.281882] [INFO ] [LOCAL ] AFG: Starting deletion of primary data. [2025-10-20T17:04:19.287484] [DEBUG] [LOCAL ] AFG: No primary data file found to delete. [2025-10-20T17:04:19.287597] [INFO ] [MAIN ] AFG: Starting full primary refresh. [2025-10-20T17:04:19.455980] [INFO ] [REMOTE ] AESI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:19.482536] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-01. [2025-10-20T17:04:19.761347] [INFO ] [REMOTE ] AFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:19.761681] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-01. [2025-10-20T17:04:19.948453] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:04:19.951321] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-02. [2025-10-20T17:04:20.722215] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T17:04:20.725086] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-03. [2025-10-20T17:04:20.897873] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T17:04:20.905028] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-02. [2025-10-20T17:04:20.962267] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T17:04:20.971470] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-02. [2025-10-20T17:04:21.028466] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:04:21.038956] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-02. [2025-10-20T17:04:21.287631] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T17:04:21.290956] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-04. [2025-10-20T17:04:21.946994] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T17:04:21.956306] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-03. [2025-10-20T17:04:21.973599] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T17:04:21.978888] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-05. [2025-10-20T17:04:22.502947] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T17:04:22.562680] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T17:04:22.651334] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-03. [2025-10-20T17:04:22.758266] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:04:23.451440] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-03. [2025-10-20T17:04:23.647388] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T17:04:23.852424] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T17:04:24.251447] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-04. [2025-10-20T17:04:25.041940] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T17:04:25.053090] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-04. [2025-10-20T17:04:25.712829] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:04:25.851362] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-05. [2025-10-20T17:04:26.651475] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-06. [2025-10-20T17:04:26.913700] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T17:04:27.096887] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T17:04:27.316314] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T17:04:27.451455] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-07. [2025-10-20T17:04:28.251344] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-05. [2025-10-20T17:04:28.334333] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T17:04:29.051433] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-04. [2025-10-20T17:04:29.766217] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:04:29.851360] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-06. [2025-10-20T17:04:30.651422] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-06. [2025-10-20T17:04:30.921709] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T17:04:31.451402] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-05. [2025-10-20T17:04:31.498352] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T17:04:32.095199] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T17:04:32.251419] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-08. [2025-10-20T17:04:33.051363] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-07. [2025-10-20T17:04:33.126844] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T17:04:33.391899] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T17:04:33.851375] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-06. [2025-10-20T17:04:34.411231] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:04:34.651368] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-08. [2025-10-20T17:04:35.451356] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-09. [2025-10-20T17:04:35.852901] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:04:36.004379] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:04:36.007367] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T17:04:36.251401] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-09. [2025-10-20T17:04:37.051425] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-07. [2025-10-20T17:04:37.777458] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:04:37.851371] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-10. [2025-10-20T17:04:38.651378] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-10. [2025-10-20T17:04:39.012688] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:04:39.017924] [INFO ] [REMOTE ] AFG: Full fetch complete (46719 records). [2025-10-20T17:04:39.018111] [DEBUG] [VALIDATOR ] AFG: Validating primary data. [2025-10-20T17:04:39.404487] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-10-20T17:04:39.456996] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-08. [2025-10-20T17:04:39.508516] [DEBUG] [VALIDATOR ] AFG: All expected trading days are present. [2025-10-20T17:04:39.508589] [DEBUG] [VALIDATOR ] AFG: Primary data validation passed. [2025-10-20T17:04:39.509079] [INFO ] [MAIN ] AFG: Fetched primary data is valid. [2025-10-20T17:04:39.509253] [DEBUG] [LOCAL ] AFG: Saving primary data. [2025-10-20T17:04:39.510369] [DEBUG] [LOCAL ] AFG: Trimmed 46719 records to 31954 (keeping 200 days of data) [2025-10-20T17:04:39.611026] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T17:04:39.636781] [INFO ] [REMOTE ] AFB: Full fetch complete (16449 records). [2025-10-20T17:04:39.648970] [DEBUG] [VALIDATOR ] AFB: Validating primary data. [2025-10-20T17:04:39.825435] [DEBUG] [LOCAL ] AFG: Primary data saved (31954 records). [2025-10-20T17:04:39.841296] [INFO ] [MAIN ] AFG: Full history saved. [2025-10-20T17:04:39.851707] [INFO ] [MAIN ] Update Progress: 42/2367 symbols completed. AFG took 21.5s. Estimated remaining: 4h 14m 8s. Rate: 9.1 symbols/min [2025-10-20T17:04:40.254270] [INFO ] [MAIN ] AFGB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:40.260579] [DEBUG] [LOCAL ] AFGB: No existing data file found in primary storage. [2025-10-20T17:04:40.254387] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-07. [2025-10-20T17:04:40.264600] [WARN ] [MAIN ] AFGB: No existing data found, performing full refresh instead. [2025-10-20T17:04:40.290751] [INFO ] [MAIN ] AFGB: Wiping existing primary data before full refresh. [2025-10-20T17:04:40.303490] [INFO ] [LOCAL ] AFGB: Starting deletion of primary data. [2025-10-20T17:04:40.303911] [DEBUG] [LOCAL ] AFGB: No primary data file found to delete. [2025-10-20T17:04:40.304082] [DEBUG] [VALIDATOR ] AFB: All expected trading days are present. [2025-10-20T17:04:40.304179] [INFO ] [MAIN ] AFGB: Starting full primary refresh. [2025-10-20T17:04:40.304274] [DEBUG] [VALIDATOR ] AFB: Primary data validation passed. [2025-10-20T17:04:40.304752] [INFO ] [MAIN ] AFB: Fetched primary data is valid. [2025-10-20T17:04:40.310025] [DEBUG] [LOCAL ] AFB: Saving primary data. [2025-10-20T17:04:40.371756] [DEBUG] [LOCAL ] AFB: Trimmed 16449 records to 11115 (keeping 200 days of data) [2025-10-20T17:04:40.563131] [DEBUG] [LOCAL ] AFB: Primary data saved (11115 records). [2025-10-20T17:04:40.600211] [INFO ] [MAIN ] AFB: Full history saved. [2025-10-20T17:04:40.613727] [INFO ] [MAIN ] Update Progress: 43/2367 symbols completed. AFB took 24.8s. Estimated remaining: 4h 8m 48s. Rate: 9.3 symbols/min [2025-10-20T17:04:40.927166] [INFO ] [REMOTE ] AFGB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:40.947885] [INFO ] [MAIN ] AFGC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:40.953212] [DEBUG] [LOCAL ] AFGC: Loaded 2584 existing records from primary storage. [2025-10-20T17:04:40.953400] [INFO ] [MAIN ] AFGC: Loaded 2584 existing records. [2025-10-20T17:04:41.057317] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-01. [2025-10-20T17:04:41.131114] [WARN ] [MAIN ] AFGC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:41.131667] [INFO ] [MAIN ] AFGC: Wiping existing primary data before full refresh. [2025-10-20T17:04:41.131756] [INFO ] [LOCAL ] AFGC: Starting deletion of primary data. [2025-10-20T17:04:41.131919] [DEBUG] [LOCAL ] AFGC: Deleted primary data file. [2025-10-20T17:04:41.131977] [INFO ] [MAIN ] AFGC: Starting full primary refresh. [2025-10-20T17:04:41.523226] [INFO ] [REMOTE ] AFGC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:41.567469] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T17:04:41.604297] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T17:04:41.858963] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-02. [2025-10-20T17:04:41.865442] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T17:04:42.184353] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T17:04:42.651381] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-09. [2025-10-20T17:04:43.451427] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-01. [2025-10-20T17:04:43.871961] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T17:04:44.251446] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-02. [2025-10-20T17:04:44.692085] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:04:44.754442] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-10-20T17:04:45.051416] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-03. [2025-10-20T17:04:45.552710] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T17:04:45.851372] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-10. [2025-10-20T17:04:46.651442] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-03. [2025-10-20T17:04:47.208508] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T17:04:47.451451] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-04. [2025-10-20T17:04:47.815553] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T17:04:47.824420] [INFO ] [REMOTE ] AES: Full fetch complete (77846 records). [2025-10-20T17:04:47.824605] [DEBUG] [VALIDATOR ] AES: Validating primary data. [2025-10-20T17:04:47.940382] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T17:04:48.242297] [DEBUG] [VALIDATOR ] AES: All expected trading days are present. [2025-10-20T17:04:48.242389] [DEBUG] [VALIDATOR ] AES: Primary data validation passed. [2025-10-20T17:04:48.243211] [INFO ] [MAIN ] AES: Fetched primary data is valid. [2025-10-20T17:04:48.243278] [DEBUG] [LOCAL ] AES: Saving primary data. [2025-10-20T17:04:48.244903] [DEBUG] [LOCAL ] AES: Trimmed 77846 records to 53668 (keeping 200 days of data) [2025-10-20T17:04:48.300690] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-08. [2025-10-20T17:04:48.592680] [DEBUG] [LOCAL ] AES: Primary data saved (53668 records). [2025-10-20T17:04:48.592842] [INFO ] [MAIN ] AES: Full history saved. [2025-10-20T17:04:48.592930] [INFO ] [MAIN ] Update Progress: 44/2367 symbols completed. AES took 32.8s. Estimated remaining: 4h 10m 4s. Rate: 9.3 symbols/min [2025-10-20T17:04:48.765706] [INFO ] [MAIN ] AFGD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:48.769932] [DEBUG] [LOCAL ] AFGD: Loaded 1791 existing records from primary storage. [2025-10-20T17:04:48.770097] [INFO ] [MAIN ] AFGD: Loaded 1791 existing records. [2025-10-20T17:04:48.961545] [WARN ] [MAIN ] AFGD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:48.961784] [INFO ] [MAIN ] AFGD: Wiping existing primary data before full refresh. [2025-10-20T17:04:48.961833] [INFO ] [LOCAL ] AFGD: Starting deletion of primary data. [2025-10-20T17:04:48.961966] [DEBUG] [LOCAL ] AFGD: Deleted primary data file. [2025-10-20T17:04:48.962042] [INFO ] [MAIN ] AFGD: Starting full primary refresh. [2025-10-20T17:04:49.060405] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-05. [2025-10-20T17:04:49.322712] [INFO ] [REMOTE ] AFGD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:49.545277] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T17:04:49.851380] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-06. [2025-10-20T17:04:49.904121] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T17:04:50.309392] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T17:04:50.651408] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-09. [2025-10-20T17:04:51.451363] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-01. [2025-10-20T17:04:52.052446] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T17:04:52.111694] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:04:52.251394] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-07. [2025-10-20T17:04:52.819874] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T17:04:53.051464] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-04. [2025-10-20T17:04:53.473635] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T17:04:53.851395] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-08. [2025-10-20T17:04:54.275507] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T17:04:54.651392] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-05. [2025-10-20T17:04:55.055053] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-10-20T17:04:55.451419] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-10. [2025-10-20T17:04:56.251420] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-06. [2025-10-20T17:04:56.777805] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T17:04:57.051386] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-09. [2025-10-20T17:04:57.308696] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T17:04:57.316337] [INFO ] [REMOTE ] AESI: Full fetch complete (71403 records). [2025-10-20T17:04:57.316448] [DEBUG] [VALIDATOR ] AESI: Validating primary data. [2025-10-20T17:04:57.420052] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-10-20T17:04:57.774424] [DEBUG] [VALIDATOR ] AESI: All expected trading days are present. [2025-10-20T17:04:57.774544] [DEBUG] [VALIDATOR ] AESI: Primary data validation passed. [2025-10-20T17:04:57.775326] [INFO ] [MAIN ] AESI: Fetched primary data is valid. [2025-10-20T17:04:57.775398] [DEBUG] [LOCAL ] AESI: Saving primary data. [2025-10-20T17:04:57.777305] [DEBUG] [LOCAL ] AESI: Trimmed 71403 records to 49828 (keeping 200 days of data) [2025-10-20T17:04:57.869430] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-10. [2025-10-20T17:04:58.095561] [DEBUG] [LOCAL ] AESI: Primary data saved (49828 records). [2025-10-20T17:04:58.095794] [INFO ] [MAIN ] AESI: Full history saved. [2025-10-20T17:04:58.095900] [INFO ] [MAIN ] Update Progress: 45/2367 symbols completed. AESI took 42.3s. Estimated remaining: 4h 12m 34s. Rate: 9.2 symbols/min [2025-10-20T17:04:58.275250] [INFO ] [MAIN ] AFGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:58.279513] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T17:04:58.280933] [INFO ] [REMOTE ] AFGC: Full fetch complete (3606 records). [2025-10-20T17:04:58.281028] [DEBUG] [VALIDATOR ] AFGC: Validating primary data. [2025-10-20T17:04:58.367947] [DEBUG] [LOCAL ] AFGE: Loaded 2387 existing records from primary storage. [2025-10-20T17:04:58.394890] [INFO ] [MAIN ] AFGE: Loaded 2387 existing records. [2025-10-20T17:04:58.653800] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-07. [2025-10-20T17:04:58.833612] [WARN ] [MAIN ] AFGE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:58.883427] [INFO ] [MAIN ] AFGE: Wiping existing primary data before full refresh. [2025-10-20T17:04:58.888742] [DEBUG] [VALIDATOR ] AFGC: All expected trading days are present. [2025-10-20T17:04:58.888631] [INFO ] [LOCAL ] AFGE: Starting deletion of primary data. [2025-10-20T17:04:58.888942] [DEBUG] [VALIDATOR ] AFGC: Primary data validation passed. [2025-10-20T17:04:58.889215] [INFO ] [MAIN ] AFGC: Fetched primary data is valid. [2025-10-20T17:04:58.889289] [DEBUG] [LOCAL ] AFGE: Deleted primary data file. [2025-10-20T17:04:58.889501] [INFO ] [MAIN ] AFGE: Starting full primary refresh. [2025-10-20T17:04:58.889408] [DEBUG] [LOCAL ] AFGC: Saving primary data. [2025-10-20T17:04:58.953850] [DEBUG] [LOCAL ] AFGC: Trimmed 3606 records to 2561 (keeping 200 days of data) [2025-10-20T17:04:59.024074] [DEBUG] [LOCAL ] AFGC: Primary data saved (2561 records). [2025-10-20T17:04:59.024183] [INFO ] [MAIN ] AFGC: Full history saved. [2025-10-20T17:04:59.024332] [INFO ] [MAIN ] Update Progress: 46/2367 symbols completed. AFGC took 18.4s. Estimated remaining: 4h 7m 45s. Rate: 9.4 symbols/min [2025-10-20T17:04:59.246538] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T17:04:59.414774] [INFO ] [MAIN ] AFL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:59.462686] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-02. [2025-10-20T17:04:59.513617] [INFO ] [REMOTE ] AFGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:59.558691] [DEBUG] [LOCAL ] AFL: Loaded 53965 existing records from primary storage. [2025-10-20T17:04:59.559057] [INFO ] [MAIN ] AFL: Loaded 53965 existing records. [2025-10-20T17:04:59.731269] [WARN ] [MAIN ] AFL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:59.731507] [INFO ] [MAIN ] AFL: Wiping existing primary data before full refresh. [2025-10-20T17:04:59.731565] [INFO ] [LOCAL ] AFL: Starting deletion of primary data. [2025-10-20T17:04:59.731770] [DEBUG] [LOCAL ] AFL: Deleted primary data file. [2025-10-20T17:04:59.731831] [INFO ] [MAIN ] AFL: Starting full primary refresh. [2025-10-20T17:04:59.904673] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:05:00.081135] [INFO ] [REMOTE ] AFL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:00.251379] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-03. [2025-10-20T17:05:00.757472] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-10-20T17:05:01.051384] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-01. [2025-10-20T17:05:01.642122] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T17:05:01.851389] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-02. [2025-10-20T17:05:02.315336] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T17:05:02.651413] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-04. [2025-10-20T17:05:03.119715] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-10-20T17:05:03.451417] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-03. [2025-10-20T17:05:03.814797] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-10-20T17:05:04.251415] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-05. [2025-10-20T17:05:04.569798] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-10-20T17:05:05.051439] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-01. [2025-10-20T17:05:05.851422] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-08. [2025-10-20T17:05:06.301889] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T17:05:06.651387] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-09. [2025-10-20T17:05:06.976958] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:05:07.142732] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T17:05:07.451419] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-04. [2025-10-20T17:05:07.876442] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T17:05:08.251412] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-05. [2025-10-20T17:05:08.671120] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T17:05:09.051402] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-02. [2025-10-20T17:05:09.851431] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-06. [2025-10-20T17:05:10.345206] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T17:05:10.553768] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T17:05:10.651361] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-03. [2025-10-20T17:05:11.451373] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-07. [2025-10-20T17:05:12.022162] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T17:05:12.251375] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-10. [2025-10-20T17:05:12.699072] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T17:05:12.733622] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T17:05:12.734569] [INFO ] [REMOTE ] AFGB: Full fetch complete (3467 records). [2025-10-20T17:05:12.734637] [DEBUG] [VALIDATOR ] AFGB: Validating primary data. [2025-10-20T17:05:13.060473] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-04. [2025-10-20T17:05:13.093834] [DEBUG] [VALIDATOR ] AFGB: All expected trading days are present. [2025-10-20T17:05:13.093930] [DEBUG] [VALIDATOR ] AFGB: Primary data validation passed. [2025-10-20T17:05:13.094100] [INFO ] [MAIN ] AFGB: Fetched primary data is valid. [2025-10-20T17:05:13.094143] [DEBUG] [LOCAL ] AFGB: Saving primary data. [2025-10-20T17:05:13.094498] [DEBUG] [LOCAL ] AFGB: Trimmed 3467 records to 2245 (keeping 200 days of data) [2025-10-20T17:05:13.109278] [DEBUG] [LOCAL ] AFGB: Primary data saved (2245 records). [2025-10-20T17:05:13.109425] [INFO ] [MAIN ] AFGB: Full history saved. [2025-10-20T17:05:13.109504] [INFO ] [MAIN ] Update Progress: 47/2367 symbols completed. AFGB took 33.2s. Estimated remaining: 4h 13m 58s. Rate: 9.1 symbols/min [2025-10-20T17:05:13.291296] [INFO ] [MAIN ] AG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:13.339856] [DEBUG] [LOCAL ] AG: Loaded 54423 existing records from primary storage. [2025-10-20T17:05:13.340184] [INFO ] [MAIN ] AG: Loaded 54423 existing records. [2025-10-20T17:05:13.520087] [WARN ] [MAIN ] AG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:13.520315] [INFO ] [MAIN ] AG: Wiping existing primary data before full refresh. [2025-10-20T17:05:13.520362] [INFO ] [LOCAL ] AG: Starting deletion of primary data. [2025-10-20T17:05:13.520577] [DEBUG] [LOCAL ] AG: Deleted primary data file. [2025-10-20T17:05:13.520633] [INFO ] [MAIN ] AG: Starting full primary refresh. [2025-10-20T17:05:13.864578] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-06. [2025-10-20T17:05:13.905955] [INFO ] [REMOTE ] AG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:14.359543] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T17:05:14.647687] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:05:14.658364] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-01. [2025-10-20T17:05:15.451401] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-05. [2025-10-20T17:05:16.264110] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-07. [2025-10-20T17:05:16.272021] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:05:16.725211] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T17:05:17.051393] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-08. [2025-10-20T17:05:17.390704] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T17:05:17.465002] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-10-20T17:05:17.851434] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-08. [2025-10-20T17:05:18.449485] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T17:05:18.651352] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-02. [2025-10-20T17:05:19.451401] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-09. [2025-10-20T17:05:19.917570] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T17:05:20.251384] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-10. [2025-10-20T17:05:20.603482] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-10-20T17:05:20.682927] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-10-20T17:05:20.683783] [INFO ] [REMOTE ] AFGD: Full fetch complete (2778 records). [2025-10-20T17:05:20.683851] [DEBUG] [VALIDATOR ] AFGD: Validating primary data. [2025-10-20T17:05:21.045110] [DEBUG] [VALIDATOR ] AFGD: All expected trading days are present. [2025-10-20T17:05:21.045188] [DEBUG] [VALIDATOR ] AFGD: Primary data validation passed. [2025-10-20T17:05:21.045286] [INFO ] [MAIN ] AFGD: Fetched primary data is valid. [2025-10-20T17:05:21.045324] [DEBUG] [LOCAL ] AFGD: Saving primary data. [2025-10-20T17:05:21.045664] [DEBUG] [LOCAL ] AFGD: Trimmed 2778 records to 1775 (keeping 200 days of data) [2025-10-20T17:05:21.054001] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-06. [2025-10-20T17:05:21.057026] [DEBUG] [LOCAL ] AFGD: Primary data saved (1775 records). [2025-10-20T17:05:21.057984] [INFO ] [MAIN ] AFGD: Full history saved. [2025-10-20T17:05:21.058126] [INFO ] [MAIN ] Update Progress: 48/2367 symbols completed. AFGD took 32.5s. Estimated remaining: 4h 14m 58s. Rate: 9.1 symbols/min [2025-10-20T17:05:21.227784] [INFO ] [MAIN ] AGCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:21.271191] [DEBUG] [LOCAL ] AGCO: Loaded 42964 existing records from primary storage. [2025-10-20T17:05:21.271519] [INFO ] [MAIN ] AGCO: Loaded 42964 existing records. [2025-10-20T17:05:21.447054] [WARN ] [MAIN ] AGCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:21.447280] [INFO ] [MAIN ] AGCO: Wiping existing primary data before full refresh. [2025-10-20T17:05:21.447336] [INFO ] [LOCAL ] AGCO: Starting deletion of primary data. [2025-10-20T17:05:21.447883] [DEBUG] [LOCAL ] AGCO: Deleted primary data file. [2025-10-20T17:05:21.447939] [INFO ] [MAIN ] AGCO: Starting full primary refresh. [2025-10-20T17:05:21.801110] [INFO ] [REMOTE ] AGCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:21.851329] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-01. [2025-10-20T17:05:22.512574] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T17:05:22.651398] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-09. [2025-10-20T17:05:23.146806] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T17:05:23.413849] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:05:23.451327] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-02. [2025-10-20T17:05:24.251391] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-07. [2025-10-20T17:05:24.907936] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:05:25.051399] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-03. [2025-10-20T17:05:25.851348] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-03. [2025-10-20T17:05:25.983229] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:05:26.656712] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-08. [2025-10-20T17:05:26.676206] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:05:27.463954] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-04. [2025-10-20T17:05:27.494356] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T17:05:28.251350] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-10. [2025-10-20T17:05:28.500463] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T17:05:28.726465] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T17:05:28.727688] [INFO ] [REMOTE ] AFGE: Full fetch complete (3365 records). [2025-10-20T17:05:28.727798] [DEBUG] [VALIDATOR ] AFGE: Validating primary data. [2025-10-20T17:05:29.058131] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-09. [2025-10-20T17:05:29.151371] [DEBUG] [VALIDATOR ] AFGE: All expected trading days are present. [2025-10-20T17:05:29.151604] [DEBUG] [VALIDATOR ] AFGE: Primary data validation passed. [2025-10-20T17:05:29.151834] [INFO ] [MAIN ] AFGE: Fetched primary data is valid. [2025-10-20T17:05:29.151967] [DEBUG] [LOCAL ] AFGE: Saving primary data. [2025-10-20T17:05:29.152511] [DEBUG] [LOCAL ] AFGE: Trimmed 3365 records to 2313 (keeping 200 days of data) [2025-10-20T17:05:29.184631] [DEBUG] [LOCAL ] AFGE: Primary data saved (2313 records). [2025-10-20T17:05:29.196953] [INFO ] [MAIN ] AFGE: Full history saved. [2025-10-20T17:05:29.197132] [INFO ] [MAIN ] Update Progress: 49/2367 symbols completed. AFGE took 31.1s. Estimated remaining: 4h 16m 5s. Rate: 9.1 symbols/min [2025-10-20T17:05:29.243871] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:05:29.394634] [INFO ] [MAIN ] AGD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:29.414200] [DEBUG] [LOCAL ] AGD: Loaded 19496 existing records from primary storage. [2025-10-20T17:05:29.414483] [INFO ] [MAIN ] AGD: Loaded 19496 existing records. [2025-10-20T17:05:29.590463] [WARN ] [MAIN ] AGD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:29.590789] [INFO ] [MAIN ] AGD: Wiping existing primary data before full refresh. [2025-10-20T17:05:29.590850] [INFO ] [LOCAL ] AGD: Starting deletion of primary data. [2025-10-20T17:05:29.591761] [DEBUG] [LOCAL ] AGD: Deleted primary data file. [2025-10-20T17:05:29.591824] [INFO ] [MAIN ] AGD: Starting full primary refresh. [2025-10-20T17:05:29.861350] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-04. [2025-10-20T17:05:29.941453] [INFO ] [REMOTE ] AGD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:30.651392] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-01. [2025-10-20T17:05:30.917732] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T17:05:31.464495] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-10. [2025-10-20T17:05:31.467176] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T17:05:31.668966] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T17:05:32.251410] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-05. [2025-10-20T17:05:33.051364] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-05. [2025-10-20T17:05:33.361380] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T17:05:33.369769] [INFO ] [REMOTE ] AFL: Full fetch complete (77065 records). [2025-10-20T17:05:33.369919] [DEBUG] [VALIDATOR ] AFL: Validating primary data. [2025-10-20T17:05:33.782527] [DEBUG] [VALIDATOR ] AFL: All expected trading days are present. [2025-10-20T17:05:33.782872] [DEBUG] [VALIDATOR ] AFL: Primary data validation passed. [2025-10-20T17:05:33.783890] [INFO ] [MAIN ] AFL: Fetched primary data is valid. [2025-10-20T17:05:33.783982] [DEBUG] [LOCAL ] AFL: Saving primary data. [2025-10-20T17:05:33.786428] [DEBUG] [LOCAL ] AFL: Trimmed 77065 records to 53182 (keeping 200 days of data) [2025-10-20T17:05:33.869962] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-02. [2025-10-20T17:05:33.927747] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:05:34.176154] [DEBUG] [LOCAL ] AFL: Primary data saved (53182 records). [2025-10-20T17:05:34.176296] [INFO ] [MAIN ] AFL: Full history saved. [2025-10-20T17:05:34.177635] [INFO ] [MAIN ] Update Progress: 50/2367 symbols completed. AFL took 35.2s. Estimated remaining: 4h 14m 42s. Rate: 9.1 symbols/min [2025-10-20T17:05:34.347316] [INFO ] [MAIN ] AGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:34.347499] [DEBUG] [LOCAL ] AGI: No existing data file found in primary storage. [2025-10-20T17:05:34.347543] [WARN ] [MAIN ] AGI: No existing data found, performing full refresh instead. [2025-10-20T17:05:34.347677] [INFO ] [MAIN ] AGI: Wiping existing primary data before full refresh. [2025-10-20T17:05:34.347730] [INFO ] [LOCAL ] AGI: Starting deletion of primary data. [2025-10-20T17:05:34.347772] [DEBUG] [LOCAL ] AGI: No primary data file found to delete. [2025-10-20T17:05:34.347804] [INFO ] [MAIN ] AGI: Starting full primary refresh. [2025-10-20T17:05:34.646214] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T17:05:34.654721] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-06. [2025-10-20T17:05:34.737003] [INFO ] [REMOTE ] AGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:35.162599] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-05 (2.11 seconds). [2025-10-20T17:05:35.451442] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-03. [2025-10-20T17:05:35.848986] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:05:36.254074] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-01. [2025-10-20T17:05:36.258148] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:05:37.051441] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-04. [2025-10-20T17:05:37.698312] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:05:37.851464] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-06. [2025-10-20T17:05:38.055236] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:05:38.651432] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-07. [2025-10-20T17:05:39.451415] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-05. [2025-10-20T17:05:39.517587] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:05:40.176398] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T17:05:40.251346] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-08. [2025-10-20T17:05:40.508191] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T17:05:41.051381] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-06. [2025-10-20T17:05:41.531791] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T17:05:41.851369] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-07. [2025-10-20T17:05:41.939269] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T17:05:42.651458] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-02. [2025-10-20T17:05:43.451374] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-07. [2025-10-20T17:05:44.004370] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T17:05:44.051918] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-07 (2.20 seconds). [2025-10-20T17:05:44.251387] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-08. [2025-10-20T17:05:44.482734] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T17:05:45.051426] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-09. [2025-10-20T17:05:45.852126] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-03. [2025-10-20T17:05:46.012801] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T17:05:46.651457] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-08. [2025-10-20T17:05:46.789901] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:05:47.326831] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T17:05:47.451386] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-09. [2025-10-20T17:05:47.826323] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T17:05:48.251406] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-04. [2025-10-20T17:05:49.051375] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-10. [2025-10-20T17:05:49.518258] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T17:05:49.851395] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-10. [2025-10-20T17:05:50.190698] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T17:05:50.651394] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-09. [2025-10-20T17:05:50.772761] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T17:05:50.779370] [INFO ] [REMOTE ] AGCO: Full fetch complete (61605 records). [2025-10-20T17:05:50.779576] [DEBUG] [VALIDATOR ] AGCO: Validating primary data. [2025-10-20T17:05:51.185366] [DEBUG] [VALIDATOR ] AGCO: All expected trading days are present. [2025-10-20T17:05:51.185491] [DEBUG] [VALIDATOR ] AGCO: Primary data validation passed. [2025-10-20T17:05:51.186130] [INFO ] [MAIN ] AGCO: Fetched primary data is valid. [2025-10-20T17:05:51.186194] [DEBUG] [LOCAL ] AGCO: Saving primary data. [2025-10-20T17:05:51.187512] [DEBUG] [LOCAL ] AGCO: Trimmed 61605 records to 42303 (keeping 200 days of data) [2025-10-20T17:05:51.319511] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T17:05:51.453266] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-05. [2025-10-20T17:05:51.470459] [DEBUG] [LOCAL ] AGCO: Primary data saved (42303 records). [2025-10-20T17:05:51.470731] [INFO ] [MAIN ] AGCO: Full history saved. [2025-10-20T17:05:51.470853] [INFO ] [MAIN ] Update Progress: 51/2367 symbols completed. AGCO took 30.4s. Estimated remaining: 4h 22m 41s. Rate: 8.8 symbols/min [2025-10-20T17:05:51.664746] [INFO ] [MAIN ] AGL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:51.714930] [DEBUG] [LOCAL ] AGL: Loaded 53357 existing records from primary storage. [2025-10-20T17:05:51.715262] [INFO ] [MAIN ] AGL: Loaded 53357 existing records. [2025-10-20T17:05:51.900301] [WARN ] [MAIN ] AGL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:51.900567] [INFO ] [MAIN ] AGL: Wiping existing primary data before full refresh. [2025-10-20T17:05:51.900677] [INFO ] [LOCAL ] AGL: Starting deletion of primary data. [2025-10-20T17:05:51.901052] [DEBUG] [LOCAL ] AGL: Deleted primary data file. [2025-10-20T17:05:51.901259] [INFO ] [MAIN ] AGL: Starting full primary refresh. [2025-10-20T17:05:52.253548] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-10 (2.40 seconds). [2025-10-20T17:05:52.265685] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-10. [2025-10-20T17:05:52.320617] [INFO ] [REMOTE ] AG: Full fetch complete (77784 records). [2025-10-20T17:05:52.328214] [INFO ] [REMOTE ] AGL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:52.328591] [DEBUG] [VALIDATOR ] AG: Validating primary data. [2025-10-20T17:05:52.737382] [DEBUG] [VALIDATOR ] AG: All expected trading days are present. [2025-10-20T17:05:52.737478] [DEBUG] [VALIDATOR ] AG: Primary data validation passed. [2025-10-20T17:05:52.738323] [INFO ] [MAIN ] AG: Fetched primary data is valid. [2025-10-20T17:05:52.738392] [DEBUG] [LOCAL ] AG: Saving primary data. [2025-10-20T17:05:52.739991] [DEBUG] [LOCAL ] AG: Trimmed 77784 records to 53644 (keeping 200 days of data) [2025-10-20T17:05:53.059074] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-01. [2025-10-20T17:05:53.084782] [DEBUG] [LOCAL ] AG: Primary data saved (53644 records). [2025-10-20T17:05:53.085018] [INFO ] [MAIN ] AG: Full history saved. [2025-10-20T17:05:53.085986] [INFO ] [MAIN ] Update Progress: 52/2367 symbols completed. AG took 40.0s. Estimated remaining: 4h 18m 43s. Rate: 8.9 symbols/min [2025-10-20T17:05:53.284523] [INFO ] [MAIN ] AGM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:53.307380] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T17:05:53.312407] [DEBUG] [LOCAL ] AGM: Loaded 10847 existing records from primary storage. [2025-10-20T17:05:53.317240] [INFO ] [MAIN ] AGM: Loaded 10847 existing records. [2025-10-20T17:05:53.332910] [INFO ] [REMOTE ] AGD: Full fetch complete (26597 records). [2025-10-20T17:05:53.349947] [DEBUG] [VALIDATOR ] AGD: Validating primary data. [2025-10-20T17:05:53.642452] [WARN ] [MAIN ] AGM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:53.655190] [INFO ] [MAIN ] AGM: Wiping existing primary data before full refresh. [2025-10-20T17:05:53.655691] [INFO ] [LOCAL ] AGM: Starting deletion of primary data. [2025-10-20T17:05:53.666303] [DEBUG] [LOCAL ] AGM: Deleted primary data file. [2025-10-20T17:05:53.707965] [INFO ] [MAIN ] AGM: Starting full primary refresh. [2025-10-20T17:05:54.165412] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-05 (2.71 seconds). [2025-10-20T17:05:54.188996] [DEBUG] [VALIDATOR ] AGD: All expected trading days are present. [2025-10-20T17:05:54.254207] [DEBUG] [VALIDATOR ] AGD: Primary data validation passed. [2025-10-20T17:05:54.304022] [INFO ] [MAIN ] AGD: Fetched primary data is valid. [2025-10-20T17:05:54.339058] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-06. [2025-10-20T17:05:54.340047] [DEBUG] [LOCAL ] AGD: Saving primary data. [2025-10-20T17:05:54.391704] [DEBUG] [LOCAL ] AGD: Trimmed 26597 records to 19307 (keeping 200 days of data) [2025-10-20T17:05:54.604375] [DEBUG] [LOCAL ] AGD: Primary data saved (19307 records). [2025-10-20T17:05:54.620142] [INFO ] [MAIN ] AGD: Full history saved. [2025-10-20T17:05:54.630923] [INFO ] [MAIN ] Update Progress: 53/2367 symbols completed. AGD took 25.4s. Estimated remaining: 4h 14m 51s. Rate: 9.1 symbols/min [2025-10-20T17:05:54.690256] [INFO ] [REMOTE ] AGM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:54.690622] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-01. [2025-10-20T17:05:54.900812] [INFO ] [MAIN ] AGO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:54.899819] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T17:05:54.938325] [DEBUG] [LOCAL ] AGO: Loaded 28374 existing records from primary storage. [2025-10-20T17:05:54.938621] [INFO ] [MAIN ] AGO: Loaded 28374 existing records. [2025-10-20T17:05:55.114466] [WARN ] [MAIN ] AGO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:55.114702] [INFO ] [MAIN ] AGO: Wiping existing primary data before full refresh. [2025-10-20T17:05:55.114757] [INFO ] [LOCAL ] AGO: Starting deletion of primary data. [2025-10-20T17:05:55.114929] [DEBUG] [LOCAL ] AGO: Deleted primary data file. [2025-10-20T17:05:55.114965] [INFO ] [MAIN ] AGO: Starting full primary refresh. [2025-10-20T17:05:55.475280] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-02. [2025-10-20T17:05:55.494045] [INFO ] [REMOTE ] AGO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:55.494767] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T17:05:56.263471] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-01. [2025-10-20T17:05:56.272181] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T17:05:57.051381] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-02. [2025-10-20T17:05:57.156594] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T17:05:57.305638] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:05:57.518725] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T17:05:57.851418] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-02. [2025-10-20T17:05:58.651389] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-03. [2025-10-20T17:05:58.930075] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:05:59.451420] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-03. [2025-10-20T17:06:00.120831] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T17:06:00.251373] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-04. [2025-10-20T17:06:00.404081] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:06:01.051374] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-04. [2025-10-20T17:06:01.853237] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-03. [2025-10-20T17:06:01.859545] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:06:02.586103] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T17:06:02.600939] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T17:06:02.651374] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-04. [2025-10-20T17:06:03.389208] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T17:06:03.451331] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-05. [2025-10-20T17:06:04.141668] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T17:06:04.251373] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-06. [2025-10-20T17:06:05.015621] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T17:06:05.051382] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-05. [2025-10-20T17:06:05.851376] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-07. [2025-10-20T17:06:06.272402] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T17:06:06.651363] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-05. [2025-10-20T17:06:06.793968] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:06:07.451401] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-07. [2025-10-20T17:06:08.055142] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:06:08.251374] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-06. [2025-10-20T17:06:09.051379] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-08. [2025-10-20T17:06:09.224750] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T17:06:09.851353] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-08. [2025-10-20T17:06:09.897966] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T17:06:09.926326] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T17:06:10.651378] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-06. [2025-10-20T17:06:11.451347] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-09. [2025-10-20T17:06:11.789732] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T17:06:12.012376] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T17:06:12.034956] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T17:06:12.251482] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-07. [2025-10-20T17:06:13.051389] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-10. [2025-10-20T17:06:13.567587] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T17:06:13.833562] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T17:06:13.837089] [INFO ] [REMOTE ] AGM: Full fetch complete (13325 records). [2025-10-20T17:06:13.837279] [DEBUG] [VALIDATOR ] AGM: Validating primary data. [2025-10-20T17:06:13.858887] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-07. [2025-10-20T17:06:14.199951] [DEBUG] [VALIDATOR ] AGM: All expected trading days are present. [2025-10-20T17:06:14.200037] [DEBUG] [VALIDATOR ] AGM: Primary data validation passed. [2025-10-20T17:06:14.200290] [INFO ] [MAIN ] AGM: Fetched primary data is valid. [2025-10-20T17:06:14.200327] [DEBUG] [LOCAL ] AGM: Saving primary data. [2025-10-20T17:06:14.200839] [DEBUG] [LOCAL ] AGM: Trimmed 13325 records to 10702 (keeping 200 days of data) [2025-10-20T17:06:14.266526] [DEBUG] [LOCAL ] AGM: Primary data saved (10702 records). [2025-10-20T17:06:14.266698] [INFO ] [MAIN ] AGM: Full history saved. [2025-10-20T17:06:14.266788] [INFO ] [MAIN ] Update Progress: 54/2367 symbols completed. AGM took 21.2s. Estimated remaining: 4h 24m 2s. Rate: 8.8 symbols/min [2025-10-20T17:06:14.435791] [INFO ] [MAIN ] AGRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:14.467985] [DEBUG] [LOCAL ] AGRO: Loaded 33606 existing records from primary storage. [2025-10-20T17:06:14.468264] [INFO ] [MAIN ] AGRO: Loaded 33606 existing records. [2025-10-20T17:06:14.641598] [WARN ] [MAIN ] AGRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:14.641816] [INFO ] [MAIN ] AGRO: Wiping existing primary data before full refresh. [2025-10-20T17:06:14.641863] [INFO ] [LOCAL ] AGRO: Starting deletion of primary data. [2025-10-20T17:06:14.642032] [DEBUG] [LOCAL ] AGRO: Deleted primary data file. [2025-10-20T17:06:14.642069] [INFO ] [MAIN ] AGRO: Starting full primary refresh. [2025-10-20T17:06:14.656440] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-09. [2025-10-20T17:06:14.994235] [INFO ] [REMOTE ] AGRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:15.118836] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T17:06:15.451398] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-08. [2025-10-20T17:06:16.251431] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-08. [2025-10-20T17:06:16.685752] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T17:06:17.051398] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-10. [2025-10-20T17:06:17.133550] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:06:17.512025] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:06:17.851403] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-01. [2025-10-20T17:06:18.651421] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-09. [2025-10-20T17:06:18.685839] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:06:18.694308] [INFO ] [REMOTE ] AGI: Full fetch complete (77306 records). [2025-10-20T17:06:18.694494] [DEBUG] [VALIDATOR ] AGI: Validating primary data. [2025-10-20T17:06:19.111524] [DEBUG] [VALIDATOR ] AGI: All expected trading days are present. [2025-10-20T17:06:19.111622] [DEBUG] [VALIDATOR ] AGI: Primary data validation passed. [2025-10-20T17:06:19.112390] [INFO ] [MAIN ] AGI: Fetched primary data is valid. [2025-10-20T17:06:19.112447] [DEBUG] [LOCAL ] AGI: Saving primary data. [2025-10-20T17:06:19.113912] [DEBUG] [LOCAL ] AGI: Trimmed 77306 records to 53423 (keeping 200 days of data) [2025-10-20T17:06:19.349379] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:06:19.451827] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-02. [2025-10-20T17:06:19.471367] [DEBUG] [LOCAL ] AGI: Primary data saved (53423 records). [2025-10-20T17:06:19.471519] [INFO ] [MAIN ] AGI: Full history saved. [2025-10-20T17:06:19.471592] [INFO ] [MAIN ] Update Progress: 55/2367 symbols completed. AGI took 45.3s. Estimated remaining: 4h 22m 46s. Rate: 8.8 symbols/min [2025-10-20T17:06:19.645732] [INFO ] [MAIN ] AGX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:19.677624] [DEBUG] [LOCAL ] AGX: Loaded 33316 existing records from primary storage. [2025-10-20T17:06:19.677932] [INFO ] [MAIN ] AGX: Loaded 33316 existing records. [2025-10-20T17:06:19.853565] [WARN ] [MAIN ] AGX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:19.853841] [INFO ] [MAIN ] AGX: Wiping existing primary data before full refresh. [2025-10-20T17:06:19.853903] [INFO ] [LOCAL ] AGX: Starting deletion of primary data. [2025-10-20T17:06:19.854195] [DEBUG] [LOCAL ] AGX: Deleted primary data file. [2025-10-20T17:06:19.854269] [INFO ] [MAIN ] AGX: Starting full primary refresh. [2025-10-20T17:06:20.224326] [INFO ] [REMOTE ] AGX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:20.254195] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-01. [2025-10-20T17:06:20.257301] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T17:06:20.413494] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T17:06:21.051359] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-09. [2025-10-20T17:06:21.851435] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-03. [2025-10-20T17:06:22.040602] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:06:22.092380] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:06:22.651413] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-02. [2025-10-20T17:06:23.329624] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T17:06:23.451392] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-10. [2025-10-20T17:06:24.098738] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T17:06:24.251477] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-10. [2025-10-20T17:06:24.311576] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T17:06:24.316635] [INFO ] [REMOTE ] AGO: Full fetch complete (39782 records). [2025-10-20T17:06:24.316828] [DEBUG] [VALIDATOR ] AGO: Validating primary data. [2025-10-20T17:06:24.733713] [DEBUG] [VALIDATOR ] AGO: All expected trading days are present. [2025-10-20T17:06:24.733827] [DEBUG] [VALIDATOR ] AGO: Primary data validation passed. [2025-10-20T17:06:24.734307] [INFO ] [MAIN ] AGO: Fetched primary data is valid. [2025-10-20T17:06:24.734352] [DEBUG] [LOCAL ] AGO: Saving primary data. [2025-10-20T17:06:24.735377] [DEBUG] [LOCAL ] AGO: Trimmed 39782 records to 27832 (keeping 200 days of data) [2025-10-20T17:06:24.900415] [DEBUG] [LOCAL ] AGO: Primary data saved (27832 records). [2025-10-20T17:06:24.900566] [INFO ] [MAIN ] AGO: Full history saved. [2025-10-20T17:06:24.901279] [INFO ] [MAIN ] Update Progress: 56/2367 symbols completed. AGO took 30.2s. Estimated remaining: 4h 21m 42s. Rate: 8.8 symbols/min [2025-10-20T17:06:25.069026] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-03. [2025-10-20T17:06:25.073391] [INFO ] [MAIN ] AHH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:25.113253] [DEBUG] [LOCAL ] AHH: Loaded 41118 existing records from primary storage. [2025-10-20T17:06:25.113711] [INFO ] [MAIN ] AHH: Loaded 41118 existing records. [2025-10-20T17:06:25.296712] [WARN ] [MAIN ] AHH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:25.296977] [INFO ] [MAIN ] AHH: Wiping existing primary data before full refresh. [2025-10-20T17:06:25.297030] [INFO ] [LOCAL ] AHH: Starting deletion of primary data. [2025-10-20T17:06:25.297240] [DEBUG] [LOCAL ] AHH: Deleted primary data file. [2025-10-20T17:06:25.297279] [INFO ] [MAIN ] AHH: Starting full primary refresh. [2025-10-20T17:06:25.657936] [INFO ] [REMOTE ] AHH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:25.851420] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-04. [2025-10-20T17:06:25.884869] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:06:25.893930] [INFO ] [REMOTE ] AGL: Full fetch complete (75529 records). [2025-10-20T17:06:25.894119] [DEBUG] [VALIDATOR ] AGL: Validating primary data. [2025-10-20T17:06:25.894985] [FATAL] [VALIDATOR ] AGL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:06:25.895081] [FATAL] [MAIN ] AGL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:06:25.895125] [WARN ] [MAIN ] AGL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:06:25.895191] [INFO ] [LOCAL ] AGL: Starting deletion of primary data. [2025-10-20T17:06:25.895257] [DEBUG] [LOCAL ] AGL: No primary data file found to delete. [2025-10-20T17:06:25.895292] [INFO ] [LOCAL ] AGL: Starting deletion of derivative data. [2025-10-20T17:06:25.895478] [DEBUG] [LOCAL ] AGL: Deleted derivative data file. [2025-10-20T17:06:25.895569] [INFO ] [MAIN ] Update Progress: 57/2367 symbols completed. AGL took 34.4s. Estimated remaining: 4h 17m 40s. Rate: 9.0 symbols/min [2025-10-20T17:06:26.065097] [INFO ] [MAIN ] AHL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:26.065310] [DEBUG] [LOCAL ] AHL: No existing data file found in primary storage. [2025-10-20T17:06:26.065348] [WARN ] [MAIN ] AHL: No existing data found, performing full refresh instead. [2025-10-20T17:06:26.065468] [INFO ] [MAIN ] AHL: Wiping existing primary data before full refresh. [2025-10-20T17:06:26.065511] [INFO ] [LOCAL ] AHL: Starting deletion of primary data. [2025-10-20T17:06:26.065554] [DEBUG] [LOCAL ] AHL: No primary data file found to delete. [2025-10-20T17:06:26.065587] [INFO ] [MAIN ] AHL: Starting full primary refresh. [2025-10-20T17:06:26.432476] [INFO ] [REMOTE ] AHL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:26.635007] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T17:06:26.651350] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-04. [2025-10-20T17:06:27.456527] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-01. [2025-10-20T17:06:27.475148] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T17:06:28.125842] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:06:28.251395] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-05. [2025-10-20T17:06:28.725281] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:06:29.051407] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-01. [2025-10-20T17:06:29.245377] [DEBUG] [REMOTE ] AHL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:06:29.245711] [WARN ] [REMOTE ] AHL: No data returned for month 2025-01 [2025-10-20T17:06:29.245798] [FATAL] [REMOTE ] AHL: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:06:29.245844] [FATAL] [MAIN ] AHL: Failed to fetch primary data. [2025-10-20T17:06:29.245900] [WARN ] [MAIN ] AHL: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:06:29.245972] [INFO ] [LOCAL ] AHL: Starting deletion of primary data. [2025-10-20T17:06:29.246036] [DEBUG] [LOCAL ] AHL: No primary data file found to delete. [2025-10-20T17:06:29.246072] [INFO ] [LOCAL ] AHL: Starting deletion of derivative data. [2025-10-20T17:06:29.246111] [DEBUG] [LOCAL ] AHL: No derivative data file found to delete. [2025-10-20T17:06:29.246155] [INFO ] [MAIN ] Update Progress: 58/2367 symbols completed. AHL took 3.4s. Estimated remaining: 4h 15m 21s. Rate: 9.0 symbols/min [2025-10-20T17:06:29.424517] [INFO ] [MAIN ] AHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:29.424805] [DEBUG] [LOCAL ] AHR: No existing data file found in primary storage. [2025-10-20T17:06:29.424857] [WARN ] [MAIN ] AHR: No existing data found, performing full refresh instead. [2025-10-20T17:06:29.424969] [INFO ] [MAIN ] AHR: Wiping existing primary data before full refresh. [2025-10-20T17:06:29.425011] [INFO ] [LOCAL ] AHR: Starting deletion of primary data. [2025-10-20T17:06:29.425052] [DEBUG] [LOCAL ] AHR: No primary data file found to delete. [2025-10-20T17:06:29.425083] [INFO ] [MAIN ] AHR: Starting full primary refresh. [2025-10-20T17:06:29.786554] [INFO ] [REMOTE ] AHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:29.807278] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:06:29.851360] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-06. [2025-10-20T17:06:30.651399] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-01. [2025-10-20T17:06:31.401524] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:06:31.451357] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-07. [2025-10-20T17:06:32.122956] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T17:06:32.251389] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-02. [2025-10-20T17:06:32.573716] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T17:06:33.051381] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-05. [2025-10-20T17:06:33.714221] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:06:33.851344] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-03. [2025-10-20T17:06:34.279206] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T17:06:34.651399] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-08. [2025-10-20T17:06:35.225416] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:06:35.451378] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-04. [2025-10-20T17:06:36.016953] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T17:06:36.251413] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-09. [2025-10-20T17:06:36.703616] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T17:06:37.051429] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-02. [2025-10-20T17:06:37.836846] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T17:06:37.851375] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-10. [2025-10-20T17:06:38.344468] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T17:06:38.651448] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-05. [2025-10-20T17:06:39.327611] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T17:06:39.332505] [INFO ] [REMOTE ] AGX: Full fetch complete (48956 records). [2025-10-20T17:06:39.332674] [DEBUG] [VALIDATOR ] AGX: Validating primary data. [2025-10-20T17:06:39.456555] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-06. [2025-10-20T17:06:39.769001] [DEBUG] [VALIDATOR ] AGX: All expected trading days are present. [2025-10-20T17:06:39.769097] [DEBUG] [VALIDATOR ] AGX: Primary data validation passed. [2025-10-20T17:06:39.769685] [INFO ] [MAIN ] AGX: Fetched primary data is valid. [2025-10-20T17:06:39.769753] [DEBUG] [LOCAL ] AGX: Saving primary data. [2025-10-20T17:06:39.771167] [DEBUG] [LOCAL ] AGX: Trimmed 48956 records to 32688 (keeping 200 days of data) [2025-10-20T17:06:39.979872] [DEBUG] [LOCAL ] AGX: Primary data saved (32688 records). [2025-10-20T17:06:39.980073] [INFO ] [MAIN ] AGX: Full history saved. [2025-10-20T17:06:39.980964] [INFO ] [MAIN ] Update Progress: 59/2367 symbols completed. AGX took 20.5s. Estimated remaining: 4h 17m 54s. Rate: 8.9 symbols/min [2025-10-20T17:06:40.159909] [INFO ] [MAIN ] AHT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:40.198690] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T17:06:40.214078] [DEBUG] [LOCAL ] AHT: Loaded 6493 existing records from primary storage. [2025-10-20T17:06:40.215539] [INFO ] [MAIN ] AHT: Loaded 6493 existing records. [2025-10-20T17:06:40.259075] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-06. [2025-10-20T17:06:40.405834] [WARN ] [MAIN ] AHT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:40.406062] [INFO ] [MAIN ] AHT: Wiping existing primary data before full refresh. [2025-10-20T17:06:40.406110] [INFO ] [LOCAL ] AHT: Starting deletion of primary data. [2025-10-20T17:06:40.406267] [DEBUG] [LOCAL ] AHT: Deleted primary data file. [2025-10-20T17:06:40.406323] [INFO ] [MAIN ] AHT: Starting full primary refresh. [2025-10-20T17:06:40.759246] [INFO ] [REMOTE ] AHT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:40.784819] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:06:41.051408] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-03. [2025-10-20T17:06:41.847899] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T17:06:41.858302] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-07. [2025-10-20T17:06:42.651429] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-01. [2025-10-20T17:06:43.159506] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T17:06:43.298188] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:06:43.451352] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-04. [2025-10-20T17:06:43.524973] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T17:06:44.251414] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-07. [2025-10-20T17:06:45.051379] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-08. [2025-10-20T17:06:45.112173] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T17:06:45.851379] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-02. [2025-10-20T17:06:46.043100] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:06:46.440344] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:06:46.568726] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T17:06:46.651433] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-03. [2025-10-20T17:06:47.105093] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T17:06:47.451600] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-08. [2025-10-20T17:06:48.251441] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-04. [2025-10-20T17:06:48.781477] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-10-20T17:06:48.980279] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:06:49.051438] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-05. [2025-10-20T17:06:49.751095] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:06:49.851379] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-06. [2025-10-20T17:06:50.581894] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T17:06:50.651357] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-09. [2025-10-20T17:06:51.451402] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-05. [2025-10-20T17:06:51.998574] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:06:52.251425] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-10. [2025-10-20T17:06:52.714442] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T17:06:53.051479] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-06. [2025-10-20T17:06:53.638699] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:06:53.643484] [INFO ] [REMOTE ] AGRO: Full fetch complete (48855 records). [2025-10-20T17:06:53.643604] [DEBUG] [VALIDATOR ] AGRO: Validating primary data. [2025-10-20T17:06:53.644203] [FATAL] [VALIDATOR ] AGRO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:06:53.644289] [FATAL] [MAIN ] AGRO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:06:53.644331] [WARN ] [MAIN ] AGRO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:06:53.644369] [INFO ] [LOCAL ] AGRO: Starting deletion of primary data. [2025-10-20T17:06:53.644425] [DEBUG] [LOCAL ] AGRO: No primary data file found to delete. [2025-10-20T17:06:53.644458] [INFO ] [LOCAL ] AGRO: Starting deletion of derivative data. [2025-10-20T17:06:53.644711] [DEBUG] [LOCAL ] AGRO: Deleted derivative data file. [2025-10-20T17:06:53.644938] [INFO ] [MAIN ] Update Progress: 60/2367 symbols completed. AGRO took 39.4s. Estimated remaining: 4h 22m 15s. Rate: 8.8 symbols/min [2025-10-20T17:06:53.823963] [INFO ] [MAIN ] AI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:53.856387] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-09. [2025-10-20T17:06:53.880173] [DEBUG] [LOCAL ] AI: Loaded 54545 existing records from primary storage. [2025-10-20T17:06:53.880513] [INFO ] [MAIN ] AI: Loaded 54545 existing records. [2025-10-20T17:06:54.067006] [WARN ] [MAIN ] AI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:54.067613] [INFO ] [MAIN ] AI: Wiping existing primary data before full refresh. [2025-10-20T17:06:54.067694] [INFO ] [LOCAL ] AI: Starting deletion of primary data. [2025-10-20T17:06:54.067919] [DEBUG] [LOCAL ] AI: Deleted primary data file. [2025-10-20T17:06:54.067974] [INFO ] [MAIN ] AI: Starting full primary refresh. [2025-10-20T17:06:54.437699] [INFO ] [REMOTE ] AI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:54.460196] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T17:06:54.651345] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-01. [2025-10-20T17:06:55.451417] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-07. [2025-10-20T17:06:55.700238] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T17:06:56.251414] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-10. [2025-10-20T17:06:56.608289] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:06:56.949489] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-01 (2.30 seconds). [2025-10-20T17:06:57.051368] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-02. [2025-10-20T17:06:57.740256] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:06:57.747256] [INFO ] [REMOTE ] AHR: Full fetch complete (70806 records). [2025-10-20T17:06:57.747363] [DEBUG] [VALIDATOR ] AHR: Validating primary data. [2025-10-20T17:06:57.856537] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-08. [2025-10-20T17:06:58.130448] [DEBUG] [VALIDATOR ] AHR: All expected trading days are present. [2025-10-20T17:06:58.130521] [DEBUG] [VALIDATOR ] AHR: Primary data validation passed. [2025-10-20T17:06:58.131206] [INFO ] [MAIN ] AHR: Fetched primary data is valid. [2025-10-20T17:06:58.131267] [DEBUG] [LOCAL ] AHR: Saving primary data. [2025-10-20T17:06:58.132503] [DEBUG] [LOCAL ] AHR: Trimmed 70806 records to 49006 (keeping 200 days of data) [2025-10-20T17:06:58.418524] [DEBUG] [LOCAL ] AHR: Primary data saved (49006 records). [2025-10-20T17:06:58.418621] [INFO ] [MAIN ] AHR: Full history saved. [2025-10-20T17:06:58.418728] [INFO ] [MAIN ] Update Progress: 61/2367 symbols completed. AHR took 29.2s. Estimated remaining: 4h 20m 51s. Rate: 8.8 symbols/min [2025-10-20T17:06:58.582732] [INFO ] [MAIN ] AIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:58.629932] [DEBUG] [LOCAL ] AIG: Loaded 54247 existing records from primary storage. [2025-10-20T17:06:58.630246] [INFO ] [MAIN ] AIG: Loaded 54247 existing records. [2025-10-20T17:06:58.661050] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-07. [2025-10-20T17:06:58.836731] [WARN ] [MAIN ] AIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:58.836987] [INFO ] [MAIN ] AIG: Wiping existing primary data before full refresh. [2025-10-20T17:06:58.837040] [INFO ] [LOCAL ] AIG: Starting deletion of primary data. [2025-10-20T17:06:58.837383] [DEBUG] [LOCAL ] AIG: Deleted primary data file. [2025-10-20T17:06:58.837440] [INFO ] [MAIN ] AIG: Starting full primary refresh. [2025-10-20T17:06:59.218207] [INFO ] [REMOTE ] AIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:59.271943] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T17:06:59.281733] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-02 (2.23 seconds). [2025-10-20T17:06:59.451385] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-09. [2025-10-20T17:06:59.460705] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T17:07:00.251387] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-08. [2025-10-20T17:07:00.398215] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T17:07:01.003182] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T17:07:01.051385] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-09. [2025-10-20T17:07:01.718959] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T17:07:01.851352] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-10. [2025-10-20T17:07:02.651372] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-01. [2025-10-20T17:07:03.451426] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-10. [2025-10-20T17:07:03.522802] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T17:07:03.528529] [INFO ] [REMOTE ] AHH: Full fetch complete (59561 records). [2025-10-20T17:07:03.528662] [DEBUG] [VALIDATOR ] AHH: Validating primary data. [2025-10-20T17:07:03.916743] [DEBUG] [VALIDATOR ] AHH: All expected trading days are present. [2025-10-20T17:07:03.916844] [DEBUG] [VALIDATOR ] AHH: Primary data validation passed. [2025-10-20T17:07:03.917498] [INFO ] [MAIN ] AHH: Fetched primary data is valid. [2025-10-20T17:07:03.917562] [DEBUG] [LOCAL ] AHH: Saving primary data. [2025-10-20T17:07:03.918903] [DEBUG] [LOCAL ] AHH: Trimmed 59561 records to 40379 (keeping 200 days of data) [2025-10-20T17:07:04.159910] [DEBUG] [LOCAL ] AHH: Primary data saved (40379 records). [2025-10-20T17:07:04.160074] [INFO ] [MAIN ] AHH: Full history saved. [2025-10-20T17:07:04.160939] [INFO ] [MAIN ] Update Progress: 62/2367 symbols completed. AHH took 39.3s. Estimated remaining: 4h 20m 5s. Rate: 8.9 symbols/min [2025-10-20T17:07:04.256613] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-03. [2025-10-20T17:07:04.370690] [INFO ] [MAIN ] AII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:04.371037] [DEBUG] [LOCAL ] AII: No existing data file found in primary storage. [2025-10-20T17:07:04.371084] [WARN ] [MAIN ] AII: No existing data found, performing full refresh instead. [2025-10-20T17:07:04.371200] [INFO ] [MAIN ] AII: Wiping existing primary data before full refresh. [2025-10-20T17:07:04.371242] [INFO ] [LOCAL ] AII: Starting deletion of primary data. [2025-10-20T17:07:04.371283] [DEBUG] [LOCAL ] AII: No primary data file found to delete. [2025-10-20T17:07:04.371317] [INFO ] [MAIN ] AII: Starting full primary refresh. [2025-10-20T17:07:04.679908] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T17:07:04.716387] [INFO ] [REMOTE ] AHT: Full fetch complete (9114 records). [2025-10-20T17:07:04.753454] [DEBUG] [VALIDATOR ] AHT: Validating primary data. [2025-10-20T17:07:04.759199] [INFO ] [REMOTE ] AII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:05.063993] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-01. [2025-10-20T17:07:05.112310] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-01 (2.46 seconds). [2025-10-20T17:07:05.200332] [DEBUG] [VALIDATOR ] AHT: All expected trading days are present. [2025-10-20T17:07:05.200407] [DEBUG] [VALIDATOR ] AHT: Primary data validation passed. [2025-10-20T17:07:05.200638] [INFO ] [MAIN ] AHT: Fetched primary data is valid. [2025-10-20T17:07:05.200718] [DEBUG] [LOCAL ] AHT: Saving primary data. [2025-10-20T17:07:05.201123] [DEBUG] [LOCAL ] AHT: Trimmed 9114 records to 6438 (keeping 200 days of data) [2025-10-20T17:07:05.246870] [DEBUG] [LOCAL ] AHT: Primary data saved (6438 records). [2025-10-20T17:07:05.247085] [INFO ] [MAIN ] AHT: Full history saved. [2025-10-20T17:07:05.247197] [INFO ] [MAIN ] Update Progress: 63/2367 symbols completed. AHT took 25.3s. Estimated remaining: 4h 16m 31s. Rate: 9.0 symbols/min [2025-10-20T17:07:05.344584] [DEBUG] [REMOTE ] AII: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:07:05.357195] [WARN ] [REMOTE ] AII: No data returned for month 2025-01 [2025-10-20T17:07:05.357359] [FATAL] [REMOTE ] AII: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:07:05.357414] [FATAL] [MAIN ] AII: Failed to fetch primary data. [2025-10-20T17:07:05.357977] [WARN ] [MAIN ] AII: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:07:05.368746] [INFO ] [LOCAL ] AII: Starting deletion of primary data. [2025-10-20T17:07:05.384253] [DEBUG] [LOCAL ] AII: No primary data file found to delete. [2025-10-20T17:07:05.399366] [INFO ] [LOCAL ] AII: Starting deletion of derivative data. [2025-10-20T17:07:05.400801] [DEBUG] [LOCAL ] AII: No derivative data file found to delete. [2025-10-20T17:07:05.400898] [INFO ] [MAIN ] Update Progress: 64/2367 symbols completed. AII took 1.2s. Estimated remaining: 4h 12m 29s. Rate: 9.1 symbols/min [2025-10-20T17:07:05.506767] [INFO ] [MAIN ] AIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:05.628015] [INFO ] [MAIN ] AIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:05.639690] [DEBUG] [LOCAL ] AIN: Loaded 24822 existing records from primary storage. [2025-10-20T17:07:05.640072] [INFO ] [MAIN ] AIN: Loaded 24822 existing records. [2025-10-20T17:07:05.757769] [DEBUG] [LOCAL ] AIO: Loaded 13513 existing records from primary storage. [2025-10-20T17:07:05.761775] [INFO ] [MAIN ] AIO: Loaded 13513 existing records. [2025-10-20T17:07:05.861698] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-02. [2025-10-20T17:07:05.960858] [WARN ] [MAIN ] AIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:05.971987] [INFO ] [MAIN ] AIN: Wiping existing primary data before full refresh. [2025-10-20T17:07:05.972610] [INFO ] [LOCAL ] AIN: Starting deletion of primary data. [2025-10-20T17:07:06.007971] [DEBUG] [LOCAL ] AIN: Deleted primary data file. [2025-10-20T17:07:06.017547] [INFO ] [MAIN ] AIN: Starting full primary refresh. [2025-10-20T17:07:06.117199] [WARN ] [MAIN ] AIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:06.134382] [INFO ] [MAIN ] AIO: Wiping existing primary data before full refresh. [2025-10-20T17:07:06.145092] [INFO ] [LOCAL ] AIO: Starting deletion of primary data. [2025-10-20T17:07:06.155937] [DEBUG] [LOCAL ] AIO: Deleted primary data file. [2025-10-20T17:07:06.156034] [INFO ] [MAIN ] AIO: Starting full primary refresh. [2025-10-20T17:07:06.713767] [INFO ] [REMOTE ] AIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:06.732035] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-01. [2025-10-20T17:07:06.837951] [INFO ] [REMOTE ] AIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:07.138443] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-03 (2.88 seconds). [2025-10-20T17:07:07.454425] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-01. [2025-10-20T17:07:07.524215] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:07:07.655847] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:07:08.251421] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-04. [2025-10-20T17:07:08.358691] [DEBUG] [REMOTE ] AI: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T17:07:08.359068] [WARN ] [REMOTE ] AI: No data returned for month 2025-04 [2025-10-20T17:07:08.359177] [FATAL] [REMOTE ] AI: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:07:08.359261] [FATAL] [MAIN ] AI: Failed to fetch primary data. [2025-10-20T17:07:08.359296] [WARN ] [MAIN ] AI: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:07:08.359331] [INFO ] [LOCAL ] AI: Starting deletion of primary data. [2025-10-20T17:07:08.359387] [DEBUG] [LOCAL ] AI: No primary data file found to delete. [2025-10-20T17:07:08.359420] [INFO ] [LOCAL ] AI: Starting deletion of derivative data. [2025-10-20T17:07:08.359613] [DEBUG] [LOCAL ] AI: Deleted derivative data file. [2025-10-20T17:07:08.360067] [INFO ] [MAIN ] Update Progress: 65/2367 symbols completed. AI took 14.7s. Estimated remaining: 4h 10m 15s. Rate: 9.2 symbols/min [2025-10-20T17:07:08.552785] [INFO ] [MAIN ] AIR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:08.585523] [DEBUG] [LOCAL ] AIR: Loaded 29335 existing records from primary storage. [2025-10-20T17:07:08.586067] [INFO ] [MAIN ] AIR: Loaded 29335 existing records. [2025-10-20T17:07:08.729314] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:07:08.817495] [WARN ] [MAIN ] AIR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:08.817759] [INFO ] [MAIN ] AIR: Wiping existing primary data before full refresh. [2025-10-20T17:07:08.817816] [INFO ] [LOCAL ] AIR: Starting deletion of primary data. [2025-10-20T17:07:08.818066] [DEBUG] [LOCAL ] AIR: Deleted primary data file. [2025-10-20T17:07:08.818123] [INFO ] [MAIN ] AIR: Starting full primary refresh. [2025-10-20T17:07:09.058261] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-03. [2025-10-20T17:07:09.220428] [INFO ] [REMOTE ] AIR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:09.851379] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-02. [2025-10-20T17:07:10.651361] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-02. [2025-10-20T17:07:10.695732] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T17:07:10.807194] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:07:11.451370] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-03. [2025-10-20T17:07:11.570381] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T17:07:12.251357] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-01. [2025-10-20T17:07:12.560186] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T17:07:13.051423] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-03. [2025-10-20T17:07:13.323629] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T17:07:13.691628] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T17:07:13.851400] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-04. [2025-10-20T17:07:14.651437] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-04. [2025-10-20T17:07:15.399812] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:07:15.451356] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-02. [2025-10-20T17:07:15.761810] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T17:07:16.251373] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-05. [2025-10-20T17:07:16.693050] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T17:07:16.919741] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-10-20T17:07:17.051395] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-04. [2025-10-20T17:07:17.851404] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-03. [2025-10-20T17:07:18.439108] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:07:18.651438] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-05. [2025-10-20T17:07:19.118013] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T17:07:19.451430] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-06. [2025-10-20T17:07:19.584033] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T17:07:20.251351] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-05. [2025-10-20T17:07:20.476810] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T17:07:21.051441] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-07. [2025-10-20T17:07:21.682769] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-10-20T17:07:21.838820] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T17:07:21.851340] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-06. [2025-10-20T17:07:22.651402] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-04. [2025-10-20T17:07:23.451348] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-08. [2025-10-20T17:07:23.639340] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T17:07:23.730267] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T17:07:23.991219] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T17:07:24.251393] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-05. [2025-10-20T17:07:25.051427] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-09. [2025-10-20T17:07:25.101391] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T17:07:25.851388] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-07. [2025-10-20T17:07:26.037468] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T17:07:26.651416] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-10. [2025-10-20T17:07:27.451378] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-06. [2025-10-20T17:07:27.810283] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T17:07:27.811241] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:07:27.827721] [INFO ] [REMOTE ] AIO: Full fetch complete (20099 records). [2025-10-20T17:07:27.827985] [DEBUG] [VALIDATOR ] AIO: Validating primary data. [2025-10-20T17:07:28.219595] [DEBUG] [VALIDATOR ] AIO: All expected trading days are present. [2025-10-20T17:07:28.219705] [DEBUG] [VALIDATOR ] AIO: Primary data validation passed. [2025-10-20T17:07:28.220082] [INFO ] [MAIN ] AIO: Fetched primary data is valid. [2025-10-20T17:07:28.220123] [DEBUG] [LOCAL ] AIO: Saving primary data. [2025-10-20T17:07:28.220723] [DEBUG] [LOCAL ] AIO: Trimmed 20099 records to 13123 (keeping 200 days of data) [2025-10-20T17:07:28.257238] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-08. [2025-10-20T17:07:28.306982] [DEBUG] [LOCAL ] AIO: Primary data saved (13123 records). [2025-10-20T17:07:28.307247] [INFO ] [MAIN ] AIO: Full history saved. [2025-10-20T17:07:28.307362] [INFO ] [MAIN ] Update Progress: 66/2367 symbols completed. AIO took 22.9s. Estimated remaining: 4h 17m 56s. Rate: 8.9 symbols/min [2025-10-20T17:07:28.499545] [INFO ] [MAIN ] AIT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:28.530926] [DEBUG] [LOCAL ] AIT: Loaded 27357 existing records from primary storage. [2025-10-20T17:07:28.531333] [INFO ] [MAIN ] AIT: Loaded 27357 existing records. [2025-10-20T17:07:28.741436] [WARN ] [MAIN ] AIT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:28.757278] [INFO ] [MAIN ] AIT: Wiping existing primary data before full refresh. [2025-10-20T17:07:28.758175] [INFO ] [LOCAL ] AIT: Starting deletion of primary data. [2025-10-20T17:07:28.759041] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:07:28.764706] [DEBUG] [LOCAL ] AIT: Deleted primary data file. [2025-10-20T17:07:28.765437] [INFO ] [MAIN ] AIT: Starting full primary refresh. [2025-10-20T17:07:29.056544] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-07. [2025-10-20T17:07:29.165108] [INFO ] [REMOTE ] AIT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:29.851385] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-06. [2025-10-20T17:07:29.921769] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T17:07:30.249583] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:07:30.656474] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-09. [2025-10-20T17:07:30.664706] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T17:07:31.451471] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-01. [2025-10-20T17:07:32.251381] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-07. [2025-10-20T17:07:32.297345] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T17:07:32.833084] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T17:07:33.051458] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-02. [2025-10-20T17:07:33.484879] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T17:07:33.851438] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-08. [2025-10-20T17:07:33.965357] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T17:07:34.651361] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-08. [2025-10-20T17:07:34.722860] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T17:07:35.451392] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-10. [2025-10-20T17:07:35.941472] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T17:07:36.031745] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T17:07:36.039594] [INFO ] [REMOTE ] AIG: Full fetch complete (77569 records). [2025-10-20T17:07:36.039784] [DEBUG] [VALIDATOR ] AIG: Validating primary data. [2025-10-20T17:07:36.258149] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-09. [2025-10-20T17:07:36.452443] [DEBUG] [VALIDATOR ] AIG: All expected trading days are present. [2025-10-20T17:07:36.452535] [DEBUG] [VALIDATOR ] AIG: Primary data validation passed. [2025-10-20T17:07:36.453420] [INFO ] [MAIN ] AIG: Fetched primary data is valid. [2025-10-20T17:07:36.453488] [DEBUG] [LOCAL ] AIG: Saving primary data. [2025-10-20T17:07:36.455090] [DEBUG] [LOCAL ] AIG: Trimmed 77569 records to 53465 (keeping 200 days of data) [2025-10-20T17:07:36.801690] [DEBUG] [LOCAL ] AIG: Primary data saved (53465 records). [2025-10-20T17:07:36.801884] [INFO ] [MAIN ] AIG: Full history saved. [2025-10-20T17:07:36.801980] [INFO ] [MAIN ] Update Progress: 67/2367 symbols completed. AIG took 38.4s. Estimated remaining: 4h 18m 50s. Rate: 8.9 symbols/min [2025-10-20T17:07:36.993317] [INFO ] [MAIN ] AIV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:37.036915] [DEBUG] [LOCAL ] AIV: Loaded 42194 existing records from primary storage. [2025-10-20T17:07:37.037419] [INFO ] [MAIN ] AIV: Loaded 42194 existing records. [2025-10-20T17:07:37.060116] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-03. [2025-10-20T17:07:37.235567] [WARN ] [MAIN ] AIV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:37.236159] [INFO ] [MAIN ] AIV: Wiping existing primary data before full refresh. [2025-10-20T17:07:37.236395] [INFO ] [LOCAL ] AIV: Starting deletion of primary data. [2025-10-20T17:07:37.236969] [DEBUG] [LOCAL ] AIV: Deleted primary data file. [2025-10-20T17:07:37.237033] [INFO ] [MAIN ] AIV: Starting full primary refresh. [2025-10-20T17:07:37.629739] [INFO ] [REMOTE ] AIV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:37.656276] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T17:07:37.851393] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-01. [2025-10-20T17:07:38.508925] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T17:07:38.651383] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-04. [2025-10-20T17:07:39.074078] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T17:07:39.451408] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-02. [2025-10-20T17:07:40.109365] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:07:40.251394] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-10. [2025-10-20T17:07:40.568226] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:07:41.051397] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-09. [2025-10-20T17:07:41.851368] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-03. [2025-10-20T17:07:41.943453] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:07:41.949266] [INFO ] [REMOTE ] AIR: Full fetch complete (40306 records). [2025-10-20T17:07:41.949466] [DEBUG] [VALIDATOR ] AIR: Validating primary data. [2025-10-20T17:07:42.369451] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T17:07:42.408348] [DEBUG] [VALIDATOR ] AIR: All expected trading days are present. [2025-10-20T17:07:42.408439] [DEBUG] [VALIDATOR ] AIR: Primary data validation passed. [2025-10-20T17:07:42.409085] [INFO ] [MAIN ] AIR: Fetched primary data is valid. [2025-10-20T17:07:42.409152] [DEBUG] [LOCAL ] AIR: Saving primary data. [2025-10-20T17:07:42.410300] [DEBUG] [LOCAL ] AIR: Trimmed 40306 records to 28742 (keeping 200 days of data) [2025-10-20T17:07:42.591942] [DEBUG] [LOCAL ] AIR: Primary data saved (28742 records). [2025-10-20T17:07:42.592125] [INFO ] [MAIN ] AIR: Full history saved. [2025-10-20T17:07:42.592210] [INFO ] [MAIN ] Update Progress: 68/2367 symbols completed. AIR took 34.2s. Estimated remaining: 4h 18m 11s. Rate: 8.9 symbols/min [2025-10-20T17:07:42.656541] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-05. [2025-10-20T17:07:42.765541] [INFO ] [MAIN ] AIZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:42.765752] [DEBUG] [LOCAL ] AIZ: No existing data file found in primary storage. [2025-10-20T17:07:42.765798] [WARN ] [MAIN ] AIZ: No existing data found, performing full refresh instead. [2025-10-20T17:07:42.765909] [INFO ] [MAIN ] AIZ: Wiping existing primary data before full refresh. [2025-10-20T17:07:42.765950] [INFO ] [LOCAL ] AIZ: Starting deletion of primary data. [2025-10-20T17:07:42.765989] [DEBUG] [LOCAL ] AIZ: No primary data file found to delete. [2025-10-20T17:07:42.766021] [INFO ] [MAIN ] AIZ: Starting full primary refresh. [2025-10-20T17:07:43.131488] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T17:07:43.150467] [INFO ] [REMOTE ] AIZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:43.451412] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-04. [2025-10-20T17:07:43.940689] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T17:07:44.251440] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-01. [2025-10-20T17:07:44.925578] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:07:45.051428] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-05. [2025-10-20T17:07:45.209125] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T17:07:45.851374] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-02. [2025-10-20T17:07:46.200898] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T17:07:46.651388] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-06. [2025-10-20T17:07:47.179605] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:07:47.451390] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-10. [2025-10-20T17:07:47.593626] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T17:07:48.251373] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-07. [2025-10-20T17:07:48.475391] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T17:07:48.479188] [INFO ] [REMOTE ] AIN: Full fetch complete (33704 records). [2025-10-20T17:07:48.479261] [DEBUG] [VALIDATOR ] AIN: Validating primary data. [2025-10-20T17:07:48.863408] [DEBUG] [VALIDATOR ] AIN: All expected trading days are present. [2025-10-20T17:07:48.863485] [DEBUG] [VALIDATOR ] AIN: Primary data validation passed. [2025-10-20T17:07:48.863952] [INFO ] [MAIN ] AIN: Fetched primary data is valid. [2025-10-20T17:07:48.864013] [DEBUG] [LOCAL ] AIN: Saving primary data. [2025-10-20T17:07:48.864801] [DEBUG] [LOCAL ] AIN: Trimmed 33704 records to 24466 (keeping 200 days of data) [2025-10-20T17:07:49.008435] [DEBUG] [LOCAL ] AIN: Primary data saved (24466 records). [2025-10-20T17:07:49.008597] [INFO ] [MAIN ] AIN: Full history saved. [2025-10-20T17:07:49.008703] [INFO ] [MAIN ] Update Progress: 69/2367 symbols completed. AIN took 43.8s. Estimated remaining: 4h 17m 53s. Rate: 8.9 symbols/min [2025-10-20T17:07:49.056521] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-03. [2025-10-20T17:07:49.185363] [INFO ] [MAIN ] AIZN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:49.190474] [DEBUG] [LOCAL ] AIZN: Loaded 3995 existing records from primary storage. [2025-10-20T17:07:49.190630] [INFO ] [MAIN ] AIZN: Loaded 3995 existing records. [2025-10-20T17:07:49.377803] [WARN ] [MAIN ] AIZN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:49.378092] [INFO ] [MAIN ] AIZN: Wiping existing primary data before full refresh. [2025-10-20T17:07:49.378154] [INFO ] [LOCAL ] AIZN: Starting deletion of primary data. [2025-10-20T17:07:49.378373] [DEBUG] [LOCAL ] AIZN: Deleted primary data file. [2025-10-20T17:07:49.378419] [INFO ] [MAIN ] AIZN: Starting full primary refresh. [2025-10-20T17:07:49.643699] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:07:49.773396] [INFO ] [REMOTE ] AIZN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:49.851367] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-08. [2025-10-20T17:07:50.599005] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:07:50.651375] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-06. [2025-10-20T17:07:51.408325] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T17:07:51.451351] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-09. [2025-10-20T17:07:51.525486] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T17:07:52.251399] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-04. [2025-10-20T17:07:53.051450] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-07. [2025-10-20T17:07:53.178760] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T17:07:53.472929] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T17:07:53.851395] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-01. [2025-10-20T17:07:54.443687] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:07:54.519679] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T17:07:54.651373] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-10. [2025-10-20T17:07:55.451375] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-05. [2025-10-20T17:07:56.251367] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-08. [2025-10-20T17:07:56.391961] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:07:56.399141] [INFO ] [REMOTE ] AIV: Full fetch complete (60395 records). [2025-10-20T17:07:56.399274] [DEBUG] [VALIDATOR ] AIV: Validating primary data. [2025-10-20T17:07:56.805327] [DEBUG] [VALIDATOR ] AIV: All expected trading days are present. [2025-10-20T17:07:56.805419] [DEBUG] [VALIDATOR ] AIV: Primary data validation passed. [2025-10-20T17:07:56.806065] [INFO ] [MAIN ] AIV: Fetched primary data is valid. [2025-10-20T17:07:56.806129] [DEBUG] [LOCAL ] AIV: Saving primary data. [2025-10-20T17:07:56.807398] [DEBUG] [LOCAL ] AIV: Trimmed 60395 records to 41799 (keeping 200 days of data) [2025-10-20T17:07:57.032260] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T17:07:57.054360] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-02. [2025-10-20T17:07:57.081165] [DEBUG] [LOCAL ] AIV: Primary data saved (41799 records). [2025-10-20T17:07:57.081301] [INFO ] [MAIN ] AIV: Full history saved. [2025-10-20T17:07:57.081382] [INFO ] [MAIN ] Update Progress: 70/2367 symbols completed. AIV took 20.3s. Estimated remaining: 4h 18m 31s. Rate: 8.9 symbols/min [2025-10-20T17:07:57.263511] [INFO ] [MAIN ] AJG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:57.305169] [DEBUG] [LOCAL ] AJG: Loaded 47258 existing records from primary storage. [2025-10-20T17:07:57.305499] [INFO ] [MAIN ] AJG: Loaded 47258 existing records. [2025-10-20T17:07:57.495333] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T17:07:57.501137] [WARN ] [MAIN ] AJG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:57.506781] [INFO ] [MAIN ] AJG: Wiping existing primary data before full refresh. [2025-10-20T17:07:57.507174] [INFO ] [LOCAL ] AJG: Starting deletion of primary data. [2025-10-20T17:07:57.507574] [DEBUG] [LOCAL ] AJG: Deleted primary data file. [2025-10-20T17:07:57.507632] [INFO ] [MAIN ] AJG: Starting full primary refresh. [2025-10-20T17:07:57.831708] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T17:07:57.863933] [INFO ] [REMOTE ] AJG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:57.856961] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-09. [2025-10-20T17:07:58.651380] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-06. [2025-10-20T17:07:58.812785] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T17:07:59.451376] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-10. [2025-10-20T17:07:59.940357] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:08:00.251381] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-07. [2025-10-20T17:08:00.279824] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T17:08:00.285179] [INFO ] [REMOTE ] AIT: Full fetch complete (39190 records). [2025-10-20T17:08:00.285440] [DEBUG] [VALIDATOR ] AIT: Validating primary data. [2025-10-20T17:08:00.710924] [DEBUG] [VALIDATOR ] AIT: All expected trading days are present. [2025-10-20T17:08:00.711014] [DEBUG] [VALIDATOR ] AIT: Primary data validation passed. [2025-10-20T17:08:00.711507] [INFO ] [MAIN ] AIT: Fetched primary data is valid. [2025-10-20T17:08:00.711548] [DEBUG] [LOCAL ] AIT: Saving primary data. [2025-10-20T17:08:00.712571] [DEBUG] [LOCAL ] AIT: Trimmed 39190 records to 26804 (keeping 200 days of data) [2025-10-20T17:08:00.877127] [DEBUG] [LOCAL ] AIT: Primary data saved (26804 records). [2025-10-20T17:08:00.877275] [INFO ] [MAIN ] AIT: Full history saved. [2025-10-20T17:08:00.877353] [INFO ] [MAIN ] Update Progress: 71/2367 symbols completed. AIT took 32.6s. Estimated remaining: 4h 16m 48s. Rate: 8.9 symbols/min [2025-10-20T17:08:01.054788] [INFO ] [MAIN ] AKA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:01.052851] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-01. [2025-10-20T17:08:01.055981] [DEBUG] [LOCAL ] AKA: No existing data file found in primary storage. [2025-10-20T17:08:01.056049] [WARN ] [MAIN ] AKA: No existing data found, performing full refresh instead. [2025-10-20T17:08:01.056195] [INFO ] [MAIN ] AKA: Wiping existing primary data before full refresh. [2025-10-20T17:08:01.056250] [INFO ] [LOCAL ] AKA: Starting deletion of primary data. [2025-10-20T17:08:01.056327] [DEBUG] [LOCAL ] AKA: No primary data file found to delete. [2025-10-20T17:08:01.056371] [INFO ] [MAIN ] AKA: Starting full primary refresh. [2025-10-20T17:08:01.483463] [INFO ] [REMOTE ] AKA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:01.645353] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:08:01.851373] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-08. [2025-10-20T17:08:02.651397] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-01. [2025-10-20T17:08:02.867014] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:08:03.168708] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T17:08:03.243904] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:08:03.451372] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-09. [2025-10-20T17:08:04.251434] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-03. [2025-10-20T17:08:04.851490] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T17:08:04.959774] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T17:08:05.051373] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-04. [2025-10-20T17:08:05.620753] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T17:08:05.851372] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-05. [2025-10-20T17:08:06.413535] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T17:08:06.651378] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-10. [2025-10-20T17:08:07.451376] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-02. [2025-10-20T17:08:07.645828] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T17:08:07.649821] [INFO ] [REMOTE ] AIZ: Full fetch complete (42798 records). [2025-10-20T17:08:07.649950] [DEBUG] [VALIDATOR ] AIZ: Validating primary data. [2025-10-20T17:08:08.036171] [DEBUG] [VALIDATOR ] AIZ: All expected trading days are present. [2025-10-20T17:08:08.036249] [DEBUG] [VALIDATOR ] AIZ: Primary data validation passed. [2025-10-20T17:08:08.036729] [INFO ] [MAIN ] AIZ: Fetched primary data is valid. [2025-10-20T17:08:08.036789] [DEBUG] [LOCAL ] AIZ: Saving primary data. [2025-10-20T17:08:08.037699] [DEBUG] [LOCAL ] AIZ: Trimmed 42798 records to 29471 (keeping 200 days of data) [2025-10-20T17:08:08.217771] [DEBUG] [LOCAL ] AIZ: Primary data saved (29471 records). [2025-10-20T17:08:08.217911] [INFO ] [MAIN ] AIZ: Full history saved. [2025-10-20T17:08:08.217981] [INFO ] [MAIN ] Update Progress: 72/2367 symbols completed. AIZ took 25.6s. Estimated remaining: 4h 17m 2s. Rate: 8.9 symbols/min [2025-10-20T17:08:08.256632] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-06. [2025-10-20T17:08:08.399559] [INFO ] [MAIN ] AKR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:08.442467] [DEBUG] [LOCAL ] AKR: Loaded 46788 existing records from primary storage. [2025-10-20T17:08:08.442799] [INFO ] [MAIN ] AKR: Loaded 46788 existing records. [2025-10-20T17:08:08.623513] [WARN ] [MAIN ] AKR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:08.623782] [INFO ] [MAIN ] AKR: Wiping existing primary data before full refresh. [2025-10-20T17:08:08.623832] [INFO ] [LOCAL ] AKR: Starting deletion of primary data. [2025-10-20T17:08:08.624030] [DEBUG] [LOCAL ] AKR: Deleted primary data file. [2025-10-20T17:08:08.624114] [INFO ] [MAIN ] AKR: Starting full primary refresh. [2025-10-20T17:08:08.965088] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T17:08:08.995363] [INFO ] [REMOTE ] AKR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:09.051332] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-01. [2025-10-20T17:08:09.133592] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T17:08:09.851380] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-02. [2025-10-20T17:08:10.164305] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-10-20T17:08:10.657774] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-07. [2025-10-20T17:08:10.660181] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:08:11.003430] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T17:08:11.451385] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-02. [2025-10-20T17:08:12.251387] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-08. [2025-10-20T17:08:12.540403] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T17:08:12.818864] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-10-20T17:08:13.051415] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-03. [2025-10-20T17:08:13.604133] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T17:08:13.851395] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-03. [2025-10-20T17:08:14.651395] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-09. [2025-10-20T17:08:15.305839] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T17:08:15.451396] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-03. [2025-10-20T17:08:15.566105] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:08:16.251408] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-04. [2025-10-20T17:08:16.944377] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T17:08:17.051411] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-10. [2025-10-20T17:08:17.591200] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:08:17.617487] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T17:08:17.619347] [INFO ] [REMOTE ] AIZN: Full fetch complete (5826 records). [2025-10-20T17:08:17.619457] [DEBUG] [VALIDATOR ] AIZN: Validating primary data. [2025-10-20T17:08:17.856447] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-05. [2025-10-20T17:08:18.043703] [DEBUG] [VALIDATOR ] AIZN: All expected trading days are present. [2025-10-20T17:08:18.043791] [DEBUG] [VALIDATOR ] AIZN: Primary data validation passed. [2025-10-20T17:08:18.044015] [INFO ] [MAIN ] AIZN: Fetched primary data is valid. [2025-10-20T17:08:18.044055] [DEBUG] [LOCAL ] AIZN: Saving primary data. [2025-10-20T17:08:18.044444] [DEBUG] [LOCAL ] AIZN: Trimmed 5826 records to 3848 (keeping 200 days of data) [2025-10-20T17:08:18.069802] [DEBUG] [LOCAL ] AIZN: Primary data saved (3848 records). [2025-10-20T17:08:18.069941] [INFO ] [MAIN ] AIZN: Full history saved. [2025-10-20T17:08:18.070022] [INFO ] [MAIN ] Update Progress: 73/2367 symbols completed. AIZN took 29.1s. Estimated remaining: 4h 18m 33s. Rate: 8.9 symbols/min [2025-10-20T17:08:18.255796] [INFO ] [MAIN ] AL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:18.297360] [DEBUG] [LOCAL ] AL: Loaded 45438 existing records from primary storage. [2025-10-20T17:08:18.297686] [INFO ] [MAIN ] AL: Loaded 45438 existing records. [2025-10-20T17:08:18.475199] [WARN ] [MAIN ] AL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:18.475496] [INFO ] [MAIN ] AL: Wiping existing primary data before full refresh. [2025-10-20T17:08:18.475552] [INFO ] [LOCAL ] AL: Starting deletion of primary data. [2025-10-20T17:08:18.475856] [DEBUG] [LOCAL ] AL: Deleted primary data file. [2025-10-20T17:08:18.475912] [INFO ] [MAIN ] AL: Starting full primary refresh. [2025-10-20T17:08:18.657457] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-04. [2025-10-20T17:08:18.844937] [INFO ] [REMOTE ] AL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:19.451401] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-01. [2025-10-20T17:08:19.610982] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T17:08:20.251417] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-06. [2025-10-20T17:08:20.405798] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T17:08:21.051356] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-04. [2025-10-20T17:08:21.116078] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:08:21.349059] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-10-20T17:08:21.596163] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T17:08:21.851358] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-07. [2025-10-20T17:08:22.651386] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-05. [2025-10-20T17:08:23.192169] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T17:08:23.214303] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T17:08:23.451417] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-06. [2025-10-20T17:08:23.916909] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T17:08:24.251408] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-08. [2025-10-20T17:08:25.051423] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-05. [2025-10-20T17:08:25.851461] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-07. [2025-10-20T17:08:26.340978] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-10-20T17:08:26.651464] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-08. [2025-10-20T17:08:26.702116] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-10-20T17:08:26.825533] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:08:26.981405] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-10-20T17:08:27.451391] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-09. [2025-10-20T17:08:28.251387] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-06. [2025-10-20T17:08:28.867509] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T17:08:29.051402] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-02. [2025-10-20T17:08:29.765195] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T17:08:29.851410] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-07. [2025-10-20T17:08:30.560031] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T17:08:30.651379] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-03. [2025-10-20T17:08:31.347469] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:08:31.451372] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-08. [2025-10-20T17:08:32.251405] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-09. [2025-10-20T17:08:32.304933] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T17:08:32.569607] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-10-20T17:08:32.974961] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:08:33.051400] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-09. [2025-10-20T17:08:33.851434] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-04. [2025-10-20T17:08:34.659073] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-10. [2025-10-20T17:08:34.677147] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T17:08:35.451391] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-10. [2025-10-20T17:08:35.501415] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T17:08:35.501714] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T17:08:35.514188] [INFO ] [REMOTE ] AKA: Full fetch complete (2801 records). [2025-10-20T17:08:35.514280] [DEBUG] [VALIDATOR ] AKA: Validating primary data. [2025-10-20T17:08:35.726213] [WARN ] [VALIDATOR ] AKA: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T17:08:35.726458] [FATAL] [MAIN ] AKA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:08:35.726530] [WARN ] [MAIN ] AKA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:08:35.726583] [INFO ] [LOCAL ] AKA: Starting deletion of primary data. [2025-10-20T17:08:35.726736] [DEBUG] [LOCAL ] AKA: No primary data file found to delete. [2025-10-20T17:08:35.726782] [INFO ] [LOCAL ] AKA: Starting deletion of derivative data. [2025-10-20T17:08:35.726825] [DEBUG] [LOCAL ] AKA: No derivative data file found to delete. [2025-10-20T17:08:35.726900] [INFO ] [MAIN ] Update Progress: 74/2367 symbols completed. AKA took 34.8s. Estimated remaining: 4h 24m 4s. Rate: 8.7 symbols/min [2025-10-20T17:08:35.905719] [INFO ] [MAIN ] ALB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:35.962188] [DEBUG] [LOCAL ] ALB: Loaded 54123 existing records from primary storage. [2025-10-20T17:08:35.962525] [INFO ] [MAIN ] ALB: Loaded 54123 existing records. [2025-10-20T17:08:36.152975] [WARN ] [MAIN ] ALB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:36.153231] [INFO ] [MAIN ] ALB: Wiping existing primary data before full refresh. [2025-10-20T17:08:36.153282] [INFO ] [LOCAL ] ALB: Starting deletion of primary data. [2025-10-20T17:08:36.153497] [DEBUG] [LOCAL ] ALB: Deleted primary data file. [2025-10-20T17:08:36.153552] [INFO ] [MAIN ] ALB: Starting full primary refresh. [2025-10-20T17:08:36.257130] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-05. [2025-10-20T17:08:36.531334] [INFO ] [REMOTE ] ALB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:37.051383] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-01. [2025-10-20T17:08:37.237679] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T17:08:37.244463] [INFO ] [REMOTE ] AKR: Full fetch complete (65453 records). [2025-10-20T17:08:37.244593] [DEBUG] [VALIDATOR ] AKR: Validating primary data. [2025-10-20T17:08:37.626288] [DEBUG] [VALIDATOR ] AKR: All expected trading days are present. [2025-10-20T17:08:37.626398] [DEBUG] [VALIDATOR ] AKR: Primary data validation passed. [2025-10-20T17:08:37.627166] [INFO ] [MAIN ] AKR: Fetched primary data is valid. [2025-10-20T17:08:37.627251] [DEBUG] [LOCAL ] AKR: Saving primary data. [2025-10-20T17:08:37.628838] [DEBUG] [LOCAL ] AKR: Trimmed 65453 records to 46127 (keeping 200 days of data) [2025-10-20T17:08:37.763243] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:08:37.853099] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-10. [2025-10-20T17:08:37.965493] [DEBUG] [LOCAL ] AKR: Primary data saved (46127 records). [2025-10-20T17:08:37.965718] [INFO ] [MAIN ] AKR: Full history saved. [2025-10-20T17:08:37.966272] [INFO ] [MAIN ] Update Progress: 75/2367 symbols completed. AKR took 29.7s. Estimated remaining: 4h 21m 34s. Rate: 8.8 symbols/min [2025-10-20T17:08:38.155902] [INFO ] [MAIN ] ALC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:38.205466] [DEBUG] [LOCAL ] ALC: Loaded 51959 existing records from primary storage. [2025-10-20T17:08:38.205824] [INFO ] [MAIN ] ALC: Loaded 51959 existing records. [2025-10-20T17:08:38.384080] [WARN ] [MAIN ] ALC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:38.384293] [INFO ] [MAIN ] ALC: Wiping existing primary data before full refresh. [2025-10-20T17:08:38.384340] [INFO ] [LOCAL ] ALC: Starting deletion of primary data. [2025-10-20T17:08:38.384540] [DEBUG] [LOCAL ] ALC: Deleted primary data file. [2025-10-20T17:08:38.384577] [INFO ] [MAIN ] ALC: Starting full primary refresh. [2025-10-20T17:08:38.651662] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-06. [2025-10-20T17:08:38.745911] [INFO ] [REMOTE ] ALC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:38.902766] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T17:08:39.451426] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-01. [2025-10-20T17:08:39.733348] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T17:08:39.741208] [INFO ] [REMOTE ] AJG: Full fetch complete (68999 records). [2025-10-20T17:08:39.741408] [DEBUG] [VALIDATOR ] AJG: Validating primary data. [2025-10-20T17:08:40.177713] [DEBUG] [VALIDATOR ] AJG: All expected trading days are present. [2025-10-20T17:08:40.178268] [DEBUG] [VALIDATOR ] AJG: Primary data validation passed. [2025-10-20T17:08:40.179062] [INFO ] [MAIN ] AJG: Fetched primary data is valid. [2025-10-20T17:08:40.181316] [DEBUG] [LOCAL ] AJG: Saving primary data. [2025-10-20T17:08:40.181210] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T17:08:40.192328] [DEBUG] [LOCAL ] AJG: Trimmed 68999 records to 46490 (keeping 200 days of data) [2025-10-20T17:08:40.255810] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-07. [2025-10-20T17:08:40.497308] [DEBUG] [LOCAL ] AJG: Primary data saved (46490 records). [2025-10-20T17:08:40.497559] [INFO ] [MAIN ] AJG: Full history saved. [2025-10-20T17:08:40.498492] [INFO ] [MAIN ] Update Progress: 76/2367 symbols completed. AJG took 43.4s. Estimated remaining: 4h 19m 18s. Rate: 8.8 symbols/min [2025-10-20T17:08:40.673374] [INFO ] [MAIN ] ALE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:40.705202] [DEBUG] [LOCAL ] ALE: Loaded 32507 existing records from primary storage. [2025-10-20T17:08:40.705479] [INFO ] [MAIN ] ALE: Loaded 32507 existing records. [2025-10-20T17:08:40.887563] [WARN ] [MAIN ] ALE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:40.887827] [INFO ] [MAIN ] ALE: Wiping existing primary data before full refresh. [2025-10-20T17:08:40.887878] [INFO ] [LOCAL ] ALE: Starting deletion of primary data. [2025-10-20T17:08:40.888108] [DEBUG] [LOCAL ] ALE: Deleted primary data file. [2025-10-20T17:08:40.888169] [INFO ] [MAIN ] ALE: Starting full primary refresh. [2025-10-20T17:08:41.062133] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-02. [2025-10-20T17:08:41.254680] [INFO ] [REMOTE ] ALE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:41.285320] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:08:41.394869] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T17:08:41.851417] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-02. [2025-10-20T17:08:42.651446] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-01. [2025-10-20T17:08:42.854679] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T17:08:43.451451] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-03. [2025-10-20T17:08:43.522729] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T17:08:43.695224] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:08:44.251373] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-02. [2025-10-20T17:08:45.041777] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T17:08:45.051321] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-03. [2025-10-20T17:08:45.143944] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T17:08:45.851413] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-03. [2025-10-20T17:08:46.155767] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T17:08:46.651434] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-08. [2025-10-20T17:08:47.451414] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-04. [2025-10-20T17:08:47.595473] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:08:47.848903] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T17:08:48.251390] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-09. [2025-10-20T17:08:49.051400] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-04. [2025-10-20T17:08:49.496522] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T17:08:49.851388] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-05. [2025-10-20T17:08:49.963484] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T17:08:50.651383] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-04. [2025-10-20T17:08:50.831152] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:08:51.451353] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-05. [2025-10-20T17:08:51.853530] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T17:08:51.971575] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T17:08:52.251449] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-05. [2025-10-20T17:08:53.051393] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-10. [2025-10-20T17:08:53.290786] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T17:08:53.573899] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T17:08:53.851365] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-06. [2025-10-20T17:08:54.651392] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-06. [2025-10-20T17:08:54.777682] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T17:08:54.809944] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:08:54.816880] [INFO ] [REMOTE ] AL: Full fetch complete (63611 records). [2025-10-20T17:08:54.816978] [DEBUG] [VALIDATOR ] AL: Validating primary data. [2025-10-20T17:08:55.221788] [DEBUG] [VALIDATOR ] AL: All expected trading days are present. [2025-10-20T17:08:55.221889] [DEBUG] [VALIDATOR ] AL: Primary data validation passed. [2025-10-20T17:08:55.222577] [INFO ] [MAIN ] AL: Fetched primary data is valid. [2025-10-20T17:08:55.222670] [DEBUG] [LOCAL ] AL: Saving primary data. [2025-10-20T17:08:55.224105] [DEBUG] [LOCAL ] AL: Trimmed 63611 records to 44827 (keeping 200 days of data) [2025-10-20T17:08:55.452393] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-06. [2025-10-20T17:08:55.496000] [DEBUG] [LOCAL ] AL: Primary data saved (44827 records). [2025-10-20T17:08:55.496197] [INFO ] [MAIN ] AL: Full history saved. [2025-10-20T17:08:55.496884] [INFO ] [MAIN ] Update Progress: 77/2367 symbols completed. AL took 37.4s. Estimated remaining: 4h 23m 15s. Rate: 8.7 symbols/min [2025-10-20T17:08:56.251352] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-07. [2025-10-20T17:08:56.661555] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-10-20T17:08:56.973866] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:08:57.051366] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-07. [2025-10-20T17:08:57.732838] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:08:57.851401] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-08. [2025-10-20T17:08:58.651404] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-07. [2025-10-20T17:08:58.708934] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T17:08:59.038499] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T17:08:59.451369] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-09. [2025-10-20T17:09:00.251372] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-08. [2025-10-20T17:09:00.913511] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:09:01.051370] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-10. [2025-10-20T17:09:01.139220] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-07 (2.49 seconds). [2025-10-20T17:09:01.642133] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:09:01.851349] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-09. [2025-10-20T17:09:02.515385] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T17:09:02.521719] [INFO ] [REMOTE ] ALE: Full fetch complete (44169 records). [2025-10-20T17:09:02.521890] [DEBUG] [VALIDATOR ] ALE: Validating primary data. [2025-10-20T17:09:02.656576] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-08. [2025-10-20T17:09:02.958043] [DEBUG] [VALIDATOR ] ALE: All expected trading days are present. [2025-10-20T17:09:02.958148] [DEBUG] [VALIDATOR ] ALE: Primary data validation passed. [2025-10-20T17:09:02.958790] [INFO ] [MAIN ] ALE: Fetched primary data is valid. [2025-10-20T17:09:02.958869] [DEBUG] [LOCAL ] ALE: Saving primary data. [2025-10-20T17:09:02.960060] [DEBUG] [LOCAL ] ALE: Trimmed 44169 records to 32104 (keeping 200 days of data) [2025-10-20T17:09:03.163417] [DEBUG] [LOCAL ] ALE: Primary data saved (32104 records). [2025-10-20T17:09:03.163546] [INFO ] [MAIN ] ALE: Full history saved. [2025-10-20T17:09:03.163625] [INFO ] [MAIN ] Update Progress: 78/2367 symbols completed. ALE took 22.7s. Estimated remaining: 4h 23m 30s. Rate: 8.7 symbols/min [2025-10-20T17:09:03.878769] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T17:09:03.889814] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-10. [2025-10-20T17:09:05.039492] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-08 (2.38 seconds). [2025-10-20T17:09:05.051171] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-09. [2025-10-20T17:09:05.609375] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T17:09:05.617220] [INFO ] [REMOTE ] ALC: Full fetch complete (73767 records). [2025-10-20T17:09:05.617392] [DEBUG] [VALIDATOR ] ALC: Validating primary data. [2025-10-20T17:09:06.007692] [DEBUG] [VALIDATOR ] ALC: All expected trading days are present. [2025-10-20T17:09:06.007795] [DEBUG] [VALIDATOR ] ALC: Primary data validation passed. [2025-10-20T17:09:06.008530] [INFO ] [MAIN ] ALC: Fetched primary data is valid. [2025-10-20T17:09:06.008620] [DEBUG] [LOCAL ] ALC: Saving primary data. [2025-10-20T17:09:06.010122] [DEBUG] [LOCAL ] ALC: Trimmed 73767 records to 51212 (keeping 200 days of data) [2025-10-20T17:09:06.318039] [DEBUG] [LOCAL ] ALC: Primary data saved (51212 records). [2025-10-20T17:09:06.318195] [INFO ] [MAIN ] ALC: Full history saved. [2025-10-20T17:09:06.318899] [INFO ] [MAIN ] Update Progress: 79/2367 symbols completed. ALC took 28.4s. Estimated remaining: 4h 21m 35s. Rate: 8.7 symbols/min [2025-10-20T17:09:07.089122] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T17:09:07.100461] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-10. [2025-10-20T17:09:09.719955] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-10 (2.62 seconds). [2025-10-20T17:09:09.727828] [INFO ] [REMOTE ] ALB: Full fetch complete (77075 records). [2025-10-20T17:09:09.727983] [DEBUG] [VALIDATOR ] ALB: Validating primary data. [2025-10-20T17:09:10.130460] [DEBUG] [VALIDATOR ] ALB: All expected trading days are present. [2025-10-20T17:09:10.130573] [DEBUG] [VALIDATOR ] ALB: Primary data validation passed. [2025-10-20T17:09:10.131480] [INFO ] [MAIN ] ALB: Fetched primary data is valid. [2025-10-20T17:09:10.131555] [DEBUG] [LOCAL ] ALB: Saving primary data. [2025-10-20T17:09:10.133245] [DEBUG] [LOCAL ] ALB: Trimmed 77075 records to 53378 (keeping 200 days of data) [2025-10-20T17:09:10.450065] [DEBUG] [LOCAL ] ALB: Primary data saved (53378 records). [2025-10-20T17:09:10.450216] [INFO ] [MAIN ] ALB: Full history saved. [2025-10-20T17:09:10.450912] [INFO ] [MAIN ] Update Progress: 80/2367 symbols completed. ALB took 34.7s. Estimated remaining: 4h 20m 10s. Rate: 8.8 symbols/min [2025-10-20T17:09:11.063336] [INFO ] [MAIN ] ALIT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:11.090972] [INFO ] [MAIN ] ALG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:11.212609] [INFO ] [MAIN ] ALH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:11.213290] [DEBUG] [LOCAL ] ALH: No existing data file found in primary storage. [2025-10-20T17:09:11.228602] [WARN ] [MAIN ] ALH: No existing data found, performing full refresh instead. [2025-10-20T17:09:11.232310] [INFO ] [MAIN ] ALEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:11.232688] [INFO ] [MAIN ] ALH: Wiping existing primary data before full refresh. [2025-10-20T17:09:11.233586] [INFO ] [LOCAL ] ALH: Starting deletion of primary data. [2025-10-20T17:09:11.233695] [DEBUG] [LOCAL ] ALH: No primary data file found to delete. [2025-10-20T17:09:11.233751] [INFO ] [MAIN ] ALH: Starting full primary refresh. [2025-10-20T17:09:11.347826] [DEBUG] [LOCAL ] ALG: Loaded 9877 existing records from primary storage. [2025-10-20T17:09:11.354800] [INFO ] [MAIN ] ALG: Loaded 9877 existing records. [2025-10-20T17:09:11.585114] [DEBUG] [LOCAL ] ALEX: Loaded 31633 existing records from primary storage. [2025-10-20T17:09:11.669013] [INFO ] [MAIN ] ALEX: Loaded 31633 existing records. [2025-10-20T17:09:11.790239] [WARN ] [MAIN ] ALG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:11.849027] [INFO ] [MAIN ] ALG: Wiping existing primary data before full refresh. [2025-10-20T17:09:11.830236] [DEBUG] [LOCAL ] ALIT: Loaded 53560 existing records from primary storage. [2025-10-20T17:09:11.854337] [INFO ] [LOCAL ] ALG: Starting deletion of primary data. [2025-10-20T17:09:11.908151] [DEBUG] [LOCAL ] ALG: Deleted primary data file. [2025-10-20T17:09:11.875737] [INFO ] [MAIN ] ALIT: Loaded 53560 existing records. [2025-10-20T17:09:11.908400] [INFO ] [MAIN ] ALG: Starting full primary refresh. [2025-10-20T17:09:12.038761] [INFO ] [REMOTE ] ALH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:12.125679] [INFO ] [REMOTE ] ALH: Fetching primary data for 2025-01. [2025-10-20T17:09:12.247852] [WARN ] [MAIN ] ALEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:12.277082] [INFO ] [MAIN ] ALEX: Wiping existing primary data before full refresh. [2025-10-20T17:09:12.277420] [INFO ] [LOCAL ] ALEX: Starting deletion of primary data. [2025-10-20T17:09:12.308723] [DEBUG] [LOCAL ] ALEX: Deleted primary data file. [2025-10-20T17:09:12.398750] [INFO ] [MAIN ] ALEX: Starting full primary refresh. [2025-10-20T17:09:12.402564] [WARN ] [MAIN ] ALIT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:12.467372] [INFO ] [MAIN ] ALIT: Wiping existing primary data before full refresh. [2025-10-20T17:09:12.468220] [INFO ] [LOCAL ] ALIT: Starting deletion of primary data. [2025-10-20T17:09:12.484825] [DEBUG] [REMOTE ] ALH: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:09:12.493943] [WARN ] [REMOTE ] ALH: No data returned for month 2025-01 [2025-10-20T17:09:12.499237] [FATAL] [REMOTE ] ALH: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:09:12.520121] [FATAL] [MAIN ] ALH: Failed to fetch primary data. [2025-10-20T17:09:12.549766] [DEBUG] [LOCAL ] ALIT: Deleted primary data file. [2025-10-20T17:09:12.625680] [INFO ] [MAIN ] ALIT: Starting full primary refresh. [2025-10-20T17:09:12.615171] [WARN ] [MAIN ] ALH: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:09:12.641548] [INFO ] [LOCAL ] ALH: Starting deletion of primary data. [2025-10-20T17:09:12.734620] [DEBUG] [LOCAL ] ALH: No primary data file found to delete. [2025-10-20T17:09:12.852471] [INFO ] [LOCAL ] ALH: Starting deletion of derivative data. [2025-10-20T17:09:12.937624] [DEBUG] [LOCAL ] ALH: No derivative data file found to delete. [2025-10-20T17:09:13.029435] [INFO ] [REMOTE ] ALG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:13.080158] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-01. [2025-10-20T17:09:13.079829] [INFO ] [MAIN ] Update Progress: 81/2367 symbols completed. ALH took 2.6s. Estimated remaining: 4h 18m 5s. Rate: 8.9 symbols/min [2025-10-20T17:09:13.446950] [INFO ] [REMOTE ] ALEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:13.468409] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-01. [2025-10-20T17:09:13.580311] [INFO ] [MAIN ] ALK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:13.635165] [INFO ] [REMOTE ] ALIT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:13.635787] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-01. [2025-10-20T17:09:13.693362] [DEBUG] [LOCAL ] ALK: Loaded 53860 existing records from primary storage. [2025-10-20T17:09:13.693890] [INFO ] [MAIN ] ALK: Loaded 53860 existing records. [2025-10-20T17:09:13.898279] [WARN ] [MAIN ] ALK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:13.898947] [INFO ] [MAIN ] ALK: Wiping existing primary data before full refresh. [2025-10-20T17:09:13.899054] [INFO ] [LOCAL ] ALK: Starting deletion of primary data. [2025-10-20T17:09:13.899602] [DEBUG] [LOCAL ] ALK: Deleted primary data file. [2025-10-20T17:09:13.899810] [INFO ] [MAIN ] ALK: Starting full primary refresh. [2025-10-20T17:09:14.281153] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:09:14.326706] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-02. [2025-10-20T17:09:14.325803] [INFO ] [REMOTE ] ALK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:14.328078] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-01. [2025-10-20T17:09:14.676187] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:09:14.682208] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-02. [2025-10-20T17:09:15.209463] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T17:09:15.212556] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-03. [2025-10-20T17:09:15.463795] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:09:15.475129] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-02. [2025-10-20T17:09:15.490849] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T17:09:15.496337] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-03. [2025-10-20T17:09:15.852761] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T17:09:15.856400] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-04. [2025-10-20T17:09:16.331189] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:09:16.341628] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-02. [2025-10-20T17:09:16.692121] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:09:16.713794] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:09:16.714913] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-03. [2025-10-20T17:09:16.718876] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-05. [2025-10-20T17:09:16.860820] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T17:09:17.051402] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-04. [2025-10-20T17:09:17.408926] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T17:09:17.674935] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:09:17.851413] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-03. [2025-10-20T17:09:18.316839] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T17:09:18.474206] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T17:09:18.651370] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-06. [2025-10-20T17:09:19.456458] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-05. [2025-10-20T17:09:19.472989] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:09:19.562777] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T17:09:20.251394] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-04. [2025-10-20T17:09:20.628327] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T17:09:21.051424] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-07. [2025-10-20T17:09:21.710521] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T17:09:21.825859] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T17:09:21.851340] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-05. [2025-10-20T17:09:22.651371] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-06. [2025-10-20T17:09:23.452671] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-08. [2025-10-20T17:09:23.455291] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T17:09:23.496491] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:09:24.035425] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T17:09:24.251380] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-07. [2025-10-20T17:09:25.051358] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-04. [2025-10-20T17:09:25.324418] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T17:09:25.851419] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-08. [2025-10-20T17:09:26.651449] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-06. [2025-10-20T17:09:27.149450] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T17:09:27.184771] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-10-20T17:09:27.451379] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-09. [2025-10-20T17:09:28.062918] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T17:09:28.136352] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T17:09:28.251415] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-05. [2025-10-20T17:09:29.051442] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-07. [2025-10-20T17:09:29.851367] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-09. [2025-10-20T17:09:29.982230] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T17:09:30.651370] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-06. [2025-10-20T17:09:30.895728] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:09:31.026752] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T17:09:31.451386] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-10. [2025-10-20T17:09:32.000268] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:09:32.204903] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T17:09:32.206577] [INFO ] [REMOTE ] ALG: Full fetch complete (15295 records). [2025-10-20T17:09:32.206686] [DEBUG] [VALIDATOR ] ALG: Validating primary data. [2025-10-20T17:09:32.260331] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-10. [2025-10-20T17:09:32.577437] [DEBUG] [VALIDATOR ] ALG: All expected trading days are present. [2025-10-20T17:09:32.577518] [DEBUG] [VALIDATOR ] ALG: Primary data validation passed. [2025-10-20T17:09:32.577777] [INFO ] [MAIN ] ALG: Fetched primary data is valid. [2025-10-20T17:09:32.577838] [DEBUG] [LOCAL ] ALG: Saving primary data. [2025-10-20T17:09:32.578325] [DEBUG] [LOCAL ] ALG: Trimmed 15295 records to 9748 (keeping 200 days of data) [2025-10-20T17:09:32.639854] [DEBUG] [LOCAL ] ALG: Primary data saved (9748 records). [2025-10-20T17:09:32.640004] [INFO ] [MAIN ] ALG: Full history saved. [2025-10-20T17:09:32.640082] [INFO ] [MAIN ] Update Progress: 82/2367 symbols completed. ALG took 22.2s. Estimated remaining: 4h 23m 54s. Rate: 8.7 symbols/min [2025-10-20T17:09:32.815027] [INFO ] [MAIN ] ALL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:32.865990] [DEBUG] [LOCAL ] ALL: Loaded 50503 existing records from primary storage. [2025-10-20T17:09:32.866351] [INFO ] [MAIN ] ALL: Loaded 50503 existing records. [2025-10-20T17:09:33.060516] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-07. [2025-10-20T17:09:33.066253] [WARN ] [MAIN ] ALL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:33.067518] [INFO ] [MAIN ] ALL: Wiping existing primary data before full refresh. [2025-10-20T17:09:33.067570] [INFO ] [LOCAL ] ALL: Starting deletion of primary data. [2025-10-20T17:09:33.067940] [DEBUG] [LOCAL ] ALL: Deleted primary data file. [2025-10-20T17:09:33.068000] [INFO ] [MAIN ] ALL: Starting full primary refresh. [2025-10-20T17:09:33.432971] [INFO ] [REMOTE ] ALL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:33.627098] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:09:33.632350] [INFO ] [REMOTE ] ALEX: Full fetch complete (44539 records). [2025-10-20T17:09:33.632527] [DEBUG] [VALIDATOR ] ALEX: Validating primary data. [2025-10-20T17:09:33.856784] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-08. [2025-10-20T17:09:34.063897] [DEBUG] [VALIDATOR ] ALEX: All expected trading days are present. [2025-10-20T17:09:34.063992] [DEBUG] [VALIDATOR ] ALEX: Primary data validation passed. [2025-10-20T17:09:34.064532] [INFO ] [MAIN ] ALEX: Fetched primary data is valid. [2025-10-20T17:09:34.064573] [DEBUG] [LOCAL ] ALEX: Saving primary data. [2025-10-20T17:09:34.065745] [DEBUG] [LOCAL ] ALEX: Trimmed 44539 records to 31005 (keeping 200 days of data) [2025-10-20T17:09:34.265026] [DEBUG] [LOCAL ] ALEX: Primary data saved (31005 records). [2025-10-20T17:09:34.265183] [INFO ] [MAIN ] ALEX: Full history saved. [2025-10-20T17:09:34.265268] [INFO ] [MAIN ] Update Progress: 83/2367 symbols completed. ALEX took 23.8s. Estimated remaining: 4h 21m 21s. Rate: 8.7 symbols/min [2025-10-20T17:09:34.457498] [INFO ] [MAIN ] ALLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:34.504281] [DEBUG] [LOCAL ] ALLE: Loaded 44772 existing records from primary storage. [2025-10-20T17:09:34.504724] [INFO ] [MAIN ] ALLE: Loaded 44772 existing records. [2025-10-20T17:09:34.667739] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-01. [2025-10-20T17:09:34.706562] [WARN ] [MAIN ] ALLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:34.708058] [INFO ] [MAIN ] ALLE: Wiping existing primary data before full refresh. [2025-10-20T17:09:34.708181] [INFO ] [LOCAL ] ALLE: Starting deletion of primary data. [2025-10-20T17:09:34.708807] [DEBUG] [LOCAL ] ALLE: Deleted primary data file. [2025-10-20T17:09:34.708891] [INFO ] [MAIN ] ALLE: Starting full primary refresh. [2025-10-20T17:09:34.989813] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T17:09:35.122499] [INFO ] [REMOTE ] ALLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:35.451375] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-01. [2025-10-20T17:09:35.560257] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T17:09:36.251422] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-09. [2025-10-20T17:09:36.744994] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T17:09:36.814265] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T17:09:37.051396] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-02. [2025-10-20T17:09:37.851384] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-02. [2025-10-20T17:09:38.143879] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T17:09:38.651378] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-10. [2025-10-20T17:09:38.742680] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:09:39.451402] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-08. [2025-10-20T17:09:39.500142] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:09:40.251342] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-03. [2025-10-20T17:09:40.620200] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T17:09:40.628062] [INFO ] [REMOTE ] ALK: Full fetch complete (77025 records). [2025-10-20T17:09:40.628202] [DEBUG] [VALIDATOR ] ALK: Validating primary data. [2025-10-20T17:09:41.036173] [DEBUG] [VALIDATOR ] ALK: All expected trading days are present. [2025-10-20T17:09:41.036274] [DEBUG] [VALIDATOR ] ALK: Primary data validation passed. [2025-10-20T17:09:41.037049] [INFO ] [MAIN ] ALK: Fetched primary data is valid. [2025-10-20T17:09:41.037111] [DEBUG] [LOCAL ] ALK: Saving primary data. [2025-10-20T17:09:41.038587] [DEBUG] [LOCAL ] ALK: Trimmed 77025 records to 53085 (keeping 200 days of data) [2025-10-20T17:09:41.088076] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-03. [2025-10-20T17:09:41.344343] [DEBUG] [LOCAL ] ALK: Primary data saved (53085 records). [2025-10-20T17:09:41.344496] [INFO ] [MAIN ] ALK: Full history saved. [2025-10-20T17:09:41.344580] [INFO ] [MAIN ] Update Progress: 84/2367 symbols completed. ALK took 28.3s. Estimated remaining: 4h 21m 20s. Rate: 8.7 symbols/min [2025-10-20T17:09:41.532770] [INFO ] [MAIN ] ALLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:41.563517] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T17:09:41.619192] [DEBUG] [LOCAL ] ALLY: Loaded 54000 existing records from primary storage. [2025-10-20T17:09:41.619539] [INFO ] [MAIN ] ALLY: Loaded 54000 existing records. [2025-10-20T17:09:41.796280] [WARN ] [MAIN ] ALLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:41.796526] [INFO ] [MAIN ] ALLY: Wiping existing primary data before full refresh. [2025-10-20T17:09:41.796573] [INFO ] [LOCAL ] ALLY: Starting deletion of primary data. [2025-10-20T17:09:41.797031] [DEBUG] [LOCAL ] ALLY: Deleted primary data file. [2025-10-20T17:09:41.797087] [INFO ] [MAIN ] ALLY: Starting full primary refresh. [2025-10-20T17:09:41.879102] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-09. [2025-10-20T17:09:42.174263] [INFO ] [REMOTE ] ALLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:42.210663] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T17:09:42.651373] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-04. [2025-10-20T17:09:42.988512] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T17:09:43.451363] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-04. [2025-10-20T17:09:43.755255] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T17:09:44.251385] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-10. [2025-10-20T17:09:44.426928] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:09:45.051382] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-05. [2025-10-20T17:09:45.230893] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:09:45.742543] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:09:45.750166] [INFO ] [REMOTE ] ALIT: Full fetch complete (76618 records). [2025-10-20T17:09:45.750278] [DEBUG] [VALIDATOR ] ALIT: Validating primary data. [2025-10-20T17:09:45.856544] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-01. [2025-10-20T17:09:46.135169] [DEBUG] [VALIDATOR ] ALIT: All expected trading days are present. [2025-10-20T17:09:46.135249] [DEBUG] [VALIDATOR ] ALIT: Primary data validation passed. [2025-10-20T17:09:46.135988] [INFO ] [MAIN ] ALIT: Fetched primary data is valid. [2025-10-20T17:09:46.136053] [DEBUG] [LOCAL ] ALIT: Saving primary data. [2025-10-20T17:09:46.137416] [DEBUG] [LOCAL ] ALIT: Trimmed 76618 records to 52790 (keeping 200 days of data) [2025-10-20T17:09:46.435393] [DEBUG] [LOCAL ] ALIT: Primary data saved (52790 records). [2025-10-20T17:09:46.435542] [INFO ] [MAIN ] ALIT: Full history saved. [2025-10-20T17:09:46.436093] [INFO ] [MAIN ] Update Progress: 85/2367 symbols completed. ALIT took 36.0s. Estimated remaining: 4h 20m 26s. Rate: 8.8 symbols/min [2025-10-20T17:09:46.607056] [INFO ] [MAIN ] ALSN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:46.646097] [DEBUG] [LOCAL ] ALSN: Loaded 44062 existing records from primary storage. [2025-10-20T17:09:46.646389] [INFO ] [MAIN ] ALSN: Loaded 44062 existing records. [2025-10-20T17:09:46.676938] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-05. [2025-10-20T17:09:46.828598] [WARN ] [MAIN ] ALSN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:46.828936] [INFO ] [MAIN ] ALSN: Wiping existing primary data before full refresh. [2025-10-20T17:09:46.828981] [INFO ] [LOCAL ] ALSN: Starting deletion of primary data. [2025-10-20T17:09:46.829232] [DEBUG] [LOCAL ] ALSN: Deleted primary data file. [2025-10-20T17:09:46.829270] [INFO ] [MAIN ] ALSN: Starting full primary refresh. [2025-10-20T17:09:47.220069] [INFO ] [REMOTE ] ALSN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:47.239515] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-05 (2.19 seconds). [2025-10-20T17:09:47.451362] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-06. [2025-10-20T17:09:47.653103] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:09:48.251437] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-02. [2025-10-20T17:09:48.304161] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T17:09:48.744993] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:09:49.051379] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-06. [2025-10-20T17:09:49.851422] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-01. [2025-10-20T17:09:50.042772] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T17:09:50.244766] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:09:50.651371] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-03. [2025-10-20T17:09:51.038900] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:09:51.451349] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-07. [2025-10-20T17:09:52.144907] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T17:09:52.251364] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-04. [2025-10-20T17:09:53.051369] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-02. [2025-10-20T17:09:53.354943] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T17:09:53.665275] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T17:09:53.851395] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-08. [2025-10-20T17:09:54.507635] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:09:54.651363] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-03. [2025-10-20T17:09:55.237488] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:09:55.451383] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-07. [2025-10-20T17:09:55.865771] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T17:09:56.251377] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-05. [2025-10-20T17:09:57.051345] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-09. [2025-10-20T17:09:57.160679] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T17:09:57.715083] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T17:09:57.851401] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-06. [2025-10-20T17:09:58.651345] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-08. [2025-10-20T17:09:58.721740] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T17:09:59.451384] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-10. [2025-10-20T17:09:59.633255] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:10:00.251453] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-04. [2025-10-20T17:10:00.394635] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:10:01.051381] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-09. [2025-10-20T17:10:01.406509] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T17:10:01.414282] [INFO ] [REMOTE ] ALL: Full fetch complete (72555 records). [2025-10-20T17:10:01.414476] [DEBUG] [VALIDATOR ] ALL: Validating primary data. [2025-10-20T17:10:01.852254] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-07. [2025-10-20T17:10:01.877410] [DEBUG] [VALIDATOR ] ALL: All expected trading days are present. [2025-10-20T17:10:01.877531] [DEBUG] [VALIDATOR ] ALL: Primary data validation passed. [2025-10-20T17:10:01.878917] [INFO ] [MAIN ] ALL: Fetched primary data is valid. [2025-10-20T17:10:01.879136] [DEBUG] [LOCAL ] ALL: Saving primary data. [2025-10-20T17:10:01.881174] [DEBUG] [LOCAL ] ALL: Trimmed 72555 records to 49721 (keeping 200 days of data) [2025-10-20T17:10:02.167271] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-10-20T17:10:02.256074] [DEBUG] [LOCAL ] ALL: Primary data saved (49721 records). [2025-10-20T17:10:02.256231] [INFO ] [MAIN ] ALL: Full history saved. [2025-10-20T17:10:02.257286] [INFO ] [MAIN ] Update Progress: 86/2367 symbols completed. ALL took 29.6s. Estimated remaining: 4h 24m 17s. Rate: 8.6 symbols/min [2025-10-20T17:10:02.434345] [INFO ] [MAIN ] ALTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:02.457984] [DEBUG] [LOCAL ] ALTG: Loaded 23632 existing records from primary storage. [2025-10-20T17:10:02.458237] [INFO ] [MAIN ] ALTG: Loaded 23632 existing records. [2025-10-20T17:10:02.641067] [WARN ] [MAIN ] ALTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:02.641350] [INFO ] [MAIN ] ALTG: Wiping existing primary data before full refresh. [2025-10-20T17:10:02.641405] [INFO ] [LOCAL ] ALTG: Starting deletion of primary data. [2025-10-20T17:10:02.641674] [DEBUG] [LOCAL ] ALTG: Deleted primary data file. [2025-10-20T17:10:02.641744] [INFO ] [MAIN ] ALTG: Starting full primary refresh. [2025-10-20T17:10:02.673866] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-05. [2025-10-20T17:10:02.960676] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T17:10:03.110138] [INFO ] [REMOTE ] ALTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:03.451409] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-01. [2025-10-20T17:10:03.506056] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:10:04.215912] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T17:10:04.251370] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-06. [2025-10-20T17:10:04.656348] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:10:05.051425] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-02. [2025-10-20T17:10:05.605939] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:10:05.670833] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T17:10:05.851434] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-10. [2025-10-20T17:10:06.651393] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-07. [2025-10-20T17:10:07.451365] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-03. [2025-10-20T17:10:07.677586] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T17:10:07.684856] [INFO ] [REMOTE ] ALLE: Full fetch complete (64858 records). [2025-10-20T17:10:07.685046] [DEBUG] [VALIDATOR ] ALLE: Validating primary data. [2025-10-20T17:10:08.101437] [DEBUG] [VALIDATOR ] ALLE: All expected trading days are present. [2025-10-20T17:10:08.101532] [DEBUG] [VALIDATOR ] ALLE: Primary data validation passed. [2025-10-20T17:10:08.102220] [INFO ] [MAIN ] ALLE: Fetched primary data is valid. [2025-10-20T17:10:08.102285] [DEBUG] [LOCAL ] ALLE: Saving primary data. [2025-10-20T17:10:08.103758] [DEBUG] [LOCAL ] ALLE: Trimmed 64858 records to 44026 (keeping 200 days of data) [2025-10-20T17:10:08.284034] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-08. [2025-10-20T17:10:08.389126] [DEBUG] [LOCAL ] ALLE: Primary data saved (44026 records). [2025-10-20T17:10:08.389694] [INFO ] [MAIN ] ALLE: Full history saved. [2025-10-20T17:10:08.390795] [INFO ] [MAIN ] Update Progress: 87/2367 symbols completed. ALLE took 34.1s. Estimated remaining: 4h 23m 49s. Rate: 8.6 symbols/min [2025-10-20T17:10:08.441463] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:10:08.629081] [INFO ] [MAIN ] ALUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:08.630607] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T17:10:08.648423] [DEBUG] [LOCAL ] ALUR: Loaded 9640 existing records from primary storage. [2025-10-20T17:10:08.651598] [INFO ] [MAIN ] ALUR: Loaded 9640 existing records. [2025-10-20T17:10:08.841449] [WARN ] [MAIN ] ALUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:08.841670] [INFO ] [MAIN ] ALUR: Wiping existing primary data before full refresh. [2025-10-20T17:10:08.841729] [INFO ] [LOCAL ] ALUR: Starting deletion of primary data. [2025-10-20T17:10:08.841872] [DEBUG] [LOCAL ] ALUR: Deleted primary data file. [2025-10-20T17:10:08.841926] [INFO ] [MAIN ] ALUR: Starting full primary refresh. [2025-10-20T17:10:09.060442] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-08. [2025-10-20T17:10:09.203789] [INFO ] [REMOTE ] ALUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:09.851353] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-04. [2025-10-20T17:10:10.399932] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-10-20T17:10:10.651497] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-09. [2025-10-20T17:10:10.773734] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:10:11.037188] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T17:10:11.451398] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-05. [2025-10-20T17:10:12.093454] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T17:10:12.180635] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T17:10:12.251389] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-01. [2025-10-20T17:10:13.051395] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-09. [2025-10-20T17:10:13.753741] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:10:13.851361] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-02. [2025-10-20T17:10:14.651362] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-10. [2025-10-20T17:10:14.762817] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T17:10:14.823439] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T17:10:15.451448] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-06. [2025-10-20T17:10:16.251380] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-03. [2025-10-20T17:10:16.349241] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T17:10:16.729508] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T17:10:16.737238] [INFO ] [REMOTE ] ALLY: Full fetch complete (77235 records). [2025-10-20T17:10:16.737377] [DEBUG] [VALIDATOR ] ALLY: Validating primary data. [2025-10-20T17:10:17.056565] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-07. [2025-10-20T17:10:17.127937] [DEBUG] [VALIDATOR ] ALLY: All expected trading days are present. [2025-10-20T17:10:17.128026] [DEBUG] [VALIDATOR ] ALLY: Primary data validation passed. [2025-10-20T17:10:17.128868] [INFO ] [MAIN ] ALLY: Fetched primary data is valid. [2025-10-20T17:10:17.128969] [DEBUG] [LOCAL ] ALLY: Saving primary data. [2025-10-20T17:10:17.140577] [DEBUG] [LOCAL ] ALLY: Trimmed 77235 records to 53224 (keeping 200 days of data) [2025-10-20T17:10:17.141484] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T17:10:17.469480] [DEBUG] [LOCAL ] ALLY: Primary data saved (53224 records). [2025-10-20T17:10:17.469688] [INFO ] [MAIN ] ALLY: Full history saved. [2025-10-20T17:10:17.469817] [INFO ] [MAIN ] Update Progress: 88/2367 symbols completed. ALLY took 36.1s. Estimated remaining: 4h 24m 37s. Rate: 8.6 symbols/min [2025-10-20T17:10:17.643308] [INFO ] [MAIN ] ALV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:17.684441] [DEBUG] [LOCAL ] ALV: Loaded 42843 existing records from primary storage. [2025-10-20T17:10:17.684761] [INFO ] [MAIN ] ALV: Loaded 42843 existing records. [2025-10-20T17:10:17.851602] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-04. [2025-10-20T17:10:17.870855] [WARN ] [MAIN ] ALV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:17.871838] [INFO ] [MAIN ] ALV: Wiping existing primary data before full refresh. [2025-10-20T17:10:17.871927] [INFO ] [LOCAL ] ALV: Starting deletion of primary data. [2025-10-20T17:10:17.872135] [DEBUG] [LOCAL ] ALV: Deleted primary data file. [2025-10-20T17:10:17.872196] [INFO ] [MAIN ] ALV: Starting full primary refresh. [2025-10-20T17:10:18.236321] [INFO ] [REMOTE ] ALV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:18.450188] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:10:18.520860] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T17:10:18.651357] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-10. [2025-10-20T17:10:19.451476] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-08. [2025-10-20T17:10:20.177128] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T17:10:20.183575] [INFO ] [REMOTE ] ALSN: Full fetch complete (62570 records). [2025-10-20T17:10:20.183706] [DEBUG] [VALIDATOR ] ALSN: Validating primary data. [2025-10-20T17:10:20.258033] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-05. [2025-10-20T17:10:20.569506] [DEBUG] [VALIDATOR ] ALSN: All expected trading days are present. [2025-10-20T17:10:20.569584] [DEBUG] [VALIDATOR ] ALSN: Primary data validation passed. [2025-10-20T17:10:20.570222] [INFO ] [MAIN ] ALSN: Fetched primary data is valid. [2025-10-20T17:10:20.570283] [DEBUG] [LOCAL ] ALSN: Saving primary data. [2025-10-20T17:10:20.571459] [DEBUG] [LOCAL ] ALSN: Trimmed 62570 records to 43332 (keeping 200 days of data) [2025-10-20T17:10:20.684768] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T17:10:20.855911] [DEBUG] [LOCAL ] ALSN: Primary data saved (43332 records). [2025-10-20T17:10:20.856045] [INFO ] [MAIN ] ALSN: Full history saved. [2025-10-20T17:10:20.856698] [INFO ] [MAIN ] Update Progress: 89/2367 symbols completed. ALSN took 34.4s. Estimated remaining: 4h 22m 58s. Rate: 8.7 symbols/min [2025-10-20T17:10:21.031621] [INFO ] [MAIN ] ALX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:21.040675] [DEBUG] [LOCAL ] ALX: Loaded 7141 existing records from primary storage. [2025-10-20T17:10:21.040911] [INFO ] [MAIN ] ALX: Loaded 7141 existing records. [2025-10-20T17:10:21.062893] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-09. [2025-10-20T17:10:21.232479] [WARN ] [MAIN ] ALX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:21.232739] [INFO ] [MAIN ] ALX: Wiping existing primary data before full refresh. [2025-10-20T17:10:21.232849] [INFO ] [LOCAL ] ALX: Starting deletion of primary data. [2025-10-20T17:10:21.233034] [DEBUG] [LOCAL ] ALX: Deleted primary data file. [2025-10-20T17:10:21.233084] [INFO ] [MAIN ] ALX: Starting full primary refresh. [2025-10-20T17:10:21.552889] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T17:10:21.618895] [INFO ] [REMOTE ] ALX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:21.841679] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T17:10:21.851332] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-06. [2025-10-20T17:10:22.438810] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T17:10:22.651387] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-07. [2025-10-20T17:10:23.451386] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-01. [2025-10-20T17:10:23.627156] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:10:24.088531] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T17:10:24.251431] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-02. [2025-10-20T17:10:24.864172] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:10:25.051377] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-10. [2025-10-20T17:10:25.851472] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-01. [2025-10-20T17:10:26.191156] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T17:10:26.195215] [INFO ] [REMOTE ] ALTG: Full fetch complete (33412 records). [2025-10-20T17:10:26.195321] [DEBUG] [VALIDATOR ] ALTG: Validating primary data. [2025-10-20T17:10:26.625464] [DEBUG] [VALIDATOR ] ALTG: All expected trading days are present. [2025-10-20T17:10:26.625565] [DEBUG] [VALIDATOR ] ALTG: Primary data validation passed. [2025-10-20T17:10:26.626060] [INFO ] [MAIN ] ALTG: Fetched primary data is valid. [2025-10-20T17:10:26.626145] [DEBUG] [LOCAL ] ALTG: Saving primary data. [2025-10-20T17:10:26.627009] [DEBUG] [LOCAL ] ALTG: Trimmed 33412 records to 23275 (keeping 200 days of data) [2025-10-20T17:10:26.674633] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-03. [2025-10-20T17:10:26.764360] [DEBUG] [LOCAL ] ALTG: Primary data saved (23275 records). [2025-10-20T17:10:26.764501] [INFO ] [MAIN ] ALTG: Full history saved. [2025-10-20T17:10:26.764585] [INFO ] [MAIN ] Update Progress: 90/2367 symbols completed. ALTG took 24.5s. Estimated remaining: 4h 22m 26s. Rate: 8.7 symbols/min [2025-10-20T17:10:26.964522] [INFO ] [MAIN ] AM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:27.016318] [DEBUG] [LOCAL ] AM: Loaded 52670 existing records from primary storage. [2025-10-20T17:10:27.016691] [INFO ] [MAIN ] AM: Loaded 52670 existing records. [2025-10-20T17:10:27.199056] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T17:10:27.222711] [WARN ] [MAIN ] AM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:27.223041] [INFO ] [MAIN ] AM: Wiping existing primary data before full refresh. [2025-10-20T17:10:27.223228] [INFO ] [LOCAL ] AM: Starting deletion of primary data. [2025-10-20T17:10:27.223581] [DEBUG] [LOCAL ] AM: Deleted primary data file. [2025-10-20T17:10:27.223763] [INFO ] [MAIN ] AM: Starting full primary refresh. [2025-10-20T17:10:27.407432] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:10:27.457716] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-04. [2025-10-20T17:10:27.623198] [INFO ] [REMOTE ] AM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:28.135963] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T17:10:28.251406] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-08. [2025-10-20T17:10:29.051454] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-02. [2025-10-20T17:10:29.090410] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T17:10:29.851367] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-09. [2025-10-20T17:10:30.233325] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T17:10:30.296330] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T17:10:30.651397] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-05. [2025-10-20T17:10:31.211840] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T17:10:31.451358] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-01. [2025-10-20T17:10:32.251365] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-10. [2025-10-20T17:10:33.013780] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:10:33.051331] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-02. [2025-10-20T17:10:33.083044] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T17:10:33.084611] [INFO ] [REMOTE ] ALUR: Full fetch complete (18629 records). [2025-10-20T17:10:33.084710] [DEBUG] [VALIDATOR ] ALUR: Validating primary data. [2025-10-20T17:10:33.436107] [DEBUG] [VALIDATOR ] ALUR: All expected trading days are present. [2025-10-20T17:10:33.436185] [DEBUG] [VALIDATOR ] ALUR: Primary data validation passed. [2025-10-20T17:10:33.436459] [INFO ] [MAIN ] ALUR: Fetched primary data is valid. [2025-10-20T17:10:33.436502] [DEBUG] [LOCAL ] ALUR: Saving primary data. [2025-10-20T17:10:33.437036] [DEBUG] [LOCAL ] ALUR: Trimmed 18629 records to 9345 (keeping 200 days of data) [2025-10-20T17:10:33.487613] [DEBUG] [LOCAL ] ALUR: Primary data saved (9345 records). [2025-10-20T17:10:33.487779] [INFO ] [MAIN ] ALUR: Full history saved. [2025-10-20T17:10:33.487861] [INFO ] [MAIN ] Update Progress: 91/2367 symbols completed. ALUR took 25.1s. Estimated remaining: 4h 22m 14s. Rate: 8.7 symbols/min [2025-10-20T17:10:33.662682] [INFO ] [MAIN ] AMBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:33.696190] [DEBUG] [LOCAL ] AMBC: Loaded 36767 existing records from primary storage. [2025-10-20T17:10:33.696563] [INFO ] [MAIN ] AMBC: Loaded 36767 existing records. [2025-10-20T17:10:33.856552] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-03. [2025-10-20T17:10:33.888542] [WARN ] [MAIN ] AMBC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:33.889612] [INFO ] [MAIN ] AMBC: Wiping existing primary data before full refresh. [2025-10-20T17:10:33.889719] [INFO ] [LOCAL ] AMBC: Starting deletion of primary data. [2025-10-20T17:10:33.890025] [DEBUG] [LOCAL ] AMBC: Deleted primary data file. [2025-10-20T17:10:33.890096] [INFO ] [MAIN ] AMBC: Starting full primary refresh. [2025-10-20T17:10:34.239351] [INFO ] [REMOTE ] AMBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:34.389807] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T17:10:34.651401] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-03. [2025-10-20T17:10:35.377626] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:10:35.451422] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-06. [2025-10-20T17:10:36.091704] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T17:10:36.251453] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-04. [2025-10-20T17:10:36.390616] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T17:10:37.051414] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-04. [2025-10-20T17:10:37.851383] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-07. [2025-10-20T17:10:38.102264] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:10:38.382893] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T17:10:38.651357] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-08. [2025-10-20T17:10:38.839191] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:10:39.104731] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T17:10:39.451373] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-01. [2025-10-20T17:10:40.251373] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-05. [2025-10-20T17:10:40.397109] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T17:10:41.051408] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-02. [2025-10-20T17:10:41.851413] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-09. [2025-10-20T17:10:42.024540] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:10:42.334785] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T17:10:42.379531] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:10:42.651374] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-03. [2025-10-20T17:10:43.451399] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-06. [2025-10-20T17:10:44.251913] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-05. [2025-10-20T17:10:44.258699] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T17:10:44.908057] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T17:10:45.051365] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-10. [2025-10-20T17:10:45.762125] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:10:45.851354] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-04. [2025-10-20T17:10:46.051318] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:10:46.054372] [INFO ] [REMOTE ] ALX: Full fetch complete (7985 records). [2025-10-20T17:10:46.054476] [DEBUG] [VALIDATOR ] ALX: Validating primary data. [2025-10-20T17:10:46.435488] [DEBUG] [VALIDATOR ] ALX: All expected trading days are present. [2025-10-20T17:10:46.435577] [DEBUG] [VALIDATOR ] ALX: Primary data validation passed. [2025-10-20T17:10:46.435834] [INFO ] [MAIN ] ALX: Fetched primary data is valid. [2025-10-20T17:10:46.435895] [DEBUG] [LOCAL ] ALX: Saving primary data. [2025-10-20T17:10:46.436381] [DEBUG] [LOCAL ] ALX: Trimmed 7985 records to 7183 (keeping 200 days of data) [2025-10-20T17:10:46.482456] [DEBUG] [LOCAL ] ALX: Primary data saved (7183 records). [2025-10-20T17:10:46.482579] [INFO ] [MAIN ] ALX: Full history saved. [2025-10-20T17:10:46.482676] [INFO ] [MAIN ] Update Progress: 92/2367 symbols completed. ALX took 25.6s. Estimated remaining: 4h 24m 37s. Rate: 8.6 symbols/min [2025-10-20T17:10:46.651110] [INFO ] [MAIN ] AMBP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:46.651404] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-07. [2025-10-20T17:10:46.692729] [DEBUG] [LOCAL ] AMBP: Loaded 45132 existing records from primary storage. [2025-10-20T17:10:46.693062] [INFO ] [MAIN ] AMBP: Loaded 45132 existing records. [2025-10-20T17:10:46.877562] [WARN ] [MAIN ] AMBP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:46.877782] [INFO ] [MAIN ] AMBP: Wiping existing primary data before full refresh. [2025-10-20T17:10:46.877831] [INFO ] [LOCAL ] AMBP: Starting deletion of primary data. [2025-10-20T17:10:46.878086] [DEBUG] [LOCAL ] AMBP: Deleted primary data file. [2025-10-20T17:10:46.878142] [INFO ] [MAIN ] AMBP: Starting full primary refresh. [2025-10-20T17:10:47.244636] [INFO ] [REMOTE ] AMBP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:47.321703] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:10:47.451451] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-05. [2025-10-20T17:10:48.251430] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-06. [2025-10-20T17:10:48.472403] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T17:10:48.705204] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T17:10:49.051359] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-06. [2025-10-20T17:10:49.645174] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T17:10:49.851421] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-01. [2025-10-20T17:10:50.339308] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:10:50.651408] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-08. [2025-10-20T17:10:51.231591] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T17:10:51.451377] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-07. [2025-10-20T17:10:52.202874] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T17:10:52.251360] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-09. [2025-10-20T17:10:53.051380] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-07. [2025-10-20T17:10:53.098558] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:10:53.687541] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T17:10:53.851411] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-08. [2025-10-20T17:10:54.136397] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T17:10:54.651377] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-10. [2025-10-20T17:10:55.450314] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T17:10:55.458518] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-02. [2025-10-20T17:10:56.251372] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-09. [2025-10-20T17:10:56.570455] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:10:56.578429] [INFO ] [REMOTE ] AM: Full fetch complete (75691 records). [2025-10-20T17:10:56.578617] [DEBUG] [VALIDATOR ] AM: Validating primary data. [2025-10-20T17:10:56.944313] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:10:57.064577] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-03. [2025-10-20T17:10:57.068798] [DEBUG] [VALIDATOR ] AM: All expected trading days are present. [2025-10-20T17:10:57.069827] [DEBUG] [VALIDATOR ] AM: Primary data validation passed. [2025-10-20T17:10:57.070696] [INFO ] [MAIN ] AM: Fetched primary data is valid. [2025-10-20T17:10:57.070763] [DEBUG] [LOCAL ] AM: Saving primary data. [2025-10-20T17:10:57.072423] [DEBUG] [LOCAL ] AM: Trimmed 75691 records to 51887 (keeping 200 days of data) [2025-10-20T17:10:57.395525] [DEBUG] [LOCAL ] AM: Primary data saved (51887 records). [2025-10-20T17:10:57.395756] [INFO ] [MAIN ] AM: Full history saved. [2025-10-20T17:10:57.395879] [INFO ] [MAIN ] Update Progress: 93/2367 symbols completed. AM took 30.6s. Estimated remaining: 4h 26m 7s. Rate: 8.5 symbols/min [2025-10-20T17:10:57.534800] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T17:10:57.612932] [INFO ] [MAIN ] AMBQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:57.613112] [DEBUG] [LOCAL ] AMBQ: No existing data file found in primary storage. [2025-10-20T17:10:57.613149] [WARN ] [MAIN ] AMBQ: No existing data found, performing full refresh instead. [2025-10-20T17:10:57.613258] [INFO ] [MAIN ] AMBQ: Wiping existing primary data before full refresh. [2025-10-20T17:10:57.613300] [INFO ] [LOCAL ] AMBQ: Starting deletion of primary data. [2025-10-20T17:10:57.613340] [DEBUG] [LOCAL ] AMBQ: No primary data file found to delete. [2025-10-20T17:10:57.613371] [INFO ] [MAIN ] AMBQ: Starting full primary refresh. [2025-10-20T17:10:57.858150] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-10. [2025-10-20T17:10:57.974182] [INFO ] [REMOTE ] AMBQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:58.651365] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-01. [2025-10-20T17:10:58.741563] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T17:10:58.849984] [DEBUG] [REMOTE ] AMBQ: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:10:58.850289] [WARN ] [REMOTE ] AMBQ: No data returned for month 2025-01 [2025-10-20T17:10:58.850372] [FATAL] [REMOTE ] AMBQ: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:10:58.850420] [FATAL] [MAIN ] AMBQ: Failed to fetch primary data. [2025-10-20T17:10:58.850478] [WARN ] [MAIN ] AMBQ: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:10:58.850552] [INFO ] [LOCAL ] AMBQ: Starting deletion of primary data. [2025-10-20T17:10:58.850615] [DEBUG] [LOCAL ] AMBQ: No primary data file found to delete. [2025-10-20T17:10:58.850692] [INFO ] [LOCAL ] AMBQ: Starting deletion of derivative data. [2025-10-20T17:10:58.850794] [DEBUG] [LOCAL ] AMBQ: No derivative data file found to delete. [2025-10-20T17:10:58.850881] [INFO ] [MAIN ] Update Progress: 94/2367 symbols completed. AMBQ took 1.5s. Estimated remaining: 4h 23m 45s. Rate: 8.6 symbols/min [2025-10-20T17:10:59.026887] [INFO ] [MAIN ] AMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:59.080887] [DEBUG] [LOCAL ] AMC: Loaded 54542 existing records from primary storage. [2025-10-20T17:10:59.081260] [INFO ] [MAIN ] AMC: Loaded 54542 existing records. [2025-10-20T17:10:59.284087] [WARN ] [MAIN ] AMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:59.284382] [INFO ] [MAIN ] AMC: Wiping existing primary data before full refresh. [2025-10-20T17:10:59.284435] [INFO ] [LOCAL ] AMC: Starting deletion of primary data. [2025-10-20T17:10:59.284612] [DEBUG] [LOCAL ] AMC: Deleted primary data file. [2025-10-20T17:10:59.284689] [INFO ] [MAIN ] AMC: Starting full primary refresh. [2025-10-20T17:10:59.458470] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-08. [2025-10-20T17:10:59.652245] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T17:10:59.656514] [INFO ] [REMOTE ] AMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:59.663012] [INFO ] [REMOTE ] ALV: Full fetch complete (62304 records). [2025-10-20T17:10:59.663104] [DEBUG] [VALIDATOR ] ALV: Validating primary data. [2025-10-20T17:11:00.052164] [DEBUG] [VALIDATOR ] ALV: All expected trading days are present. [2025-10-20T17:11:00.052254] [DEBUG] [VALIDATOR ] ALV: Primary data validation passed. [2025-10-20T17:11:00.053041] [INFO ] [MAIN ] ALV: Fetched primary data is valid. [2025-10-20T17:11:00.053116] [DEBUG] [LOCAL ] ALV: Saving primary data. [2025-10-20T17:11:00.054349] [DEBUG] [LOCAL ] ALV: Trimmed 62304 records to 42190 (keeping 200 days of data) [2025-10-20T17:11:00.265986] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-04. [2025-10-20T17:11:00.315679] [DEBUG] [LOCAL ] ALV: Primary data saved (42190 records). [2025-10-20T17:11:00.315803] [INFO ] [MAIN ] ALV: Full history saved. [2025-10-20T17:11:00.316592] [INFO ] [MAIN ] Update Progress: 95/2367 symbols completed. ALV took 42.8s. Estimated remaining: 4h 21m 27s. Rate: 8.7 symbols/min [2025-10-20T17:11:00.493938] [INFO ] [MAIN ] AMCR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:00.547386] [DEBUG] [LOCAL ] AMCR: Loaded 54449 existing records from primary storage. [2025-10-20T17:11:00.547797] [INFO ] [MAIN ] AMCR: Loaded 54449 existing records. [2025-10-20T17:11:00.737906] [WARN ] [MAIN ] AMCR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:00.738292] [INFO ] [MAIN ] AMCR: Wiping existing primary data before full refresh. [2025-10-20T17:11:00.738353] [INFO ] [LOCAL ] AMCR: Starting deletion of primary data. [2025-10-20T17:11:00.758912] [DEBUG] [LOCAL ] AMCR: Deleted primary data file. [2025-10-20T17:11:00.759055] [INFO ] [MAIN ] AMCR: Starting full primary refresh. [2025-10-20T17:11:00.831841] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T17:11:01.056559] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-01. [2025-10-20T17:11:01.141203] [INFO ] [REMOTE ] AMCR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:01.633490] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T17:11:01.851363] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-01. [2025-10-20T17:11:02.651373] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-05. [2025-10-20T17:11:03.451355] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-09. [2025-10-20T17:11:03.535539] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-01 (2.48 seconds). [2025-10-20T17:11:03.709972] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:11:04.251358] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-02. [2025-10-20T17:11:04.404446] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T17:11:04.835286] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:11:05.051365] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-02. [2025-10-20T17:11:05.650813] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T17:11:05.851382] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-10. [2025-10-20T17:11:06.651372] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-06. [2025-10-20T17:11:06.956590] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T17:11:07.259048] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:11:07.264837] [INFO ] [REMOTE ] AMBC: Full fetch complete (51907 records). [2025-10-20T17:11:07.265058] [DEBUG] [VALIDATOR ] AMBC: Validating primary data. [2025-10-20T17:11:07.456506] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-03. [2025-10-20T17:11:07.648712] [DEBUG] [VALIDATOR ] AMBC: All expected trading days are present. [2025-10-20T17:11:07.648813] [DEBUG] [VALIDATOR ] AMBC: Primary data validation passed. [2025-10-20T17:11:07.649447] [INFO ] [MAIN ] AMBC: Fetched primary data is valid. [2025-10-20T17:11:07.649527] [DEBUG] [LOCAL ] AMBC: Saving primary data. [2025-10-20T17:11:07.650796] [DEBUG] [LOCAL ] AMBC: Trimmed 51907 records to 36189 (keeping 200 days of data) [2025-10-20T17:11:07.789178] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:11:07.893128] [DEBUG] [LOCAL ] AMBC: Primary data saved (36189 records). [2025-10-20T17:11:07.893284] [INFO ] [MAIN ] AMBC: Full history saved. [2025-10-20T17:11:07.894134] [INFO ] [MAIN ] Update Progress: 96/2367 symbols completed. AMBC took 34.4s. Estimated remaining: 4h 21m 36s. Rate: 8.7 symbols/min [2025-10-20T17:11:08.066739] [INFO ] [MAIN ] AME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:08.116390] [DEBUG] [LOCAL ] AME: Loaded 49338 existing records from primary storage. [2025-10-20T17:11:08.116712] [INFO ] [MAIN ] AME: Loaded 49338 existing records. [2025-10-20T17:11:08.257226] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-07. [2025-10-20T17:11:08.304575] [WARN ] [MAIN ] AME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:08.304862] [INFO ] [MAIN ] AME: Wiping existing primary data before full refresh. [2025-10-20T17:11:08.304917] [INFO ] [LOCAL ] AME: Starting deletion of primary data. [2025-10-20T17:11:08.305143] [DEBUG] [LOCAL ] AME: Deleted primary data file. [2025-10-20T17:11:08.305200] [INFO ] [MAIN ] AME: Starting full primary refresh. [2025-10-20T17:11:08.705877] [INFO ] [REMOTE ] AME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:09.051391] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-03. [2025-10-20T17:11:09.686903] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-03 (2.23 seconds). [2025-10-20T17:11:09.757925] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:11:09.851332] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-08. [2025-10-20T17:11:10.606056] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:11:10.651347] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-01. [2025-10-20T17:11:11.082574] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T17:11:11.451398] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-04. [2025-10-20T17:11:12.251362] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-09. [2025-10-20T17:11:12.421296] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T17:11:13.051396] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-02. [2025-10-20T17:11:13.596424] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:11:13.624887] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T17:11:13.851377] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-05. [2025-10-20T17:11:14.410092] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T17:11:14.651385] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-04. [2025-10-20T17:11:15.451460] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-03. [2025-10-20T17:11:16.251378] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-10. [2025-10-20T17:11:16.338775] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-05 (2.49 seconds). [2025-10-20T17:11:16.600624] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T17:11:16.912162] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T17:11:17.051396] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-04. [2025-10-20T17:11:17.694277] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T17:11:17.701389] [INFO ] [REMOTE ] AMBP: Full fetch complete (64868 records). [2025-10-20T17:11:17.701511] [DEBUG] [VALIDATOR ] AMBP: Validating primary data. [2025-10-20T17:11:17.702208] [FATAL] [VALIDATOR ] AMBP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:11:17.702291] [FATAL] [MAIN ] AMBP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:11:17.702334] [WARN ] [MAIN ] AMBP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:11:17.702403] [INFO ] [LOCAL ] AMBP: Starting deletion of primary data. [2025-10-20T17:11:17.702466] [DEBUG] [LOCAL ] AMBP: No primary data file found to delete. [2025-10-20T17:11:17.702503] [INFO ] [LOCAL ] AMBP: Starting deletion of derivative data. [2025-10-20T17:11:17.702722] [DEBUG] [LOCAL ] AMBP: Deleted derivative data file. [2025-10-20T17:11:17.703186] [INFO ] [MAIN ] Update Progress: 97/2367 symbols completed. AMBP took 31.2s. Estimated remaining: 4h 22m 37s. Rate: 8.6 symbols/min [2025-10-20T17:11:17.851898] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-05. [2025-10-20T17:11:17.871845] [INFO ] [MAIN ] AMG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:17.892866] [DEBUG] [LOCAL ] AMG: Loaded 20485 existing records from primary storage. [2025-10-20T17:11:17.893117] [INFO ] [MAIN ] AMG: Loaded 20485 existing records. [2025-10-20T17:11:18.067621] [WARN ] [MAIN ] AMG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:18.067842] [INFO ] [MAIN ] AMG: Wiping existing primary data before full refresh. [2025-10-20T17:11:18.067889] [INFO ] [LOCAL ] AMG: Starting deletion of primary data. [2025-10-20T17:11:18.068125] [DEBUG] [LOCAL ] AMG: Deleted primary data file. [2025-10-20T17:11:18.068181] [INFO ] [MAIN ] AMG: Starting full primary refresh. [2025-10-20T17:11:18.419753] [INFO ] [REMOTE ] AMG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:18.651397] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-06. [2025-10-20T17:11:18.870861] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T17:11:19.451355] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-01. [2025-10-20T17:11:19.665051] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:11:20.251382] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-06. [2025-10-20T17:11:20.360937] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:11:20.974813] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-06 (2.32 seconds). [2025-10-20T17:11:21.051344] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-07. [2025-10-20T17:11:21.664856] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T17:11:21.851433] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-07. [2025-10-20T17:11:22.651453] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-05. [2025-10-20T17:11:23.342687] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-07 (2.29 seconds). [2025-10-20T17:11:23.451386] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-08. [2025-10-20T17:11:23.764937] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T17:11:24.251354] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-02. [2025-10-20T17:11:24.434091] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T17:11:24.919210] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T17:11:25.051347] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-06. [2025-10-20T17:11:25.851406] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-03. [2025-10-20T17:11:25.971110] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-08 (2.52 seconds). [2025-10-20T17:11:26.651371] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-09. [2025-10-20T17:11:26.856701] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T17:11:27.033779] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T17:11:27.451392] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-04. [2025-10-20T17:11:28.251394] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-07. [2025-10-20T17:11:28.373568] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T17:11:28.984001] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-09 (2.33 seconds). [2025-10-20T17:11:29.051367] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-10. [2025-10-20T17:11:29.851414] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-08. [2025-10-20T17:11:30.082090] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T17:11:30.651425] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-05. [2025-10-20T17:11:31.451340] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-08. [2025-10-20T17:11:31.572153] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-10 (2.52 seconds). [2025-10-20T17:11:31.571478] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T17:11:31.590771] [INFO ] [REMOTE ] AMC: Full fetch complete (77930 records). [2025-10-20T17:11:31.590929] [DEBUG] [VALIDATOR ] AMC: Validating primary data. [2025-10-20T17:11:31.768771] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T17:11:32.005767] [DEBUG] [VALIDATOR ] AMC: All expected trading days are present. [2025-10-20T17:11:32.005859] [DEBUG] [VALIDATOR ] AMC: Primary data validation passed. [2025-10-20T17:11:32.006590] [INFO ] [MAIN ] AMC: Fetched primary data is valid. [2025-10-20T17:11:32.006635] [DEBUG] [LOCAL ] AMC: Saving primary data. [2025-10-20T17:11:32.008133] [DEBUG] [LOCAL ] AMC: Trimmed 77930 records to 53766 (keeping 200 days of data) [2025-10-20T17:11:32.256975] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-09. [2025-10-20T17:11:32.304350] [DEBUG] [LOCAL ] AMC: Primary data saved (53766 records). [2025-10-20T17:11:32.304450] [INFO ] [MAIN ] AMC: Full history saved. [2025-10-20T17:11:32.304524] [INFO ] [MAIN ] Update Progress: 98/2367 symbols completed. AMC took 33.5s. Estimated remaining: 4h 25m 27s. Rate: 8.5 symbols/min [2025-10-20T17:11:32.477608] [INFO ] [MAIN ] AMH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:32.536004] [DEBUG] [LOCAL ] AMH: Loaded 52597 existing records from primary storage. [2025-10-20T17:11:32.536408] [INFO ] [MAIN ] AMH: Loaded 52597 existing records. [2025-10-20T17:11:32.732176] [WARN ] [MAIN ] AMH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:32.732504] [INFO ] [MAIN ] AMH: Wiping existing primary data before full refresh. [2025-10-20T17:11:32.732565] [INFO ] [LOCAL ] AMH: Starting deletion of primary data. [2025-10-20T17:11:32.732952] [DEBUG] [LOCAL ] AMH: Deleted primary data file. [2025-10-20T17:11:32.733011] [INFO ] [MAIN ] AMH: Starting full primary refresh. [2025-10-20T17:11:33.053729] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-06. [2025-10-20T17:11:33.117355] [INFO ] [REMOTE ] AMH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:33.148380] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T17:11:33.851355] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-01. [2025-10-20T17:11:33.994583] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:11:34.103402] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T17:11:34.651406] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-10. [2025-10-20T17:11:35.058231] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:11:35.451397] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-07. [2025-10-20T17:11:36.165387] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T17:11:36.251369] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-08. [2025-10-20T17:11:36.861069] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T17:11:36.870088] [INFO ] [REMOTE ] AMCR: Full fetch complete (77842 records). [2025-10-20T17:11:36.870255] [DEBUG] [VALIDATOR ] AMCR: Validating primary data. [2025-10-20T17:11:37.061719] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-09. [2025-10-20T17:11:37.359519] [DEBUG] [VALIDATOR ] AMCR: All expected trading days are present. [2025-10-20T17:11:37.359626] [DEBUG] [VALIDATOR ] AMCR: Primary data validation passed. [2025-10-20T17:11:37.360421] [INFO ] [MAIN ] AMCR: Fetched primary data is valid. [2025-10-20T17:11:37.365684] [DEBUG] [LOCAL ] AMCR: Saving primary data. [2025-10-20T17:11:37.371956] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:11:37.376913] [DEBUG] [LOCAL ] AMCR: Trimmed 77842 records to 53670 (keeping 200 days of data) [2025-10-20T17:11:37.697677] [DEBUG] [LOCAL ] AMCR: Primary data saved (53670 records). [2025-10-20T17:11:37.697858] [INFO ] [MAIN ] AMCR: Full history saved. [2025-10-20T17:11:37.698639] [INFO ] [MAIN ] Update Progress: 99/2367 symbols completed. AMCR took 37.4s. Estimated remaining: 4h 24m 43s. Rate: 8.6 symbols/min [2025-10-20T17:11:37.856444] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-02. [2025-10-20T17:11:37.883490] [INFO ] [MAIN ] AMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:37.883714] [DEBUG] [LOCAL ] AMN: No existing data file found in primary storage. [2025-10-20T17:11:37.883772] [WARN ] [MAIN ] AMN: No existing data found, performing full refresh instead. [2025-10-20T17:11:37.883895] [INFO ] [MAIN ] AMN: Wiping existing primary data before full refresh. [2025-10-20T17:11:37.883959] [INFO ] [LOCAL ] AMN: Starting deletion of primary data. [2025-10-20T17:11:37.884018] [DEBUG] [LOCAL ] AMN: No primary data file found to delete. [2025-10-20T17:11:37.884064] [INFO ] [MAIN ] AMN: Starting full primary refresh. [2025-10-20T17:11:38.253688] [INFO ] [REMOTE ] AMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:38.651419] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-01. [2025-10-20T17:11:38.875582] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T17:11:39.263902] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T17:11:39.451435] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-10. [2025-10-20T17:11:39.518748] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T17:11:40.251379] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-09. [2025-10-20T17:11:41.051352] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-02. [2025-10-20T17:11:41.129531] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T17:11:41.136801] [INFO ] [REMOTE ] AME: Full fetch complete (69996 records). [2025-10-20T17:11:41.136937] [DEBUG] [VALIDATOR ] AME: Validating primary data. [2025-10-20T17:11:41.480552] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T17:11:41.559401] [DEBUG] [VALIDATOR ] AME: All expected trading days are present. [2025-10-20T17:11:41.559479] [DEBUG] [VALIDATOR ] AME: Primary data validation passed. [2025-10-20T17:11:41.560188] [INFO ] [MAIN ] AME: Fetched primary data is valid. [2025-10-20T17:11:41.560251] [DEBUG] [LOCAL ] AME: Saving primary data. [2025-10-20T17:11:41.561572] [DEBUG] [LOCAL ] AME: Trimmed 69996 records to 48570 (keeping 200 days of data) [2025-10-20T17:11:41.857800] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-03. [2025-10-20T17:11:41.889772] [DEBUG] [LOCAL ] AME: Primary data saved (48570 records). [2025-10-20T17:11:41.889896] [INFO ] [MAIN ] AME: Full history saved. [2025-10-20T17:11:41.890866] [INFO ] [MAIN ] Update Progress: 100/2367 symbols completed. AME took 34.0s. Estimated remaining: 4h 23m 32s. Rate: 8.6 symbols/min [2025-10-20T17:11:42.065409] [INFO ] [MAIN ] AMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:42.098746] [DEBUG] [LOCAL ] AMP: Loaded 32543 existing records from primary storage. [2025-10-20T17:11:42.099028] [INFO ] [MAIN ] AMP: Loaded 32543 existing records. [2025-10-20T17:11:42.284678] [WARN ] [MAIN ] AMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:42.284889] [INFO ] [MAIN ] AMP: Wiping existing primary data before full refresh. [2025-10-20T17:11:42.284935] [INFO ] [LOCAL ] AMP: Starting deletion of primary data. [2025-10-20T17:11:42.285218] [DEBUG] [LOCAL ] AMP: Deleted primary data file. [2025-10-20T17:11:42.285275] [INFO ] [MAIN ] AMP: Starting full primary refresh. [2025-10-20T17:11:42.661082] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-10. [2025-10-20T17:11:42.666670] [INFO ] [REMOTE ] AMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:42.696846] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:11:43.340801] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T17:11:43.451363] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-01. [2025-10-20T17:11:43.872737] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T17:11:43.876059] [INFO ] [REMOTE ] AMG: Full fetch complete (29606 records). [2025-10-20T17:11:43.876141] [DEBUG] [VALIDATOR ] AMG: Validating primary data. [2025-10-20T17:11:44.251899] [DEBUG] [VALIDATOR ] AMG: All expected trading days are present. [2025-10-20T17:11:44.251759] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-04. [2025-10-20T17:11:44.252997] [DEBUG] [VALIDATOR ] AMG: Primary data validation passed. [2025-10-20T17:11:44.253407] [INFO ] [MAIN ] AMG: Fetched primary data is valid. [2025-10-20T17:11:44.253469] [DEBUG] [LOCAL ] AMG: Saving primary data. [2025-10-20T17:11:44.254295] [DEBUG] [LOCAL ] AMG: Trimmed 29606 records to 20174 (keeping 200 days of data) [2025-10-20T17:11:44.380788] [DEBUG] [LOCAL ] AMG: Primary data saved (20174 records). [2025-10-20T17:11:44.380918] [INFO ] [MAIN ] AMG: Full history saved. [2025-10-20T17:11:44.380990] [INFO ] [MAIN ] Update Progress: 101/2367 symbols completed. AMG took 26.7s. Estimated remaining: 4h 21m 44s. Rate: 8.7 symbols/min [2025-10-20T17:11:44.554150] [INFO ] [MAIN ] AMPX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:44.601915] [DEBUG] [LOCAL ] AMPX: Loaded 53118 existing records from primary storage. [2025-10-20T17:11:44.602264] [INFO ] [MAIN ] AMPX: Loaded 53118 existing records. [2025-10-20T17:11:44.786255] [WARN ] [MAIN ] AMPX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:44.786616] [INFO ] [MAIN ] AMPX: Wiping existing primary data before full refresh. [2025-10-20T17:11:44.786699] [INFO ] [LOCAL ] AMPX: Starting deletion of primary data. [2025-10-20T17:11:44.787074] [DEBUG] [LOCAL ] AMPX: Deleted primary data file. [2025-10-20T17:11:44.787129] [INFO ] [MAIN ] AMPX: Starting full primary refresh. [2025-10-20T17:11:45.058507] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-03. [2025-10-20T17:11:45.187327] [INFO ] [REMOTE ] AMPX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:45.214810] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T17:11:45.851369] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-01. [2025-10-20T17:11:45.990436] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T17:11:46.651353] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-05. [2025-10-20T17:11:46.753575] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T17:11:47.451378] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-04. [2025-10-20T17:11:47.643838] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:11:48.251354] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-02. [2025-10-20T17:11:48.426323] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:11:48.982443] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T17:11:49.051340] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-05. [2025-10-20T17:11:49.738165] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:11:49.851344] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-03. [2025-10-20T17:11:50.539874] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T17:11:50.651391] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-06. [2025-10-20T17:11:51.451419] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-02. [2025-10-20T17:11:51.871057] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T17:11:51.942149] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:11:52.251364] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-07. [2025-10-20T17:11:52.533228] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:11:53.051346] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-06. [2025-10-20T17:11:53.851378] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-03. [2025-10-20T17:11:54.045507] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:11:54.399558] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:11:54.651373] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-07. [2025-10-20T17:11:55.158164] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T17:11:55.451378] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-08. [2025-10-20T17:11:56.151274] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:11:56.251333] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-04. [2025-10-20T17:11:57.051390] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-08. [2025-10-20T17:11:57.185432] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T17:11:57.851508] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-09. [2025-10-20T17:11:57.887163] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:11:58.651394] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-04. [2025-10-20T17:11:58.830498] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T17:11:59.451370] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-09. [2025-10-20T17:11:59.615274] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T17:12:00.109376] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:12:00.251431] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-10. [2025-10-20T17:12:00.802037] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:12:01.051383] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-05. [2025-10-20T17:12:01.851397] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-05. [2025-10-20T17:12:02.198143] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T17:12:02.206023] [INFO ] [REMOTE ] AMH: Full fetch complete (75082 records). [2025-10-20T17:12:02.206188] [DEBUG] [VALIDATOR ] AMH: Validating primary data. [2025-10-20T17:12:02.617465] [DEBUG] [VALIDATOR ] AMH: All expected trading days are present. [2025-10-20T17:12:02.617683] [DEBUG] [VALIDATOR ] AMH: Primary data validation passed. [2025-10-20T17:12:02.618751] [INFO ] [MAIN ] AMH: Fetched primary data is valid. [2025-10-20T17:12:02.618842] [DEBUG] [LOCAL ] AMH: Saving primary data. [2025-10-20T17:12:02.644637] [DEBUG] [LOCAL ] AMH: Trimmed 75082 records to 51831 (keeping 200 days of data) [2025-10-20T17:12:02.646339] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T17:12:02.657071] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-10. [2025-10-20T17:12:02.980394] [DEBUG] [LOCAL ] AMH: Primary data saved (51831 records). [2025-10-20T17:12:02.980536] [INFO ] [MAIN ] AMH: Full history saved. [2025-10-20T17:12:02.980614] [INFO ] [MAIN ] Update Progress: 102/2367 symbols completed. AMH took 30.7s. Estimated remaining: 4h 25m 57s. Rate: 8.5 symbols/min [2025-10-20T17:12:03.148708] [INFO ] [MAIN ] AMPY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:03.185970] [DEBUG] [LOCAL ] AMPY: Loaded 39876 existing records from primary storage. [2025-10-20T17:12:03.186273] [INFO ] [MAIN ] AMPY: Loaded 39876 existing records. [2025-10-20T17:12:03.369699] [WARN ] [MAIN ] AMPY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:03.370003] [INFO ] [MAIN ] AMPY: Wiping existing primary data before full refresh. [2025-10-20T17:12:03.370112] [INFO ] [LOCAL ] AMPY: Starting deletion of primary data. [2025-10-20T17:12:03.370404] [DEBUG] [LOCAL ] AMPY: Deleted primary data file. [2025-10-20T17:12:03.370516] [INFO ] [MAIN ] AMPY: Starting full primary refresh. [2025-10-20T17:12:03.468236] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-06. [2025-10-20T17:12:03.682444] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T17:12:03.786702] [INFO ] [REMOTE ] AMPY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:04.251428] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-06. [2025-10-20T17:12:04.438867] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T17:12:04.445178] [INFO ] [REMOTE ] AMN: Full fetch complete (60282 records). [2025-10-20T17:12:04.445293] [DEBUG] [VALIDATOR ] AMN: Validating primary data. [2025-10-20T17:12:04.918903] [DEBUG] [VALIDATOR ] AMN: All expected trading days are present. [2025-10-20T17:12:04.919123] [DEBUG] [VALIDATOR ] AMN: Primary data validation passed. [2025-10-20T17:12:04.920089] [INFO ] [MAIN ] AMN: Fetched primary data is valid. [2025-10-20T17:12:04.920183] [DEBUG] [LOCAL ] AMN: Saving primary data. [2025-10-20T17:12:04.942114] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T17:12:04.949764] [DEBUG] [LOCAL ] AMN: Trimmed 60282 records to 41731 (keeping 200 days of data) [2025-10-20T17:12:05.053764] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-07. [2025-10-20T17:12:05.216301] [DEBUG] [LOCAL ] AMN: Primary data saved (41731 records). [2025-10-20T17:12:05.216471] [INFO ] [MAIN ] AMN: Full history saved. [2025-10-20T17:12:05.216557] [INFO ] [MAIN ] Update Progress: 103/2367 symbols completed. AMN took 27.5s. Estimated remaining: 4h 24m 4s. Rate: 8.6 symbols/min [2025-10-20T17:12:05.407417] [INFO ] [MAIN ] AMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:05.441735] [DEBUG] [LOCAL ] AMR: Loaded 29322 existing records from primary storage. [2025-10-20T17:12:05.442217] [INFO ] [MAIN ] AMR: Loaded 29322 existing records. [2025-10-20T17:12:05.656571] [WARN ] [MAIN ] AMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:05.656894] [INFO ] [MAIN ] AMR: Wiping existing primary data before full refresh. [2025-10-20T17:12:05.656947] [INFO ] [LOCAL ] AMR: Starting deletion of primary data. [2025-10-20T17:12:05.657226] [DEBUG] [LOCAL ] AMR: Deleted primary data file. [2025-10-20T17:12:05.657288] [INFO ] [MAIN ] AMR: Starting full primary refresh. [2025-10-20T17:12:05.858255] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-01. [2025-10-20T17:12:06.083144] [INFO ] [REMOTE ] AMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:06.167555] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:12:06.406906] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T17:12:06.651388] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-07. [2025-10-20T17:12:07.159856] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T17:12:07.451393] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-02. [2025-10-20T17:12:08.251369] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-01. [2025-10-20T17:12:08.553107] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T17:12:08.606330] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T17:12:09.051400] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-08. [2025-10-20T17:12:09.594339] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:12:09.851387] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-08. [2025-10-20T17:12:10.651391] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-03. [2025-10-20T17:12:10.772025] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T17:12:11.456788] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-02. [2025-10-20T17:12:11.462494] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-08 (2.41 seconds). [2025-10-20T17:12:11.903421] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T17:12:12.251420] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-04. [2025-10-20T17:12:12.433120] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:12:13.051381] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-09. [2025-10-20T17:12:13.742113] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T17:12:13.851443] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-03. [2025-10-20T17:12:14.084332] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T17:12:14.651453] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-09. [2025-10-20T17:12:15.147695] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T17:12:15.451360] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-04. [2025-10-20T17:12:16.251436] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-05. [2025-10-20T17:12:16.664868] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:12:16.707121] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T17:12:17.051412] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-10. [2025-10-20T17:12:17.389238] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:12:17.851410] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-10. [2025-10-20T17:12:18.651374] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-06. [2025-10-20T17:12:19.004299] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T17:12:19.012897] [INFO ] [REMOTE ] AMPX: Full fetch complete (75220 records). [2025-10-20T17:12:19.013053] [DEBUG] [VALIDATOR ] AMPX: Validating primary data. [2025-10-20T17:12:19.412720] [DEBUG] [VALIDATOR ] AMPX: All expected trading days are present. [2025-10-20T17:12:19.412886] [DEBUG] [VALIDATOR ] AMPX: Primary data validation passed. [2025-10-20T17:12:19.413680] [INFO ] [MAIN ] AMPX: Fetched primary data is valid. [2025-10-20T17:12:19.413848] [DEBUG] [LOCAL ] AMPX: Saving primary data. [2025-10-20T17:12:19.427475] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:12:19.432549] [DEBUG] [LOCAL ] AMPX: Trimmed 75220 records to 52385 (keeping 200 days of data) [2025-10-20T17:12:19.433147] [INFO ] [REMOTE ] AMP: Full fetch complete (47590 records). [2025-10-20T17:12:19.433230] [DEBUG] [VALIDATOR ] AMP: Validating primary data. [2025-10-20T17:12:19.456476] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-05. [2025-10-20T17:12:20.096283] [DEBUG] [VALIDATOR ] AMP: All expected trading days are present. [2025-10-20T17:12:20.102478] [DEBUG] [VALIDATOR ] AMP: Primary data validation passed. [2025-10-20T17:12:20.103243] [INFO ] [MAIN ] AMP: Fetched primary data is valid. [2025-10-20T17:12:20.103438] [DEBUG] [LOCAL ] AMP: Saving primary data. [2025-10-20T17:12:20.201427] [DEBUG] [LOCAL ] AMPX: Primary data saved (52385 records). [2025-10-20T17:12:20.203153] [INFO ] [MAIN ] AMPX: Full history saved. [2025-10-20T17:12:20.203375] [INFO ] [MAIN ] Update Progress: 104/2367 symbols completed. AMPX took 35.8s. Estimated remaining: 4h 26m 51s. Rate: 8.5 symbols/min [2025-10-20T17:12:20.203025] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:12:20.218296] [DEBUG] [LOCAL ] AMP: Trimmed 47590 records to 31952 (keeping 200 days of data) [2025-10-20T17:12:20.353502] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-07. [2025-10-20T17:12:20.508055] [INFO ] [MAIN ] AMRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:20.595755] [DEBUG] [LOCAL ] AMRC: Loaded 39116 existing records from primary storage. [2025-10-20T17:12:20.620032] [INFO ] [MAIN ] AMRC: Loaded 39116 existing records. [2025-10-20T17:12:20.633673] [DEBUG] [LOCAL ] AMP: Primary data saved (31952 records). [2025-10-20T17:12:20.633931] [INFO ] [MAIN ] AMP: Full history saved. [2025-10-20T17:12:20.634753] [INFO ] [MAIN ] Update Progress: 105/2367 symbols completed. AMP took 38.7s. Estimated remaining: 4h 24m 20s. Rate: 8.6 symbols/min [2025-10-20T17:12:20.959825] [WARN ] [MAIN ] AMRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:20.986881] [INFO ] [MAIN ] AMRC: Wiping existing primary data before full refresh. [2025-10-20T17:12:21.007791] [INFO ] [MAIN ] AMRZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:21.008088] [DEBUG] [LOCAL ] AMRZ: No existing data file found in primary storage. [2025-10-20T17:12:21.008301] [WARN ] [MAIN ] AMRZ: No existing data found, performing full refresh instead. [2025-10-20T17:12:21.008532] [INFO ] [MAIN ] AMRZ: Wiping existing primary data before full refresh. [2025-10-20T17:12:21.008639] [INFO ] [LOCAL ] AMRZ: Starting deletion of primary data. [2025-10-20T17:12:21.008719] [INFO ] [LOCAL ] AMRC: Starting deletion of primary data. [2025-10-20T17:12:21.008946] [DEBUG] [LOCAL ] AMRZ: No primary data file found to delete. [2025-10-20T17:12:21.009136] [INFO ] [MAIN ] AMRZ: Starting full primary refresh. [2025-10-20T17:12:21.052708] [DEBUG] [LOCAL ] AMRC: Deleted primary data file. [2025-10-20T17:12:21.079202] [INFO ] [MAIN ] AMRC: Starting full primary refresh. [2025-10-20T17:12:21.231571] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T17:12:21.270283] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-06. [2025-10-20T17:12:21.745842] [INFO ] [REMOTE ] AMRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:21.788205] [INFO ] [REMOTE ] AMRZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:21.814559] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T17:12:21.851333] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-08. [2025-10-20T17:12:22.315224] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:12:22.651426] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-07. [2025-10-20T17:12:23.097342] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T17:12:23.451389] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-01. [2025-10-20T17:12:23.642391] [DEBUG] [REMOTE ] AMRZ: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:12:23.642762] [WARN ] [REMOTE ] AMRZ: No data returned for month 2025-01 [2025-10-20T17:12:23.642854] [FATAL] [REMOTE ] AMRZ: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:12:23.642902] [FATAL] [MAIN ] AMRZ: Failed to fetch primary data. [2025-10-20T17:12:23.642940] [WARN ] [MAIN ] AMRZ: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:12:23.643000] [INFO ] [LOCAL ] AMRZ: Starting deletion of primary data. [2025-10-20T17:12:23.643080] [DEBUG] [LOCAL ] AMRZ: No primary data file found to delete. [2025-10-20T17:12:23.643137] [INFO ] [LOCAL ] AMRZ: Starting deletion of derivative data. [2025-10-20T17:12:23.643179] [DEBUG] [LOCAL ] AMRZ: No derivative data file found to delete. [2025-10-20T17:12:23.643224] [INFO ] [MAIN ] Update Progress: 106/2367 symbols completed. AMRZ took 3.0s. Estimated remaining: 4h 22m 48s. Rate: 8.6 symbols/min [2025-10-20T17:12:23.855177] [INFO ] [MAIN ] AMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:23.918407] [DEBUG] [LOCAL ] AMT: Loaded 53232 existing records from primary storage. [2025-10-20T17:12:23.918838] [INFO ] [MAIN ] AMT: Loaded 53232 existing records. [2025-10-20T17:12:24.116769] [WARN ] [MAIN ] AMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:24.117042] [INFO ] [MAIN ] AMT: Wiping existing primary data before full refresh. [2025-10-20T17:12:24.117117] [INFO ] [LOCAL ] AMT: Starting deletion of primary data. [2025-10-20T17:12:24.117491] [DEBUG] [LOCAL ] AMT: Deleted primary data file. [2025-10-20T17:12:24.117553] [INFO ] [MAIN ] AMT: Starting full primary refresh. [2025-10-20T17:12:24.278963] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-09. [2025-10-20T17:12:24.342353] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T17:12:24.532477] [INFO ] [REMOTE ] AMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:25.051427] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-08. [2025-10-20T17:12:25.382402] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T17:12:25.851392] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-01. [2025-10-20T17:12:26.191454] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T17:12:26.651429] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-01. [2025-10-20T17:12:27.451390] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-09. [2025-10-20T17:12:27.846584] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:12:28.037633] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:12:28.251592] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-02. [2025-10-20T17:12:28.280150] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T17:12:29.051391] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-10. [2025-10-20T17:12:29.851447] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-02. [2025-10-20T17:12:29.982829] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T17:12:30.315286] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T17:12:30.320072] [INFO ] [REMOTE ] AMR: Full fetch complete (40691 records). [2025-10-20T17:12:30.320185] [DEBUG] [VALIDATOR ] AMR: Validating primary data. [2025-10-20T17:12:30.659235] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-03. [2025-10-20T17:12:30.713701] [DEBUG] [VALIDATOR ] AMR: All expected trading days are present. [2025-10-20T17:12:30.713829] [DEBUG] [VALIDATOR ] AMR: Primary data validation passed. [2025-10-20T17:12:30.714380] [INFO ] [MAIN ] AMR: Fetched primary data is valid. [2025-10-20T17:12:30.714426] [DEBUG] [LOCAL ] AMR: Saving primary data. [2025-10-20T17:12:30.715589] [DEBUG] [LOCAL ] AMR: Trimmed 40691 records to 28978 (keeping 200 days of data) [2025-10-20T17:12:30.894396] [DEBUG] [LOCAL ] AMR: Primary data saved (28978 records). [2025-10-20T17:12:30.894630] [INFO ] [MAIN ] AMR: Full history saved. [2025-10-20T17:12:30.895332] [INFO ] [MAIN ] Update Progress: 107/2367 symbols completed. AMR took 25.7s. Estimated remaining: 4h 22m 47s. Rate: 8.6 symbols/min [2025-10-20T17:12:31.076589] [INFO ] [MAIN ] AMTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:31.101017] [DEBUG] [LOCAL ] AMTB: Loaded 20848 existing records from primary storage. [2025-10-20T17:12:31.101348] [INFO ] [MAIN ] AMTB: Loaded 20848 existing records. [2025-10-20T17:12:31.227452] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T17:12:31.317039] [WARN ] [MAIN ] AMTB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:31.317328] [INFO ] [MAIN ] AMTB: Wiping existing primary data before full refresh. [2025-10-20T17:12:31.317379] [INFO ] [LOCAL ] AMTB: Starting deletion of primary data. [2025-10-20T17:12:31.317564] [DEBUG] [LOCAL ] AMTB: Deleted primary data file. [2025-10-20T17:12:31.317603] [INFO ] [MAIN ] AMTB: Starting full primary refresh. [2025-10-20T17:12:31.456621] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-03. [2025-10-20T17:12:31.686396] [INFO ] [REMOTE ] AMTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:32.224058] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:12:32.251414] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-01. [2025-10-20T17:12:33.051436] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-04. [2025-10-20T17:12:33.280762] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T17:12:33.658291] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T17:12:33.851464] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-02. [2025-10-20T17:12:34.510339] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:12:34.548749] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T17:12:34.651439] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-04. [2025-10-20T17:12:35.451382] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-05. [2025-10-20T17:12:35.863435] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:12:36.251416] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-03. [2025-10-20T17:12:36.901862] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T17:12:37.051398] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-05. [2025-10-20T17:12:37.268058] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T17:12:37.851397] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-06. [2025-10-20T17:12:38.185463] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:12:38.651407] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-06. [2025-10-20T17:12:39.200318] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:12:39.451359] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-04. [2025-10-20T17:12:39.901848] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T17:12:40.251386] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-07. [2025-10-20T17:12:40.499524] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T17:12:41.051381] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-07. [2025-10-20T17:12:41.851390] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-05. [2025-10-20T17:12:42.107078] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T17:12:42.651403] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-08. [2025-10-20T17:12:43.451388] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-10. [2025-10-20T17:12:44.171615] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-05 (2.32 seconds). [2025-10-20T17:12:44.251395] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-06. [2025-10-20T17:12:44.347348] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-07 (3.30 seconds). [2025-10-20T17:12:44.949397] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-08 (2.30 seconds). [2025-10-20T17:12:44.971824] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T17:12:45.051452] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-09. [2025-10-20T17:12:45.273613] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T17:12:45.285978] [INFO ] [REMOTE ] AMPY: Full fetch complete (53157 records). [2025-10-20T17:12:45.286149] [DEBUG] [VALIDATOR ] AMPY: Validating primary data. [2025-10-20T17:12:45.697394] [DEBUG] [VALIDATOR ] AMPY: All expected trading days are present. [2025-10-20T17:12:45.697478] [DEBUG] [VALIDATOR ] AMPY: Primary data validation passed. [2025-10-20T17:12:45.698242] [INFO ] [MAIN ] AMPY: Fetched primary data is valid. [2025-10-20T17:12:45.698305] [DEBUG] [LOCAL ] AMPY: Saving primary data. [2025-10-20T17:12:45.699496] [DEBUG] [LOCAL ] AMPY: Trimmed 53157 records to 39461 (keeping 200 days of data) [2025-10-20T17:12:45.855418] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-07. [2025-10-20T17:12:45.926213] [DEBUG] [LOCAL ] AMPY: Primary data saved (39461 records). [2025-10-20T17:12:45.926354] [INFO ] [MAIN ] AMPY: Full history saved. [2025-10-20T17:12:45.926431] [INFO ] [MAIN ] Update Progress: 108/2367 symbols completed. AMPY took 42.9s. Estimated remaining: 4h 25m 28s. Rate: 8.5 symbols/min [2025-10-20T17:12:46.104370] [INFO ] [MAIN ] AMTD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:46.109588] [DEBUG] [LOCAL ] AMTD: Loaded 3828 existing records from primary storage. [2025-10-20T17:12:46.109791] [INFO ] [MAIN ] AMTD: Loaded 3828 existing records. [2025-10-20T17:12:46.317294] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T17:12:46.321354] [WARN ] [MAIN ] AMTD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:46.329488] [INFO ] [MAIN ] AMTD: Wiping existing primary data before full refresh. [2025-10-20T17:12:46.329661] [INFO ] [LOCAL ] AMTD: Starting deletion of primary data. [2025-10-20T17:12:46.330051] [DEBUG] [LOCAL ] AMTD: Deleted primary data file. [2025-10-20T17:12:46.330107] [INFO ] [MAIN ] AMTD: Starting full primary refresh. [2025-10-20T17:12:46.663754] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-10. [2025-10-20T17:12:46.710423] [INFO ] [REMOTE ] AMTD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:47.108156] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:12:47.451380] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-08. [2025-10-20T17:12:48.251390] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-08. [2025-10-20T17:12:48.355964] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:12:48.363696] [INFO ] [REMOTE ] AMRC: Full fetch complete (54461 records). [2025-10-20T17:12:48.363906] [DEBUG] [VALIDATOR ] AMRC: Validating primary data. [2025-10-20T17:12:48.738875] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T17:12:48.787126] [DEBUG] [VALIDATOR ] AMRC: All expected trading days are present. [2025-10-20T17:12:48.787218] [DEBUG] [VALIDATOR ] AMRC: Primary data validation passed. [2025-10-20T17:12:48.787807] [INFO ] [MAIN ] AMRC: Fetched primary data is valid. [2025-10-20T17:12:48.787871] [DEBUG] [LOCAL ] AMRC: Saving primary data. [2025-10-20T17:12:48.789194] [DEBUG] [LOCAL ] AMRC: Trimmed 54461 records to 38558 (keeping 200 days of data) [2025-10-20T17:12:49.023582] [DEBUG] [LOCAL ] AMRC: Primary data saved (38558 records). [2025-10-20T17:12:49.023781] [INFO ] [MAIN ] AMRC: Full history saved. [2025-10-20T17:12:49.023874] [INFO ] [MAIN ] Update Progress: 109/2367 symbols completed. AMRC took 28.8s. Estimated remaining: 4h 23m 60s. Rate: 8.6 symbols/min [2025-10-20T17:12:49.056501] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-01. [2025-10-20T17:12:49.202487] [INFO ] [MAIN ] AMTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:49.249431] [DEBUG] [LOCAL ] AMTM: Loaded 51234 existing records from primary storage. [2025-10-20T17:12:49.249777] [INFO ] [MAIN ] AMTM: Loaded 51234 existing records. [2025-10-20T17:12:49.431096] [WARN ] [MAIN ] AMTM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:49.431345] [INFO ] [MAIN ] AMTM: Wiping existing primary data before full refresh. [2025-10-20T17:12:49.431402] [INFO ] [LOCAL ] AMTM: Starting deletion of primary data. [2025-10-20T17:12:49.431791] [DEBUG] [LOCAL ] AMTM: Deleted primary data file. [2025-10-20T17:12:49.431848] [INFO ] [MAIN ] AMTM: Starting full primary refresh. [2025-10-20T17:12:49.802869] [INFO ] [REMOTE ] AMTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:49.824711] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T17:12:49.851368] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-09. [2025-10-20T17:12:50.030529] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T17:12:50.651367] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-09. [2025-10-20T17:12:50.801589] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T17:12:51.451375] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-02. [2025-10-20T17:12:51.856414] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T17:12:52.251368] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-10. [2025-10-20T17:12:52.699332] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T17:12:53.051390] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-10. [2025-10-20T17:12:53.305168] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:12:53.309109] [INFO ] [REMOTE ] AMTB: Full fetch complete (29719 records). [2025-10-20T17:12:53.309367] [DEBUG] [VALIDATOR ] AMTB: Validating primary data. [2025-10-20T17:12:53.735382] [DEBUG] [VALIDATOR ] AMTB: All expected trading days are present. [2025-10-20T17:12:53.735502] [DEBUG] [VALIDATOR ] AMTB: Primary data validation passed. [2025-10-20T17:12:53.736039] [INFO ] [MAIN ] AMTB: Fetched primary data is valid. [2025-10-20T17:12:53.736108] [DEBUG] [LOCAL ] AMTB: Saving primary data. [2025-10-20T17:12:53.737684] [DEBUG] [LOCAL ] AMTB: Trimmed 29719 records to 20539 (keeping 200 days of data) [2025-10-20T17:12:53.858167] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-03. [2025-10-20T17:12:53.868514] [DEBUG] [LOCAL ] AMTB: Primary data saved (20539 records). [2025-10-20T17:12:53.868627] [INFO ] [MAIN ] AMTB: Full history saved. [2025-10-20T17:12:53.868745] [INFO ] [MAIN ] Update Progress: 110/2367 symbols completed. AMTB took 23.0s. Estimated remaining: 4h 23m 8s. Rate: 8.6 symbols/min [2025-10-20T17:12:54.100876] [INFO ] [MAIN ] AMWL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:54.112345] [DEBUG] [LOCAL ] AMWL: Loaded 9827 existing records from primary storage. [2025-10-20T17:12:54.112589] [INFO ] [MAIN ] AMWL: Loaded 9827 existing records. [2025-10-20T17:12:54.299678] [WARN ] [MAIN ] AMWL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:54.299927] [INFO ] [MAIN ] AMWL: Wiping existing primary data before full refresh. [2025-10-20T17:12:54.299975] [INFO ] [LOCAL ] AMWL: Starting deletion of primary data. [2025-10-20T17:12:54.300307] [DEBUG] [LOCAL ] AMWL: Deleted primary data file. [2025-10-20T17:12:54.300343] [INFO ] [MAIN ] AMWL: Starting full primary refresh. [2025-10-20T17:12:54.403772] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T17:12:54.664108] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-01. [2025-10-20T17:12:54.678478] [INFO ] [REMOTE ] AMWL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:54.950522] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T17:12:54.958321] [INFO ] [REMOTE ] AMT: Full fetch complete (76173 records). [2025-10-20T17:12:54.958459] [DEBUG] [VALIDATOR ] AMT: Validating primary data. [2025-10-20T17:12:55.339336] [DEBUG] [VALIDATOR ] AMT: All expected trading days are present. [2025-10-20T17:12:55.339427] [DEBUG] [VALIDATOR ] AMT: Primary data validation passed. [2025-10-20T17:12:55.340189] [INFO ] [MAIN ] AMT: Fetched primary data is valid. [2025-10-20T17:12:55.340253] [DEBUG] [LOCAL ] AMT: Saving primary data. [2025-10-20T17:12:55.341772] [DEBUG] [LOCAL ] AMT: Trimmed 76173 records to 52454 (keeping 200 days of data) [2025-10-20T17:12:55.456703] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-04. [2025-10-20T17:12:55.668406] [DEBUG] [LOCAL ] AMT: Primary data saved (52454 records). [2025-10-20T17:12:55.668548] [INFO ] [MAIN ] AMT: Full history saved. [2025-10-20T17:12:55.669460] [INFO ] [MAIN ] Update Progress: 111/2367 symbols completed. AMT took 32.0s. Estimated remaining: 4h 21m 15s. Rate: 8.6 symbols/min [2025-10-20T17:12:55.839305] [INFO ] [MAIN ] AMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:55.883968] [DEBUG] [LOCAL ] AMX: Loaded 49908 existing records from primary storage. [2025-10-20T17:12:55.884293] [INFO ] [MAIN ] AMX: Loaded 49908 existing records. [2025-10-20T17:12:55.988697] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-10-20T17:12:56.066168] [WARN ] [MAIN ] AMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:56.066402] [INFO ] [MAIN ] AMX: Wiping existing primary data before full refresh. [2025-10-20T17:12:56.066449] [INFO ] [LOCAL ] AMX: Starting deletion of primary data. [2025-10-20T17:12:56.066756] [DEBUG] [LOCAL ] AMX: Deleted primary data file. [2025-10-20T17:12:56.066811] [INFO ] [MAIN ] AMX: Starting full primary refresh. [2025-10-20T17:12:56.269278] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-05. [2025-10-20T17:12:56.435736] [INFO ] [REMOTE ] AMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:56.464273] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:12:56.850511] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T17:12:57.051389] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-06. [2025-10-20T17:12:57.751509] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T17:12:57.851432] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-01. [2025-10-20T17:12:58.574378] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T17:12:58.651548] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-02. [2025-10-20T17:12:59.451413] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-02. [2025-10-20T17:12:59.477809] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T17:13:00.251386] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-07. [2025-10-20T17:13:00.966361] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T17:13:01.051489] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-08. [2025-10-20T17:13:01.373050] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-10-20T17:13:01.508610] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T17:13:01.851376] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-03. [2025-10-20T17:13:02.651341] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-03. [2025-10-20T17:13:03.047943] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T17:13:03.451426] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-04. [2025-10-20T17:13:04.251465] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-01. [2025-10-20T17:13:04.692595] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T17:13:05.051433] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-05. [2025-10-20T17:13:05.286726] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-03 (2.64 seconds). [2025-10-20T17:13:05.568935] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:13:05.845310] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T17:13:05.851335] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-06. [2025-10-20T17:13:06.581239] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T17:13:06.651380] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-07. [2025-10-20T17:13:07.451401] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-02. [2025-10-20T17:13:07.514354] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T17:13:08.251423] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-08. [2025-10-20T17:13:08.865865] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T17:13:08.992441] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:13:09.051365] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-09. [2025-10-20T17:13:09.852993] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-03. [2025-10-20T17:13:09.857224] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T17:13:10.651458] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-10. [2025-10-20T17:13:11.451366] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-04. [2025-10-20T17:13:11.528800] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T17:13:11.605023] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T17:13:11.608140] [INFO ] [REMOTE ] AMWL: Full fetch complete (14622 records). [2025-10-20T17:13:11.608218] [DEBUG] [VALIDATOR ] AMWL: Validating primary data. [2025-10-20T17:13:12.013317] [DEBUG] [VALIDATOR ] AMWL: All expected trading days are present. [2025-10-20T17:13:12.013393] [DEBUG] [VALIDATOR ] AMWL: Primary data validation passed. [2025-10-20T17:13:12.013663] [INFO ] [MAIN ] AMWL: Fetched primary data is valid. [2025-10-20T17:13:12.013724] [DEBUG] [LOCAL ] AMWL: Saving primary data. [2025-10-20T17:13:12.014184] [DEBUG] [LOCAL ] AMWL: Trimmed 14622 records to 9705 (keeping 200 days of data) [2025-10-20T17:13:12.066158] [DEBUG] [LOCAL ] AMWL: Primary data saved (9705 records). [2025-10-20T17:13:12.066305] [INFO ] [MAIN ] AMWL: Full history saved. [2025-10-20T17:13:12.066384] [INFO ] [MAIN ] Update Progress: 112/2367 symbols completed. AMWL took 18.2s. Estimated remaining: 4h 24m 19s. Rate: 8.5 symbols/min [2025-10-20T17:13:12.241543] [INFO ] [MAIN ] AN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:12.259592] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-04. [2025-10-20T17:13:12.275177] [DEBUG] [LOCAL ] AN: Loaded 31816 existing records from primary storage. [2025-10-20T17:13:12.275505] [INFO ] [MAIN ] AN: Loaded 31816 existing records. [2025-10-20T17:13:12.459008] [WARN ] [MAIN ] AN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:13:12.459294] [INFO ] [MAIN ] AN: Wiping existing primary data before full refresh. [2025-10-20T17:13:12.459348] [INFO ] [LOCAL ] AN: Starting deletion of primary data. [2025-10-20T17:13:12.459579] [DEBUG] [LOCAL ] AN: Deleted primary data file. [2025-10-20T17:13:12.459636] [INFO ] [MAIN ] AN: Starting full primary refresh. [2025-10-20T17:13:12.865535] [INFO ] [REMOTE ] AN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:13.051425] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-09. [2025-10-20T17:13:13.167835] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T17:13:13.782760] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-10-20T17:13:13.852149] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-01. [2025-10-20T17:13:13.852057] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:13:14.651435] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-05. [2025-10-20T17:13:14.905287] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T17:13:15.451425] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-02. [2025-10-20T17:13:16.251465] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-10. [2025-10-20T17:13:16.333150] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:13:16.995836] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:13:17.051420] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-06. [2025-10-20T17:13:17.062433] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T17:13:17.064901] [INFO ] [REMOTE ] AMTD: Full fetch complete (5467 records). [2025-10-20T17:13:17.065123] [DEBUG] [VALIDATOR ] AMTD: Validating primary data. [2025-10-20T17:13:17.512190] [DEBUG] [VALIDATOR ] AMTD: All expected trading days are present. [2025-10-20T17:13:17.512272] [DEBUG] [VALIDATOR ] AMTD: Primary data validation passed. [2025-10-20T17:13:17.512446] [INFO ] [MAIN ] AMTD: Fetched primary data is valid. [2025-10-20T17:13:17.512487] [DEBUG] [LOCAL ] AMTD: Saving primary data. [2025-10-20T17:13:17.513200] [DEBUG] [LOCAL ] AMTD: Trimmed 5467 records to 3824 (keeping 200 days of data) [2025-10-20T17:13:17.537633] [DEBUG] [LOCAL ] AMTD: Primary data saved (3824 records). [2025-10-20T17:13:17.537867] [INFO ] [MAIN ] AMTD: Full history saved. [2025-10-20T17:13:17.537976] [INFO ] [MAIN ] Update Progress: 113/2367 symbols completed. AMTD took 31.6s. Estimated remaining: 4h 23m 41s. Rate: 8.5 symbols/min [2025-10-20T17:13:17.747628] [INFO ] [MAIN ] ANET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:17.810379] [DEBUG] [LOCAL ] ANET: Loaded 54546 existing records from primary storage. [2025-10-20T17:13:17.810855] [INFO ] [MAIN ] ANET: Loaded 54546 existing records. [2025-10-20T17:13:17.856619] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-03. [2025-10-20T17:13:18.041120] [WARN ] [MAIN ] ANET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:13:18.041417] [INFO ] [MAIN ] ANET: Wiping existing primary data before full refresh. [2025-10-20T17:13:18.041471] [INFO ] [LOCAL ] ANET: Starting deletion of primary data. [2025-10-20T17:13:18.041896] [DEBUG] [LOCAL ] ANET: Deleted primary data file. [2025-10-20T17:13:18.041958] [INFO ] [MAIN ] ANET: Starting full primary refresh. [2025-10-20T17:13:18.469376] [INFO ] [REMOTE ] ANET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:18.651380] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-01. [2025-10-20T17:13:18.744890] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:13:19.430869] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T17:13:19.451347] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-04. [2025-10-20T17:13:20.251452] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-05. [2025-10-20T17:13:20.947318] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T17:13:21.058545] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-07. [2025-10-20T17:13:21.064501] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-01 (2.41 seconds). [2025-10-20T17:13:21.516952] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T17:13:21.851363] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-06. [2025-10-20T17:13:22.422602] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:13:22.651416] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-02. [2025-10-20T17:13:23.451454] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-08. [2025-10-20T17:13:23.702267] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T17:13:24.251458] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-05. [2025-10-20T17:13:25.041303] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T17:13:25.051566] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-07. [2025-10-20T17:13:25.093155] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-02 (2.44 seconds). [2025-10-20T17:13:25.421199] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T17:13:25.851420] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-06. [2025-10-20T17:13:26.651345] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-03. [2025-10-20T17:13:26.714132] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T17:13:26.774875] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T17:13:27.451626] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-08. [2025-10-20T17:13:28.251389] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-07. [2025-10-20T17:13:28.755476] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T17:13:29.051389] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-09. [2025-10-20T17:13:29.186744] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:13:29.692888] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:13:29.851371] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-08. [2025-10-20T17:13:30.651358] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-09. [2025-10-20T17:13:30.793261] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:13:30.968124] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:13:31.451382] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-09. [2025-10-20T17:13:32.128902] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T17:13:32.251379] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-10. [2025-10-20T17:13:32.685404] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T17:13:33.051414] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-10. [2025-10-20T17:13:33.851422] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-04. [2025-10-20T17:13:33.897306] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T17:13:33.905458] [INFO ] [REMOTE ] AMTM: Full fetch complete (73747 records). [2025-10-20T17:13:33.905585] [DEBUG] [VALIDATOR ] AMTM: Validating primary data. [2025-10-20T17:13:34.301939] [DEBUG] [VALIDATOR ] AMTM: All expected trading days are present. [2025-10-20T17:13:34.302051] [DEBUG] [VALIDATOR ] AMTM: Primary data validation passed. [2025-10-20T17:13:34.302963] [INFO ] [MAIN ] AMTM: Fetched primary data is valid. [2025-10-20T17:13:34.308288] [DEBUG] [LOCAL ] AMTM: Saving primary data. [2025-10-20T17:13:34.320756] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T17:13:34.326956] [INFO ] [REMOTE ] AN: Full fetch complete (47093 records). [2025-10-20T17:13:34.327129] [DEBUG] [VALIDATOR ] AN: Validating primary data. [2025-10-20T17:13:34.327230] [DEBUG] [LOCAL ] AMTM: Trimmed 73747 records to 50401 (keeping 200 days of data) [2025-10-20T17:13:34.653571] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-10. [2025-10-20T17:13:34.978560] [DEBUG] [VALIDATOR ] AN: All expected trading days are present. [2025-10-20T17:13:34.980556] [DEBUG] [VALIDATOR ] AN: Primary data validation passed. [2025-10-20T17:13:34.982273] [INFO ] [MAIN ] AN: Fetched primary data is valid. [2025-10-20T17:13:34.982347] [DEBUG] [LOCAL ] AN: Saving primary data. [2025-10-20T17:13:34.995589] [DEBUG] [LOCAL ] AN: Trimmed 47093 records to 31264 (keeping 200 days of data) [2025-10-20T17:13:35.147474] [DEBUG] [LOCAL ] AMTM: Primary data saved (50401 records). [2025-10-20T17:13:35.151966] [INFO ] [MAIN ] AMTM: Full history saved. [2025-10-20T17:13:35.152366] [INFO ] [MAIN ] Update Progress: 114/2367 symbols completed. AMTM took 46.1s. Estimated remaining: 4h 27m 3s. Rate: 8.4 symbols/min [2025-10-20T17:13:35.347060] [INFO ] [MAIN ] ANF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:35.444750] [DEBUG] [LOCAL ] AN: Primary data saved (31264 records). [2025-10-20T17:13:35.457951] [INFO ] [MAIN ] AN: Full history saved. [2025-10-20T17:13:35.458962] [INFO ] [MAIN ] Update Progress: 115/2367 symbols completed. AN took 23.4s. Estimated remaining: 4h 24m 43s. Rate: 8.5 symbols/min [2025-10-20T17:13:35.593362] [DEBUG] [LOCAL ] ANF: Loaded 52835 existing records from primary storage. [2025-10-20T17:13:35.609144] [INFO ] [MAIN ] ANF: Loaded 52835 existing records. [2025-10-20T17:13:35.767843] [INFO ] [MAIN ] ANGX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:35.768083] [DEBUG] [LOCAL ] ANGX: No existing data file found in primary storage. [2025-10-20T17:13:35.774000] [WARN ] [MAIN ] ANGX: No existing data found, performing full refresh instead. [2025-10-20T17:13:35.807606] [INFO ] [MAIN ] ANGX: Wiping existing primary data before full refresh. [2025-10-20T17:13:35.820783] [INFO ] [LOCAL ] ANGX: Starting deletion of primary data. [2025-10-20T17:13:35.829281] [DEBUG] [LOCAL ] ANGX: No primary data file found to delete. [2025-10-20T17:13:35.850437] [WARN ] [MAIN ] ANF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:13:35.850502] [INFO ] [MAIN ] ANGX: Starting full primary refresh. [2025-10-20T17:13:35.850732] [INFO ] [MAIN ] ANF: Wiping existing primary data before full refresh. [2025-10-20T17:13:35.850861] [INFO ] [LOCAL ] ANF: Starting deletion of primary data. [2025-10-20T17:13:35.926768] [DEBUG] [LOCAL ] ANF: Deleted primary data file. [2025-10-20T17:13:35.943704] [INFO ] [MAIN ] ANF: Starting full primary refresh. [2025-10-20T17:13:36.636871] [INFO ] [REMOTE ] ANGX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:36.661981] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-01. [2025-10-20T17:13:36.673799] [INFO ] [REMOTE ] ANF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:36.683415] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-01. [2025-10-20T17:13:36.697377] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T17:13:36.705452] [INFO ] [REMOTE ] AMX: Full fetch complete (71036 records). [2025-10-20T17:13:36.705556] [DEBUG] [VALIDATOR ] AMX: Validating primary data. [2025-10-20T17:13:36.962401] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:13:37.035538] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-04 (3.18 seconds). [2025-10-20T17:13:37.041945] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-01 [2025-10-20T17:13:37.055416] [FATAL] [REMOTE ] ANGX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:13:37.075809] [FATAL] [MAIN ] ANGX: Failed to fetch primary data. [2025-10-20T17:13:37.076379] [WARN ] [MAIN ] ANGX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:13:37.081966] [INFO ] [LOCAL ] ANGX: Starting deletion of primary data. [2025-10-20T17:13:37.087279] [DEBUG] [LOCAL ] ANGX: No primary data file found to delete. [2025-10-20T17:13:37.106682] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-05. [2025-10-20T17:13:37.143809] [INFO ] [LOCAL ] ANGX: Starting deletion of derivative data. [2025-10-20T17:13:37.165888] [DEBUG] [VALIDATOR ] AMX: All expected trading days are present. [2025-10-20T17:13:37.161832] [DEBUG] [LOCAL ] ANGX: No derivative data file found to delete. [2025-10-20T17:13:37.166206] [INFO ] [MAIN ] Update Progress: 116/2367 symbols completed. ANGX took 1.7s. Estimated remaining: 4h 22m 52s. Rate: 8.6 symbols/min [2025-10-20T17:13:37.166279] [DEBUG] [VALIDATOR ] AMX: Primary data validation passed. [2025-10-20T17:13:37.167241] [INFO ] [MAIN ] AMX: Fetched primary data is valid. [2025-10-20T17:13:37.167320] [DEBUG] [LOCAL ] AMX: Saving primary data. [2025-10-20T17:13:37.273796] [DEBUG] [LOCAL ] AMX: Trimmed 71036 records to 49207 (keeping 200 days of data) [2025-10-20T17:13:37.442531] [INFO ] [MAIN ] ANRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:37.467579] [DEBUG] [LOCAL ] ANRO: Loaded 19753 existing records from primary storage. [2025-10-20T17:13:37.468621] [INFO ] [MAIN ] ANRO: Loaded 19753 existing records. [2025-10-20T17:13:37.674902] [WARN ] [MAIN ] ANRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:13:37.677850] [INFO ] [MAIN ] ANRO: Wiping existing primary data before full refresh. [2025-10-20T17:13:37.677925] [INFO ] [LOCAL ] ANRO: Starting deletion of primary data. [2025-10-20T17:13:37.678119] [DEBUG] [LOCAL ] ANRO: Deleted primary data file. [2025-10-20T17:13:37.678160] [INFO ] [MAIN ] ANRO: Starting full primary refresh. [2025-10-20T17:13:38.212322] [DEBUG] [LOCAL ] AMX: Primary data saved (49207 records). [2025-10-20T17:13:38.227544] [INFO ] [MAIN ] AMX: Full history saved. [2025-10-20T17:13:38.245808] [INFO ] [MAIN ] Update Progress: 117/2367 symbols completed. AMX took 42.6s. Estimated remaining: 4h 20m 51s. Rate: 8.6 symbols/min [2025-10-20T17:13:38.251450] [INFO ] [REMOTE ] ANRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:38.251797] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-01. [2025-10-20T17:13:38.491189] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:13:38.651381] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-02. [2025-10-20T17:13:39.117063] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T17:13:39.307894] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-05 (2.20 seconds). [2025-10-20T17:13:39.451373] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-06. [2025-10-20T17:13:40.251514] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-02. [2025-10-20T17:13:40.447163] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T17:13:41.051481] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-03. [2025-10-20T17:13:41.352333] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:13:41.447706] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:13:41.851450] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-03. [2025-10-20T17:13:42.651377] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-07. [2025-10-20T17:13:42.946005] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T17:13:43.045781] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T17:13:43.451443] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-04. [2025-10-20T17:13:44.251414] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-04. [2025-10-20T17:13:44.681996] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T17:13:45.051401] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-08. [2025-10-20T17:13:45.288600] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T17:13:45.343717] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T17:13:45.851442] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-05. [2025-10-20T17:13:46.651400] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-05. [2025-10-20T17:13:46.724133] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T17:13:47.451399] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-09. [2025-10-20T17:13:47.706995] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:13:47.747002] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T17:13:48.251418] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-06. [2025-10-20T17:13:49.051414] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-06. [2025-10-20T17:13:49.524226] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T17:13:49.851365] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-10. [2025-10-20T17:13:50.048066] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T17:13:50.279285] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T17:13:50.519361] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:13:50.527400] [INFO ] [REMOTE ] ANET: Full fetch complete (77945 records). [2025-10-20T17:13:50.527575] [DEBUG] [VALIDATOR ] ANET: Validating primary data. [2025-10-20T17:13:50.656527] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-07. [2025-10-20T17:13:50.951719] [DEBUG] [VALIDATOR ] ANET: All expected trading days are present. [2025-10-20T17:13:50.951873] [DEBUG] [VALIDATOR ] ANET: Primary data validation passed. [2025-10-20T17:13:50.953042] [INFO ] [MAIN ] ANET: Fetched primary data is valid. [2025-10-20T17:13:50.953119] [DEBUG] [LOCAL ] ANET: Saving primary data. [2025-10-20T17:13:50.955412] [DEBUG] [LOCAL ] ANET: Trimmed 77945 records to 53766 (keeping 200 days of data) [2025-10-20T17:13:51.291982] [DEBUG] [LOCAL ] ANET: Primary data saved (53766 records). [2025-10-20T17:13:51.292174] [INFO ] [MAIN ] ANET: Full history saved. [2025-10-20T17:13:51.292274] [INFO ] [MAIN ] Update Progress: 118/2367 symbols completed. ANET took 33.8s. Estimated remaining: 4h 22m 40s. Rate: 8.6 symbols/min [2025-10-20T17:13:51.451461] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-07. [2025-10-20T17:13:51.730521] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T17:13:52.251432] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-08. [2025-10-20T17:13:53.160162] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T17:13:53.164366] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-09. [2025-10-20T17:13:53.291882] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T17:13:53.662303] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T17:13:53.851398] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-08. [2025-10-20T17:13:54.651412] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-10. [2025-10-20T17:13:55.839770] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T17:13:55.851001] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-09. [2025-10-20T17:13:55.931410] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T17:13:55.936304] [INFO ] [REMOTE ] ANRO: Full fetch complete (30045 records). [2025-10-20T17:13:55.936399] [DEBUG] [VALIDATOR ] ANRO: Validating primary data. [2025-10-20T17:13:56.353817] [DEBUG] [VALIDATOR ] ANRO: All expected trading days are present. [2025-10-20T17:13:56.353923] [DEBUG] [VALIDATOR ] ANRO: Primary data validation passed. [2025-10-20T17:13:56.354397] [INFO ] [MAIN ] ANRO: Fetched primary data is valid. [2025-10-20T17:13:56.354442] [DEBUG] [LOCAL ] ANRO: Saving primary data. [2025-10-20T17:13:56.355346] [DEBUG] [LOCAL ] ANRO: Trimmed 30045 records to 19540 (keeping 200 days of data) [2025-10-20T17:13:56.469986] [DEBUG] [LOCAL ] ANRO: Primary data saved (19540 records). [2025-10-20T17:13:56.470189] [INFO ] [MAIN ] ANRO: Full history saved. [2025-10-20T17:13:56.470324] [INFO ] [MAIN ] Update Progress: 119/2367 symbols completed. ANRO took 19.3s. Estimated remaining: 4h 21m 58s. Rate: 8.6 symbols/min [2025-10-20T17:13:57.598673] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T17:13:57.611425] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-10. [2025-10-20T17:13:59.074492] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T17:13:59.082070] [INFO ] [REMOTE ] ANF: Full fetch complete (75784 records). [2025-10-20T17:13:59.082197] [DEBUG] [VALIDATOR ] ANF: Validating primary data. [2025-10-20T17:13:59.509764] [DEBUG] [VALIDATOR ] ANF: All expected trading days are present. [2025-10-20T17:13:59.509854] [DEBUG] [VALIDATOR ] ANF: Primary data validation passed. [2025-10-20T17:13:59.510574] [INFO ] [MAIN ] ANF: Fetched primary data is valid. [2025-10-20T17:13:59.510613] [DEBUG] [LOCAL ] ANF: Saving primary data. [2025-10-20T17:13:59.511986] [DEBUG] [LOCAL ] ANF: Trimmed 75784 records to 52054 (keeping 200 days of data) [2025-10-20T17:13:59.811570] [DEBUG] [LOCAL ] ANF: Primary data saved (52054 records). [2025-10-20T17:13:59.811743] [INFO ] [MAIN ] ANF: Full history saved. [2025-10-20T17:13:59.811828] [INFO ] [MAIN ] Update Progress: 120/2367 symbols completed. ANF took 24.7s. Estimated remaining: 4h 20m 43s. Rate: 8.6 symbols/min [2025-10-20T17:14:00.556193] [INFO ] [MAIN ] AOMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:00.553082] [INFO ] [MAIN ] ANVS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:00.575211] [INFO ] [MAIN ] AOMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:00.598301] [DEBUG] [LOCAL ] AOMD: No existing data file found in primary storage. [2025-10-20T17:14:00.598958] [WARN ] [MAIN ] AOMD: No existing data found, performing full refresh instead. [2025-10-20T17:14:00.603622] [INFO ] [MAIN ] AOMD: Wiping existing primary data before full refresh. [2025-10-20T17:14:00.603545] [INFO ] [MAIN ] AOD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:00.603786] [INFO ] [LOCAL ] AOMD: Starting deletion of primary data. [2025-10-20T17:14:00.604256] [DEBUG] [LOCAL ] AOMD: No primary data file found to delete. [2025-10-20T17:14:00.604788] [INFO ] [MAIN ] AOMD: Starting full primary refresh. [2025-10-20T17:14:00.610148] [DEBUG] [LOCAL ] AOD: No existing data file found in primary storage. [2025-10-20T17:14:00.635986] [WARN ] [MAIN ] AOD: No existing data found, performing full refresh instead. [2025-10-20T17:14:00.682161] [INFO ] [MAIN ] AOD: Wiping existing primary data before full refresh. [2025-10-20T17:14:00.694191] [INFO ] [LOCAL ] AOD: Starting deletion of primary data. [2025-10-20T17:14:00.699467] [DEBUG] [LOCAL ] AOD: No primary data file found to delete. [2025-10-20T17:14:00.699751] [INFO ] [MAIN ] AOD: Starting full primary refresh. [2025-10-20T17:14:00.768941] [DEBUG] [LOCAL ] AOMR: Loaded 16681 existing records from primary storage. [2025-10-20T17:14:00.825796] [INFO ] [MAIN ] AOMR: Loaded 16681 existing records. [2025-10-20T17:14:00.842073] [DEBUG] [LOCAL ] ANVS: Loaded 24303 existing records from primary storage. [2025-10-20T17:14:00.902253] [INFO ] [MAIN ] ANVS: Loaded 24303 existing records. [2025-10-20T17:14:01.609853] [WARN ] [MAIN ] ANVS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:01.660492] [INFO ] [REMOTE ] AOMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:01.691893] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-01. [2025-10-20T17:14:01.692004] [INFO ] [MAIN ] ANVS: Wiping existing primary data before full refresh. [2025-10-20T17:14:01.753665] [WARN ] [MAIN ] AOMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:01.755063] [INFO ] [LOCAL ] ANVS: Starting deletion of primary data. [2025-10-20T17:14:01.755364] [INFO ] [MAIN ] AOMR: Wiping existing primary data before full refresh. [2025-10-20T17:14:01.755438] [DEBUG] [LOCAL ] ANVS: Deleted primary data file. [2025-10-20T17:14:01.755547] [INFO ] [LOCAL ] AOMR: Starting deletion of primary data. [2025-10-20T17:14:01.768710] [INFO ] [MAIN ] ANVS: Starting full primary refresh. [2025-10-20T17:14:01.811137] [DEBUG] [LOCAL ] AOMR: Deleted primary data file. [2025-10-20T17:14:01.817438] [INFO ] [MAIN ] AOMR: Starting full primary refresh. [2025-10-20T17:14:02.053983] [INFO ] [REMOTE ] AOD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:02.098721] [DEBUG] [REMOTE ] AOMD: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:14:02.098276] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-01. [2025-10-20T17:14:02.120216] [WARN ] [REMOTE ] AOMD: No data returned for month 2025-01 [2025-10-20T17:14:02.143922] [FATAL] [REMOTE ] AOMD: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:14:02.149674] [FATAL] [MAIN ] AOMD: Failed to fetch primary data. [2025-10-20T17:14:02.161894] [WARN ] [MAIN ] AOMD: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:14:02.222285] [INFO ] [LOCAL ] AOMD: Starting deletion of primary data. [2025-10-20T17:14:02.307615] [DEBUG] [LOCAL ] AOMD: No primary data file found to delete. [2025-10-20T17:14:02.330928] [INFO ] [LOCAL ] AOMD: Starting deletion of derivative data. [2025-10-20T17:14:02.398548] [DEBUG] [LOCAL ] AOMD: No derivative data file found to delete. [2025-10-20T17:14:02.437677] [INFO ] [MAIN ] Update Progress: 121/2367 symbols completed. AOMD took 2.6s. Estimated remaining: 4h 19m 16s. Rate: 8.7 symbols/min [2025-10-20T17:14:02.813902] [INFO ] [REMOTE ] ANVS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:02.868078] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-01. [2025-10-20T17:14:02.990503] [INFO ] [REMOTE ] AOMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:02.985986] [INFO ] [MAIN ] AON: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:02.990919] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-01. [2025-10-20T17:14:03.037100] [DEBUG] [LOCAL ] AON: Loaded 45813 existing records from primary storage. [2025-10-20T17:14:03.037538] [INFO ] [MAIN ] AON: Loaded 45813 existing records. [2025-10-20T17:14:03.225543] [WARN ] [MAIN ] AON: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:03.225863] [INFO ] [MAIN ] AON: Wiping existing primary data before full refresh. [2025-10-20T17:14:03.225920] [INFO ] [LOCAL ] AON: Starting deletion of primary data. [2025-10-20T17:14:03.226181] [DEBUG] [LOCAL ] AON: Deleted primary data file. [2025-10-20T17:14:03.226275] [INFO ] [MAIN ] AON: Starting full primary refresh. [2025-10-20T17:14:03.597880] [INFO ] [REMOTE ] AON: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:03.598264] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-01. [2025-10-20T17:14:03.686084] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T17:14:03.693798] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-02. [2025-10-20T17:14:03.926620] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T17:14:03.929395] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-02. [2025-10-20T17:14:04.315370] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:14:04.324543] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-02. [2025-10-20T17:14:04.829599] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T17:14:04.836455] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-03. [2025-10-20T17:14:04.876034] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T17:14:04.883000] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-03. [2025-10-20T17:14:05.043205] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:14:05.052596] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-02. [2025-10-20T17:14:05.323624] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T17:14:05.858387] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-03. [2025-10-20T17:14:05.865166] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T17:14:05.952276] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T17:14:06.590098] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:14:06.651358] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-04. [2025-10-20T17:14:06.715606] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T17:14:07.451432] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-04. [2025-10-20T17:14:07.708507] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T17:14:08.251398] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-05. [2025-10-20T17:14:08.590782] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-10-20T17:14:09.051363] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-05. [2025-10-20T17:14:09.288032] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T17:14:09.851389] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-04. [2025-10-20T17:14:10.266157] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T17:14:10.651415] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-03. [2025-10-20T17:14:11.051226] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T17:14:11.451438] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-06. [2025-10-20T17:14:12.180035] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T17:14:12.251390] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-06. [2025-10-20T17:14:12.635871] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T17:14:12.894277] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:14:13.051427] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-07. [2025-10-20T17:14:13.851446] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-04. [2025-10-20T17:14:14.316406] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T17:14:14.651432] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-05. [2025-10-20T17:14:15.451399] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-07. [2025-10-20T17:14:15.703710] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T17:14:15.832114] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T17:14:16.251361] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-05. [2025-10-20T17:14:16.402125] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:14:17.051414] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-08. [2025-10-20T17:14:17.851407] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-08. [2025-10-20T17:14:17.911541] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T17:14:18.427529] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T17:14:18.651367] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-09. [2025-10-20T17:14:19.068847] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-10-20T17:14:19.451430] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-09. [2025-10-20T17:14:20.197673] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T17:14:20.251399] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-10. [2025-10-20T17:14:20.486577] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:14:21.051423] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-06. [2025-10-20T17:14:21.733149] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T17:14:21.738286] [INFO ] [REMOTE ] AOD: Full fetch complete (44782 records). [2025-10-20T17:14:21.738366] [DEBUG] [VALIDATOR ] AOD: Validating primary data. [2025-10-20T17:14:21.856544] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-06. [2025-10-20T17:14:22.135290] [DEBUG] [VALIDATOR ] AOD: All expected trading days are present. [2025-10-20T17:14:22.135546] [DEBUG] [VALIDATOR ] AOD: Primary data validation passed. [2025-10-20T17:14:22.136240] [INFO ] [MAIN ] AOD: Fetched primary data is valid. [2025-10-20T17:14:22.136362] [DEBUG] [LOCAL ] AOD: Saving primary data. [2025-10-20T17:14:22.157069] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T17:14:22.164478] [DEBUG] [LOCAL ] AOD: Trimmed 44782 records to 30963 (keeping 200 days of data) [2025-10-20T17:14:22.349582] [DEBUG] [LOCAL ] AOD: Primary data saved (30963 records). [2025-10-20T17:14:22.349752] [INFO ] [MAIN ] AOD: Full history saved. [2025-10-20T17:14:22.349831] [INFO ] [MAIN ] Update Progress: 122/2367 symbols completed. AOD took 22.5s. Estimated remaining: 4h 23m 8s. Rate: 8.5 symbols/min [2025-10-20T17:14:22.520703] [INFO ] [MAIN ] AORT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:22.549753] [DEBUG] [LOCAL ] AORT: Loaded 29531 existing records from primary storage. [2025-10-20T17:14:22.550048] [INFO ] [MAIN ] AORT: Loaded 29531 existing records. [2025-10-20T17:14:22.656474] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-07. [2025-10-20T17:14:22.729860] [WARN ] [MAIN ] AORT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:22.730131] [INFO ] [MAIN ] AORT: Wiping existing primary data before full refresh. [2025-10-20T17:14:22.730213] [INFO ] [LOCAL ] AORT: Starting deletion of primary data. [2025-10-20T17:14:22.730390] [DEBUG] [LOCAL ] AORT: Deleted primary data file. [2025-10-20T17:14:22.730429] [INFO ] [MAIN ] AORT: Starting full primary refresh. [2025-10-20T17:14:23.083727] [INFO ] [REMOTE ] AORT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:23.451365] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-01. [2025-10-20T17:14:23.530474] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:14:23.636311] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:14:24.251416] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-10. [2025-10-20T17:14:24.775599] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:14:25.051450] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-02. [2025-10-20T17:14:25.163636] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T17:14:25.166118] [INFO ] [REMOTE ] AOMR: Full fetch complete (23200 records). [2025-10-20T17:14:25.166230] [DEBUG] [VALIDATOR ] AOMR: Validating primary data. [2025-10-20T17:14:25.563799] [DEBUG] [VALIDATOR ] AOMR: All expected trading days are present. [2025-10-20T17:14:25.563903] [DEBUG] [VALIDATOR ] AOMR: Primary data validation passed. [2025-10-20T17:14:25.564311] [INFO ] [MAIN ] AOMR: Fetched primary data is valid. [2025-10-20T17:14:25.564380] [DEBUG] [LOCAL ] AOMR: Saving primary data. [2025-10-20T17:14:25.565363] [DEBUG] [LOCAL ] AOMR: Trimmed 23200 records to 16433 (keeping 200 days of data) [2025-10-20T17:14:25.666523] [DEBUG] [LOCAL ] AOMR: Primary data saved (16433 records). [2025-10-20T17:14:25.666701] [INFO ] [MAIN ] AOMR: Full history saved. [2025-10-20T17:14:25.666794] [INFO ] [MAIN ] Update Progress: 123/2367 symbols completed. AOMR took 25.8s. Estimated remaining: 4h 21m 53s. Rate: 8.6 symbols/min [2025-10-20T17:14:25.833374] [INFO ] [MAIN ] AOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:25.856546] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-07. [2025-10-20T17:14:25.878035] [DEBUG] [LOCAL ] AOS: Loaded 49475 existing records from primary storage. [2025-10-20T17:14:25.878336] [INFO ] [MAIN ] AOS: Loaded 49475 existing records. [2025-10-20T17:14:26.071546] [WARN ] [MAIN ] AOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:26.071829] [INFO ] [MAIN ] AOS: Wiping existing primary data before full refresh. [2025-10-20T17:14:26.071907] [INFO ] [LOCAL ] AOS: Starting deletion of primary data. [2025-10-20T17:14:26.072146] [DEBUG] [LOCAL ] AOS: Deleted primary data file. [2025-10-20T17:14:26.072219] [INFO ] [MAIN ] AOS: Starting full primary refresh. [2025-10-20T17:14:26.464979] [INFO ] [REMOTE ] AOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:26.485034] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T17:14:26.651429] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-08. [2025-10-20T17:14:27.451348] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-01. [2025-10-20T17:14:27.634915] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T17:14:27.668031] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T17:14:28.251444] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-03. [2025-10-20T17:14:29.051362] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-08. [2025-10-20T17:14:29.175396] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T17:14:29.589424] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T17:14:29.851366] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-02. [2025-10-20T17:14:30.651415] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-09. [2025-10-20T17:14:30.766617] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T17:14:31.361831] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T17:14:31.418332] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T17:14:31.451417] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-03. [2025-10-20T17:14:32.251395] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-09. [2025-10-20T17:14:33.051406] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-10. [2025-10-20T17:14:33.158873] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:14:33.851378] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-04. [2025-10-20T17:14:33.887634] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T17:14:34.475285] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T17:14:34.480138] [INFO ] [REMOTE ] ANVS: Full fetch complete (36901 records). [2025-10-20T17:14:34.480215] [DEBUG] [VALIDATOR ] ANVS: Validating primary data. [2025-10-20T17:14:34.658564] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-04. [2025-10-20T17:14:34.860158] [DEBUG] [VALIDATOR ] ANVS: All expected trading days are present. [2025-10-20T17:14:34.860325] [DEBUG] [VALIDATOR ] ANVS: Primary data validation passed. [2025-10-20T17:14:34.861035] [INFO ] [MAIN ] ANVS: Fetched primary data is valid. [2025-10-20T17:14:34.861106] [DEBUG] [LOCAL ] ANVS: Saving primary data. [2025-10-20T17:14:34.862576] [DEBUG] [LOCAL ] ANVS: Trimmed 36901 records to 24001 (keeping 200 days of data) [2025-10-20T17:14:35.003863] [DEBUG] [LOCAL ] ANVS: Primary data saved (24001 records). [2025-10-20T17:14:35.004067] [INFO ] [MAIN ] ANVS: Full history saved. [2025-10-20T17:14:35.004190] [INFO ] [MAIN ] Update Progress: 124/2367 symbols completed. ANVS took 35.2s. Estimated remaining: 4h 22m 28s. Rate: 8.5 symbols/min [2025-10-20T17:14:35.252353] [INFO ] [MAIN ] AP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:35.252705] [DEBUG] [LOCAL ] AP: No existing data file found in primary storage. [2025-10-20T17:14:35.252764] [WARN ] [MAIN ] AP: No existing data found, performing full refresh instead. [2025-10-20T17:14:35.252886] [INFO ] [MAIN ] AP: Wiping existing primary data before full refresh. [2025-10-20T17:14:35.252929] [INFO ] [LOCAL ] AP: Starting deletion of primary data. [2025-10-20T17:14:35.252971] [DEBUG] [LOCAL ] AP: No primary data file found to delete. [2025-10-20T17:14:35.253005] [INFO ] [MAIN ] AP: Starting full primary refresh. [2025-10-20T17:14:35.457531] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-10. [2025-10-20T17:14:35.667894] [INFO ] [REMOTE ] AP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:35.698413] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:14:36.193904] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T17:14:36.251418] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-05. [2025-10-20T17:14:37.051346] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-01. [2025-10-20T17:14:37.143965] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:14:37.150581] [INFO ] [REMOTE ] AON: Full fetch complete (65161 records). [2025-10-20T17:14:37.150736] [DEBUG] [VALIDATOR ] AON: Validating primary data. [2025-10-20T17:14:37.560940] [DEBUG] [VALIDATOR ] AON: All expected trading days are present. [2025-10-20T17:14:37.561093] [DEBUG] [VALIDATOR ] AON: Primary data validation passed. [2025-10-20T17:14:37.561848] [INFO ] [MAIN ] AON: Fetched primary data is valid. [2025-10-20T17:14:37.561928] [DEBUG] [LOCAL ] AON: Saving primary data. [2025-10-20T17:14:37.566967] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T17:14:37.568931] [DEBUG] [LOCAL ] AON: Trimmed 65161 records to 45175 (keeping 200 days of data) [2025-10-20T17:14:37.683111] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T17:14:37.852222] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-02. [2025-10-20T17:14:37.869059] [DEBUG] [LOCAL ] AON: Primary data saved (45175 records). [2025-10-20T17:14:37.869157] [INFO ] [MAIN ] AON: Full history saved. [2025-10-20T17:14:37.869230] [INFO ] [MAIN ] Update Progress: 125/2367 symbols completed. AON took 35.4s. Estimated remaining: 4h 21m 6s. Rate: 8.6 symbols/min [2025-10-20T17:14:38.047976] [INFO ] [MAIN ] APAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:38.083257] [DEBUG] [LOCAL ] APAM: Loaded 36578 existing records from primary storage. [2025-10-20T17:14:38.083605] [INFO ] [MAIN ] APAM: Loaded 36578 existing records. [2025-10-20T17:14:38.273604] [WARN ] [MAIN ] APAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:38.273849] [INFO ] [MAIN ] APAM: Wiping existing primary data before full refresh. [2025-10-20T17:14:38.273897] [INFO ] [LOCAL ] APAM: Starting deletion of primary data. [2025-10-20T17:14:38.274134] [DEBUG] [LOCAL ] APAM: Deleted primary data file. [2025-10-20T17:14:38.274173] [INFO ] [MAIN ] APAM: Starting full primary refresh. [2025-10-20T17:14:38.522304] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T17:14:38.647739] [INFO ] [REMOTE ] APAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:38.651339] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-01. [2025-10-20T17:14:39.451371] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-03. [2025-10-20T17:14:39.777880] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T17:14:40.031671] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T17:14:40.251368] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-06. [2025-10-20T17:14:41.051353] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-02. [2025-10-20T17:14:41.444474] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:14:41.851372] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-05. [2025-10-20T17:14:42.446123] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T17:14:42.651377] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-03. [2025-10-20T17:14:43.451390] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-04. [2025-10-20T17:14:43.530253] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:14:44.088329] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T17:14:44.114464] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T17:14:44.251358] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-04. [2025-10-20T17:14:45.051385] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-07. [2025-10-20T17:14:45.734698] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:14:45.851369] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-05. [2025-10-20T17:14:46.386293] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T17:14:46.651384] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-06. [2025-10-20T17:14:47.304095] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T17:14:47.451365] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-05. [2025-10-20T17:14:48.040902] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T17:14:48.251343] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-06. [2025-10-20T17:14:48.306946] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:14:48.990217] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T17:14:49.051338] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-07. [2025-10-20T17:14:49.851426] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-08. [2025-10-20T17:14:49.879859] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T17:14:50.651386] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-07. [2025-10-20T17:14:51.226463] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T17:14:51.451406] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-06. [2025-10-20T17:14:52.251359] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-08. [2025-10-20T17:14:52.284809] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T17:14:52.735324] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T17:14:52.919273] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T17:14:53.051344] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-08. [2025-10-20T17:14:53.851389] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-07. [2025-10-20T17:14:54.492589] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T17:14:54.651432] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-09. [2025-10-20T17:14:55.274012] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:14:55.398435] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T17:14:55.451334] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-09. [2025-10-20T17:14:56.251380] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-08. [2025-10-20T17:14:57.056456] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-10. [2025-10-20T17:14:57.068042] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T17:14:57.161164] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T17:14:57.855186] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T17:14:57.854352] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-09. [2025-10-20T17:14:57.857406] [INFO ] [REMOTE ] AP: Full fetch complete (9046 records). [2025-10-20T17:14:57.857581] [DEBUG] [VALIDATOR ] AP: Validating primary data. [2025-10-20T17:14:58.212470] [DEBUG] [VALIDATOR ] AP: All expected trading days are present. [2025-10-20T17:14:58.212546] [DEBUG] [VALIDATOR ] AP: Primary data validation passed. [2025-10-20T17:14:58.212772] [INFO ] [MAIN ] AP: Fetched primary data is valid. [2025-10-20T17:14:58.212879] [DEBUG] [LOCAL ] AP: Saving primary data. [2025-10-20T17:14:58.213386] [DEBUG] [LOCAL ] AP: Trimmed 9046 records to 7408 (keeping 200 days of data) [2025-10-20T17:14:58.254599] [DEBUG] [LOCAL ] AP: Primary data saved (7408 records). [2025-10-20T17:14:58.254727] [INFO ] [MAIN ] AP: Full history saved. [2025-10-20T17:14:58.254800] [INFO ] [MAIN ] Update Progress: 126/2367 symbols completed. AP took 23.3s. Estimated remaining: 4h 24m 58s. Rate: 8.5 symbols/min [2025-10-20T17:14:58.428914] [INFO ] [MAIN ] APD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:58.477472] [DEBUG] [LOCAL ] APD: Loaded 46187 existing records from primary storage. [2025-10-20T17:14:58.477903] [INFO ] [MAIN ] APD: Loaded 46187 existing records. [2025-10-20T17:14:58.651790] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-09. [2025-10-20T17:14:58.680374] [WARN ] [MAIN ] APD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:58.680887] [INFO ] [MAIN ] APD: Wiping existing primary data before full refresh. [2025-10-20T17:14:58.680950] [INFO ] [LOCAL ] APD: Starting deletion of primary data. [2025-10-20T17:14:58.681220] [DEBUG] [LOCAL ] APD: Deleted primary data file. [2025-10-20T17:14:58.681277] [INFO ] [MAIN ] APD: Starting full primary refresh. [2025-10-20T17:14:59.068773] [INFO ] [REMOTE ] APD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:59.307563] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T17:14:59.451360] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-10. [2025-10-20T17:15:00.079911] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:15:00.251364] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-10. [2025-10-20T17:15:00.689184] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T17:15:00.694805] [INFO ] [REMOTE ] APAM: Full fetch complete (52123 records). [2025-10-20T17:15:00.694942] [DEBUG] [VALIDATOR ] APAM: Validating primary data. [2025-10-20T17:15:01.072444] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-10. [2025-10-20T17:15:01.090924] [DEBUG] [VALIDATOR ] APAM: All expected trading days are present. [2025-10-20T17:15:01.091003] [DEBUG] [VALIDATOR ] APAM: Primary data validation passed. [2025-10-20T17:15:01.091578] [INFO ] [MAIN ] APAM: Fetched primary data is valid. [2025-10-20T17:15:01.091620] [DEBUG] [LOCAL ] APAM: Saving primary data. [2025-10-20T17:15:01.093189] [DEBUG] [LOCAL ] APAM: Trimmed 52123 records to 36007 (keeping 200 days of data) [2025-10-20T17:15:01.315925] [DEBUG] [LOCAL ] APAM: Primary data saved (36007 records). [2025-10-20T17:15:01.316145] [INFO ] [MAIN ] APAM: Full history saved. [2025-10-20T17:15:01.316737] [INFO ] [MAIN ] Update Progress: 127/2367 symbols completed. APAM took 23.4s. Estimated remaining: 4h 23m 40s. Rate: 8.5 symbols/min [2025-10-20T17:15:01.508263] [INFO ] [MAIN ] APG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:01.532373] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T17:15:01.537006] [INFO ] [REMOTE ] AORT: Full fetch complete (42377 records). [2025-10-20T17:15:01.537168] [DEBUG] [VALIDATOR ] AORT: Validating primary data. [2025-10-20T17:15:01.857815] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-01. [2025-10-20T17:15:01.920989] [DEBUG] [LOCAL ] APG: Loaded 51343 existing records from primary storage. [2025-10-20T17:15:01.930415] [INFO ] [MAIN ] APG: Loaded 51343 existing records. [2025-10-20T17:15:02.167081] [DEBUG] [VALIDATOR ] AORT: All expected trading days are present. [2025-10-20T17:15:02.167227] [DEBUG] [VALIDATOR ] AORT: Primary data validation passed. [2025-10-20T17:15:02.167870] [INFO ] [MAIN ] AORT: Fetched primary data is valid. [2025-10-20T17:15:02.167957] [DEBUG] [LOCAL ] AORT: Saving primary data. [2025-10-20T17:15:02.219821] [DEBUG] [LOCAL ] AORT: Trimmed 42377 records to 28981 (keeping 200 days of data) [2025-10-20T17:15:02.252901] [WARN ] [MAIN ] APG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:02.298955] [INFO ] [MAIN ] APG: Wiping existing primary data before full refresh. [2025-10-20T17:15:02.299030] [INFO ] [LOCAL ] APG: Starting deletion of primary data. [2025-10-20T17:15:02.334502] [DEBUG] [LOCAL ] APG: Deleted primary data file. [2025-10-20T17:15:02.335044] [INFO ] [MAIN ] APG: Starting full primary refresh. [2025-10-20T17:15:02.814437] [INFO ] [REMOTE ] APG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:02.817903] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-01. [2025-10-20T17:15:02.825441] [DEBUG] [LOCAL ] AORT: Primary data saved (28981 records). [2025-10-20T17:15:02.825538] [INFO ] [MAIN ] AORT: Full history saved. [2025-10-20T17:15:02.825669] [INFO ] [MAIN ] Update Progress: 128/2367 symbols completed. AORT took 40.5s. Estimated remaining: 4h 21m 55s. Rate: 8.5 symbols/min [2025-10-20T17:15:03.007848] [INFO ] [MAIN ] APH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:03.061323] [DEBUG] [LOCAL ] APH: Loaded 54471 existing records from primary storage. [2025-10-20T17:15:03.061755] [INFO ] [MAIN ] APH: Loaded 54471 existing records. [2025-10-20T17:15:03.269053] [WARN ] [MAIN ] APH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:03.280870] [INFO ] [MAIN ] APH: Wiping existing primary data before full refresh. [2025-10-20T17:15:03.281666] [INFO ] [LOCAL ] APH: Starting deletion of primary data. [2025-10-20T17:15:03.282621] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T17:15:03.289629] [DEBUG] [LOCAL ] APH: Deleted primary data file. [2025-10-20T17:15:03.289876] [INFO ] [MAIN ] APH: Starting full primary refresh. [2025-10-20T17:15:03.337522] [INFO ] [REMOTE ] AOS: Full fetch complete (71026 records). [2025-10-20T17:15:03.366825] [DEBUG] [VALIDATOR ] AOS: Validating primary data. [2025-10-20T17:15:04.085826] [INFO ] [REMOTE ] APH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:04.096807] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-01. [2025-10-20T17:15:04.186967] [DEBUG] [VALIDATOR ] AOS: All expected trading days are present. [2025-10-20T17:15:04.187141] [DEBUG] [VALIDATOR ] AOS: Primary data validation passed. [2025-10-20T17:15:04.188760] [INFO ] [MAIN ] AOS: Fetched primary data is valid. [2025-10-20T17:15:04.188912] [DEBUG] [LOCAL ] AOS: Saving primary data. [2025-10-20T17:15:04.191397] [DEBUG] [LOCAL ] AOS: Trimmed 71026 records to 48689 (keeping 200 days of data) [2025-10-20T17:15:04.311575] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-01 (2.45 seconds). [2025-10-20T17:15:04.343243] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-02. [2025-10-20T17:15:04.548174] [DEBUG] [LOCAL ] AOS: Primary data saved (48689 records). [2025-10-20T17:15:04.548370] [INFO ] [MAIN ] AOS: Full history saved. [2025-10-20T17:15:04.548499] [INFO ] [MAIN ] Update Progress: 129/2367 symbols completed. AOS took 38.9s. Estimated remaining: 4h 20m 16s. Rate: 8.6 symbols/min [2025-10-20T17:15:04.738443] [INFO ] [MAIN ] APLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:04.798936] [DEBUG] [LOCAL ] APLE: Loaded 52733 existing records from primary storage. [2025-10-20T17:15:04.799452] [INFO ] [MAIN ] APLE: Loaded 52733 existing records. [2025-10-20T17:15:05.007719] [WARN ] [MAIN ] APLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:05.028673] [INFO ] [MAIN ] APLE: Wiping existing primary data before full refresh. [2025-10-20T17:15:05.030356] [INFO ] [LOCAL ] APLE: Starting deletion of primary data. [2025-10-20T17:15:05.030203] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-01 (2.21 seconds). [2025-10-20T17:15:05.040069] [DEBUG] [LOCAL ] APLE: Deleted primary data file. [2025-10-20T17:15:05.040153] [INFO ] [MAIN ] APLE: Starting full primary refresh. [2025-10-20T17:15:05.073852] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-02. [2025-10-20T17:15:05.422851] [INFO ] [REMOTE ] APLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:05.851463] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-01. [2025-10-20T17:15:06.094837] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T17:15:06.197021] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:15:06.651355] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-02. [2025-10-20T17:15:06.758126] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T17:15:07.451442] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-03. [2025-10-20T17:15:07.708749] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:15:08.259949] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-03. [2025-10-20T17:15:08.265712] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T17:15:08.909387] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T17:15:09.051436] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-03. [2025-10-20T17:15:09.851374] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-04. [2025-10-20T17:15:10.078118] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T17:15:10.589798] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:15:10.651351] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-04. [2025-10-20T17:15:11.451395] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-04. [2025-10-20T17:15:11.599533] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T17:15:12.251431] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-05. [2025-10-20T17:15:12.634618] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T17:15:13.051438] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-05. [2025-10-20T17:15:13.105291] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T17:15:13.698801] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T17:15:13.851413] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-02. [2025-10-20T17:15:14.651354] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-06. [2025-10-20T17:15:15.052700] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T17:15:15.451419] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-05. [2025-10-20T17:15:15.509470] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:15:16.250052] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T17:15:16.259521] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-03. [2025-10-20T17:15:17.025685] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:15:17.051563] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-06. [2025-10-20T17:15:17.851541] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-06. [2025-10-20T17:15:18.278559] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T17:15:18.651350] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-07. [2025-10-20T17:15:19.005279] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T17:15:19.161780] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:15:19.451481] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-04. [2025-10-20T17:15:20.251361] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-07. [2025-10-20T17:15:20.451479] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T17:15:21.051397] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-08. [2025-10-20T17:15:21.419486] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T17:15:21.749928] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:15:21.851377] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-05. [2025-10-20T17:15:22.651397] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-08. [2025-10-20T17:15:22.752953] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T17:15:23.237970] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T17:15:23.451364] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-09. [2025-10-20T17:15:23.834236] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T17:15:24.251393] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-06. [2025-10-20T17:15:25.051492] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-09. [2025-10-20T17:15:25.130137] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:15:25.851411] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-10. [2025-10-20T17:15:26.015815] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T17:15:26.651367] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-07. [2025-10-20T17:15:26.703305] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:15:27.451423] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-07. [2025-10-20T17:15:27.737928] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T17:15:27.747408] [INFO ] [REMOTE ] APG: Full fetch complete (72197 records). [2025-10-20T17:15:27.747676] [DEBUG] [VALIDATOR ] APG: Validating primary data. [2025-10-20T17:15:28.155257] [DEBUG] [VALIDATOR ] APG: All expected trading days are present. [2025-10-20T17:15:28.155353] [DEBUG] [VALIDATOR ] APG: Primary data validation passed. [2025-10-20T17:15:28.156119] [INFO ] [MAIN ] APG: Fetched primary data is valid. [2025-10-20T17:15:28.156185] [DEBUG] [LOCAL ] APG: Saving primary data. [2025-10-20T17:15:28.157710] [DEBUG] [LOCAL ] APG: Trimmed 72197 records to 50628 (keeping 200 days of data) [2025-10-20T17:15:28.254061] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-10. [2025-10-20T17:15:28.460072] [DEBUG] [LOCAL ] APG: Primary data saved (50628 records). [2025-10-20T17:15:28.460290] [INFO ] [MAIN ] APG: Full history saved. [2025-10-20T17:15:28.461008] [INFO ] [MAIN ] Update Progress: 130/2367 symbols completed. APG took 27.1s. Estimated remaining: 4h 25m 1s. Rate: 8.4 symbols/min [2025-10-20T17:15:28.672075] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T17:15:28.675046] [INFO ] [MAIN ] APO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:28.686065] [DEBUG] [LOCAL ] APO: No existing data file found in primary storage. [2025-10-20T17:15:28.686342] [WARN ] [MAIN ] APO: No existing data found, performing full refresh instead. [2025-10-20T17:15:28.686612] [INFO ] [MAIN ] APO: Wiping existing primary data before full refresh. [2025-10-20T17:15:28.686694] [INFO ] [LOCAL ] APO: Starting deletion of primary data. [2025-10-20T17:15:28.686757] [DEBUG] [LOCAL ] APO: No primary data file found to delete. [2025-10-20T17:15:28.687073] [INFO ] [MAIN ] APO: Starting full primary refresh. [2025-10-20T17:15:29.053527] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-08. [2025-10-20T17:15:29.060594] [INFO ] [REMOTE ] APO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:29.201185] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T17:15:29.690324] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T17:15:29.697500] [INFO ] [REMOTE ] APD: Full fetch complete (67393 records). [2025-10-20T17:15:29.697638] [DEBUG] [VALIDATOR ] APD: Validating primary data. [2025-10-20T17:15:29.854479] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-08. [2025-10-20T17:15:30.092179] [DEBUG] [VALIDATOR ] APD: All expected trading days are present. [2025-10-20T17:15:30.092307] [DEBUG] [VALIDATOR ] APD: Primary data validation passed. [2025-10-20T17:15:30.093050] [INFO ] [MAIN ] APD: Fetched primary data is valid. [2025-10-20T17:15:30.093113] [DEBUG] [LOCAL ] APD: Saving primary data. [2025-10-20T17:15:30.094667] [DEBUG] [LOCAL ] APD: Trimmed 67393 records to 45522 (keeping 200 days of data) [2025-10-20T17:15:30.382080] [DEBUG] [LOCAL ] APD: Primary data saved (45522 records). [2025-10-20T17:15:30.382233] [INFO ] [MAIN ] APD: Full history saved. [2025-10-20T17:15:30.382972] [INFO ] [MAIN ] Update Progress: 131/2367 symbols completed. APD took 32.1s. Estimated remaining: 4h 23m 25s. Rate: 8.5 symbols/min [2025-10-20T17:15:30.564396] [INFO ] [MAIN ] APOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:30.564739] [DEBUG] [LOCAL ] APOS: No existing data file found in primary storage. [2025-10-20T17:15:30.564815] [WARN ] [MAIN ] APOS: No existing data found, performing full refresh instead. [2025-10-20T17:15:30.564952] [INFO ] [MAIN ] APOS: Wiping existing primary data before full refresh. [2025-10-20T17:15:30.565015] [INFO ] [LOCAL ] APOS: Starting deletion of primary data. [2025-10-20T17:15:30.565193] [DEBUG] [LOCAL ] APOS: No primary data file found to delete. [2025-10-20T17:15:30.565276] [INFO ] [MAIN ] APOS: Starting full primary refresh. [2025-10-20T17:15:30.656492] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-01. [2025-10-20T17:15:30.861258] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T17:15:30.967033] [INFO ] [REMOTE ] APOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:31.451429] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-01. [2025-10-20T17:15:31.760419] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:15:31.998256] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T17:15:32.251414] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-02. [2025-10-20T17:15:32.705315] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T17:15:32.733866] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T17:15:33.051367] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-02. [2025-10-20T17:15:33.851365] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-09. [2025-10-20T17:15:34.651380] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-09. [2025-10-20T17:15:34.939607] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T17:15:35.451400] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-03. [2025-10-20T17:15:35.715875] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T17:15:36.251340] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-10. [2025-10-20T17:15:36.264282] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:15:36.822967] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T17:15:37.051413] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-04. [2025-10-20T17:15:37.802041] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T17:15:37.851395] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-05. [2025-10-20T17:15:37.893944] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T17:15:37.901790] [INFO ] [REMOTE ] APLE: Full fetch complete (74742 records). [2025-10-20T17:15:37.901955] [DEBUG] [VALIDATOR ] APLE: Validating primary data. [2025-10-20T17:15:38.296258] [DEBUG] [VALIDATOR ] APLE: All expected trading days are present. [2025-10-20T17:15:38.296337] [DEBUG] [VALIDATOR ] APLE: Primary data validation passed. [2025-10-20T17:15:38.297096] [INFO ] [MAIN ] APLE: Fetched primary data is valid. [2025-10-20T17:15:38.297159] [DEBUG] [LOCAL ] APLE: Saving primary data. [2025-10-20T17:15:38.298556] [DEBUG] [LOCAL ] APLE: Trimmed 74742 records to 51975 (keeping 200 days of data) [2025-10-20T17:15:38.392733] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T17:15:38.619752] [DEBUG] [LOCAL ] APLE: Primary data saved (51975 records). [2025-10-20T17:15:38.619887] [INFO ] [MAIN ] APLE: Full history saved. [2025-10-20T17:15:38.619979] [INFO ] [MAIN ] Update Progress: 132/2367 symbols completed. APLE took 34.1s. Estimated remaining: 4h 23m 38s. Rate: 8.5 symbols/min [2025-10-20T17:15:38.661765] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-06. [2025-10-20T17:15:38.806997] [INFO ] [MAIN ] APTV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:38.863291] [DEBUG] [LOCAL ] APTV: Loaded 52719 existing records from primary storage. [2025-10-20T17:15:38.863751] [INFO ] [MAIN ] APTV: Loaded 52719 existing records. [2025-10-20T17:15:39.064400] [WARN ] [MAIN ] APTV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:39.064701] [INFO ] [MAIN ] APTV: Wiping existing primary data before full refresh. [2025-10-20T17:15:39.064763] [INFO ] [LOCAL ] APTV: Starting deletion of primary data. [2025-10-20T17:15:39.065152] [DEBUG] [LOCAL ] APTV: Deleted primary data file. [2025-10-20T17:15:39.065238] [INFO ] [MAIN ] APTV: Starting full primary refresh. [2025-10-20T17:15:39.277794] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T17:15:39.465054] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-10. [2025-10-20T17:15:39.472803] [INFO ] [REMOTE ] APTV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:40.251417] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-07. [2025-10-20T17:15:40.697566] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T17:15:41.051376] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-08. [2025-10-20T17:15:41.449832] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-10-20T17:15:41.817866] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-10 (2.35 seconds). [2025-10-20T17:15:41.826718] [INFO ] [REMOTE ] APH: Full fetch complete (77869 records). [2025-10-20T17:15:41.826913] [DEBUG] [VALIDATOR ] APH: Validating primary data. [2025-10-20T17:15:41.858685] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-01. [2025-10-20T17:15:42.266876] [DEBUG] [VALIDATOR ] APH: All expected trading days are present. [2025-10-20T17:15:42.267061] [DEBUG] [VALIDATOR ] APH: Primary data validation passed. [2025-10-20T17:15:42.268117] [INFO ] [MAIN ] APH: Fetched primary data is valid. [2025-10-20T17:15:42.268187] [DEBUG] [LOCAL ] APH: Saving primary data. [2025-10-20T17:15:42.270633] [DEBUG] [LOCAL ] APH: Trimmed 77869 records to 53691 (keeping 200 days of data) [2025-10-20T17:15:42.644350] [DEBUG] [LOCAL ] APH: Primary data saved (53691 records). [2025-10-20T17:15:42.644590] [INFO ] [MAIN ] APH: Full history saved. [2025-10-20T17:15:42.644726] [INFO ] [MAIN ] Update Progress: 133/2367 symbols completed. APH took 39.8s. Estimated remaining: 4h 22m 40s. Rate: 8.5 symbols/min [2025-10-20T17:15:42.658244] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-03. [2025-10-20T17:15:42.865197] [INFO ] [MAIN ] AQN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:42.865404] [DEBUG] [LOCAL ] AQN: No existing data file found in primary storage. [2025-10-20T17:15:42.865444] [WARN ] [MAIN ] AQN: No existing data found, performing full refresh instead. [2025-10-20T17:15:42.865564] [INFO ] [MAIN ] AQN: Wiping existing primary data before full refresh. [2025-10-20T17:15:42.865606] [INFO ] [LOCAL ] AQN: Starting deletion of primary data. [2025-10-20T17:15:42.865677] [DEBUG] [LOCAL ] AQN: No primary data file found to delete. [2025-10-20T17:15:42.865732] [INFO ] [MAIN ] AQN: Starting full primary refresh. [2025-10-20T17:15:43.213851] [INFO ] [REMOTE ] AQN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:43.451395] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-01. [2025-10-20T17:15:43.507628] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:15:44.251398] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-02. [2025-10-20T17:15:44.459911] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T17:15:45.019853] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:15:45.051360] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-04. [2025-10-20T17:15:45.835460] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:15:45.851339] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-03. [2025-10-20T17:15:46.651365] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-02. [2025-10-20T17:15:46.933796] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T17:15:47.451351] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-09. [2025-10-20T17:15:47.675085] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T17:15:48.251351] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-04. [2025-10-20T17:15:48.333835] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T17:15:48.661082] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T17:15:49.051389] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-05. [2025-10-20T17:15:49.800873] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:15:49.851334] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-05. [2025-10-20T17:15:50.651380] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-03. [2025-10-20T17:15:51.072148] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-05 (2.02 seconds). [2025-10-20T17:15:51.451381] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-10. [2025-10-20T17:15:51.575140] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:15:52.025466] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:15:52.127842] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T17:15:52.129158] [INFO ] [REMOTE ] APOS: Full fetch complete (10240 records). [2025-10-20T17:15:52.129227] [DEBUG] [VALIDATOR ] APOS: Validating primary data. [2025-10-20T17:15:52.264903] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-04. [2025-10-20T17:15:52.484265] [DEBUG] [VALIDATOR ] APOS: All expected trading days are present. [2025-10-20T17:15:52.484333] [DEBUG] [VALIDATOR ] APOS: Primary data validation passed. [2025-10-20T17:15:52.484505] [INFO ] [MAIN ] APOS: Fetched primary data is valid. [2025-10-20T17:15:52.484543] [DEBUG] [LOCAL ] APOS: Saving primary data. [2025-10-20T17:15:52.484992] [DEBUG] [LOCAL ] APOS: Trimmed 10240 records to 7187 (keeping 200 days of data) [2025-10-20T17:15:52.527491] [DEBUG] [LOCAL ] APOS: Primary data saved (7187 records). [2025-10-20T17:15:52.527579] [INFO ] [MAIN ] APOS: Full history saved. [2025-10-20T17:15:52.527662] [INFO ] [MAIN ] Update Progress: 134/2367 symbols completed. APOS took 22.1s. Estimated remaining: 4h 23m 20s. Rate: 8.5 symbols/min [2025-10-20T17:15:52.695347] [INFO ] [MAIN ] AQNB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:52.701429] [DEBUG] [LOCAL ] AQNB: Loaded 4088 existing records from primary storage. [2025-10-20T17:15:52.701633] [INFO ] [MAIN ] AQNB: Loaded 4088 existing records. [2025-10-20T17:15:52.875321] [WARN ] [MAIN ] AQNB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:52.875501] [INFO ] [MAIN ] AQNB: Wiping existing primary data before full refresh. [2025-10-20T17:15:52.875547] [INFO ] [LOCAL ] AQNB: Starting deletion of primary data. [2025-10-20T17:15:52.875710] [DEBUG] [LOCAL ] AQNB: Deleted primary data file. [2025-10-20T17:15:52.875768] [INFO ] [MAIN ] AQNB: Starting full primary refresh. [2025-10-20T17:15:53.054422] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-06. [2025-10-20T17:15:53.235637] [INFO ] [REMOTE ] AQNB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:53.851372] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-01. [2025-10-20T17:15:54.078476] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T17:15:54.427171] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T17:15:54.600144] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:15:54.651381] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-07. [2025-10-20T17:15:55.451436] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-06. [2025-10-20T17:15:56.251381] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-02. [2025-10-20T17:15:56.421618] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T17:15:56.672903] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T17:15:57.051394] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-08. [2025-10-20T17:15:57.238442] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T17:15:57.851374] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-03. [2025-10-20T17:15:58.495100] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T17:15:58.651373] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-05. [2025-10-20T17:15:58.802495] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T17:15:59.451416] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-04. [2025-10-20T17:15:59.830418] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-10-20T17:16:00.251384] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-05. [2025-10-20T17:16:00.407914] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T17:16:01.020721] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T17:16:01.051330] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-06. [2025-10-20T17:16:01.839294] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:16:01.851323] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-07. [2025-10-20T17:16:02.443533] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T17:16:02.651369] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-06. [2025-10-20T17:16:03.451370] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-09. [2025-10-20T17:16:04.251454] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-08. [2025-10-20T17:16:04.466867] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T17:16:04.965334] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T17:16:05.051349] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-07. [2025-10-20T17:16:05.108012] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T17:16:05.851413] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-09. [2025-10-20T17:16:06.419366] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T17:16:06.651431] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-10. [2025-10-20T17:16:06.846323] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:16:07.110215] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T17:16:07.111206] [INFO ] [REMOTE ] AQNB: Full fetch complete (5420 records). [2025-10-20T17:16:07.111276] [DEBUG] [VALIDATOR ] AQNB: Validating primary data. [2025-10-20T17:16:07.459405] [DEBUG] [VALIDATOR ] AQNB: All expected trading days are present. [2025-10-20T17:16:07.459509] [DEBUG] [VALIDATOR ] AQNB: Primary data validation passed. [2025-10-20T17:16:07.459818] [INFO ] [MAIN ] AQNB: Fetched primary data is valid. [2025-10-20T17:16:07.459894] [DEBUG] [LOCAL ] AQNB: Saving primary data. [2025-10-20T17:16:07.454585] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-08. [2025-10-20T17:16:07.461111] [DEBUG] [LOCAL ] AQNB: Trimmed 5420 records to 3972 (keeping 200 days of data) [2025-10-20T17:16:07.484996] [DEBUG] [LOCAL ] AQNB: Primary data saved (3972 records). [2025-10-20T17:16:07.485092] [INFO ] [MAIN ] AQNB: Full history saved. [2025-10-20T17:16:07.485167] [INFO ] [MAIN ] Update Progress: 135/2367 symbols completed. AQNB took 15.0s. Estimated remaining: 4h 25m 23s. Rate: 8.4 symbols/min [2025-10-20T17:16:07.663367] [INFO ] [MAIN ] AR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:07.714479] [DEBUG] [LOCAL ] AR: Loaded 54366 existing records from primary storage. [2025-10-20T17:16:07.714852] [INFO ] [MAIN ] AR: Loaded 54366 existing records. [2025-10-20T17:16:07.887265] [WARN ] [MAIN ] AR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:07.887479] [INFO ] [MAIN ] AR: Wiping existing primary data before full refresh. [2025-10-20T17:16:07.887526] [INFO ] [LOCAL ] AR: Starting deletion of primary data. [2025-10-20T17:16:07.887911] [DEBUG] [LOCAL ] AR: Deleted primary data file. [2025-10-20T17:16:07.887967] [INFO ] [MAIN ] AR: Starting full primary refresh. [2025-10-20T17:16:08.237284] [INFO ] [REMOTE ] AR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:08.251335] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-01. [2025-10-20T17:16:08.785819] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T17:16:09.051379] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-09. [2025-10-20T17:16:09.851356] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-07. [2025-10-20T17:16:10.258183] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T17:16:10.405612] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:16:10.651343] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-10. [2025-10-20T17:16:11.451408] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-10. [2025-10-20T17:16:11.830229] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T17:16:12.235728] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T17:16:12.243037] [INFO ] [REMOTE ] APTV: Full fetch complete (75535 records). [2025-10-20T17:16:12.243214] [DEBUG] [VALIDATOR ] APTV: Validating primary data. [2025-10-20T17:16:12.244018] [FATAL] [VALIDATOR ] APTV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:16:12.244115] [FATAL] [MAIN ] APTV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:16:12.244157] [WARN ] [MAIN ] APTV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:16:12.244221] [INFO ] [LOCAL ] APTV: Starting deletion of primary data. [2025-10-20T17:16:12.244285] [DEBUG] [LOCAL ] APTV: No primary data file found to delete. [2025-10-20T17:16:12.244319] [INFO ] [LOCAL ] APTV: Starting deletion of derivative data. [2025-10-20T17:16:12.244491] [DEBUG] [LOCAL ] APTV: Deleted derivative data file. [2025-10-20T17:16:12.244583] [INFO ] [MAIN ] Update Progress: 136/2367 symbols completed. APTV took 33.6s. Estimated remaining: 4h 24m 37s. Rate: 8.4 symbols/min [2025-10-20T17:16:12.257565] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-08. [2025-10-20T17:16:12.420471] [INFO ] [MAIN ] ARCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:12.460941] [DEBUG] [LOCAL ] ARCO: Loaded 45143 existing records from primary storage. [2025-10-20T17:16:12.461285] [INFO ] [MAIN ] ARCO: Loaded 45143 existing records. [2025-10-20T17:16:12.679966] [WARN ] [MAIN ] ARCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:12.680285] [INFO ] [MAIN ] ARCO: Wiping existing primary data before full refresh. [2025-10-20T17:16:12.680433] [INFO ] [LOCAL ] ARCO: Starting deletion of primary data. [2025-10-20T17:16:12.687471] [DEBUG] [LOCAL ] ARCO: Deleted primary data file. [2025-10-20T17:16:12.700284] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T17:16:12.699696] [INFO ] [MAIN ] ARCO: Starting full primary refresh. [2025-10-20T17:16:12.760738] [INFO ] [REMOTE ] AQN: Full fetch complete (75371 records). [2025-10-20T17:16:12.789152] [DEBUG] [VALIDATOR ] AQN: Validating primary data. [2025-10-20T17:16:13.055818] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-02. [2025-10-20T17:16:13.585811] [DEBUG] [VALIDATOR ] AQN: All expected trading days are present. [2025-10-20T17:16:13.585529] [INFO ] [REMOTE ] ARCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:13.585977] [DEBUG] [VALIDATOR ] AQN: Primary data validation passed. [2025-10-20T17:16:13.587153] [INFO ] [MAIN ] AQN: Fetched primary data is valid. [2025-10-20T17:16:13.587221] [DEBUG] [LOCAL ] AQN: Saving primary data. [2025-10-20T17:16:13.588787] [DEBUG] [LOCAL ] AQN: Trimmed 75371 records to 51714 (keeping 200 days of data) [2025-10-20T17:16:13.866508] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-01. [2025-10-20T17:16:13.885685] [DEBUG] [LOCAL ] AQN: Primary data saved (51714 records). [2025-10-20T17:16:13.886013] [INFO ] [MAIN ] AQN: Full history saved. [2025-10-20T17:16:13.886179] [INFO ] [MAIN ] Update Progress: 137/2367 symbols completed. AQN took 31.2s. Estimated remaining: 4h 23m 1s. Rate: 8.5 symbols/min [2025-10-20T17:16:14.063320] [INFO ] [MAIN ] ARDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:14.063496] [DEBUG] [LOCAL ] ARDC: No existing data file found in primary storage. [2025-10-20T17:16:14.063626] [WARN ] [MAIN ] ARDC: No existing data found, performing full refresh instead. [2025-10-20T17:16:14.063797] [INFO ] [MAIN ] ARDC: Wiping existing primary data before full refresh. [2025-10-20T17:16:14.063915] [INFO ] [LOCAL ] ARDC: Starting deletion of primary data. [2025-10-20T17:16:14.063982] [DEBUG] [LOCAL ] ARDC: No primary data file found to delete. [2025-10-20T17:16:14.064016] [INFO ] [MAIN ] ARDC: Starting full primary refresh. [2025-10-20T17:16:14.379797] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-10-20T17:16:14.466202] [INFO ] [REMOTE ] ARDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:14.651365] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-09. [2025-10-20T17:16:14.747047] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:16:15.451395] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-03. [2025-10-20T17:16:15.546515] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T17:16:16.251443] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-02. [2025-10-20T17:16:16.612518] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T17:16:17.051396] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-01. [2025-10-20T17:16:17.349748] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T17:16:17.805774] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:16:17.851336] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-03. [2025-10-20T17:16:17.955668] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T17:16:18.651382] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-02. [2025-10-20T17:16:19.430291] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T17:16:19.456498] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-03. [2025-10-20T17:16:19.469797] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:16:20.149955] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T17:16:20.251365] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-04. [2025-10-20T17:16:21.051377] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-04. [2025-10-20T17:16:21.851399] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-04. [2025-10-20T17:16:22.036277] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T17:16:22.258863] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T17:16:22.651376] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-05. [2025-10-20T17:16:23.148005] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T17:16:23.451386] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-10. [2025-10-20T17:16:24.251396] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-05. [2025-10-20T17:16:24.514227] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:16:25.055748] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-06. [2025-10-20T17:16:25.060802] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T17:16:25.068838] [INFO ] [REMOTE ] APO: Full fetch complete (77077 records). [2025-10-20T17:16:25.069004] [DEBUG] [VALIDATOR ] APO: Validating primary data. [2025-10-20T17:16:25.069937] [FATAL] [VALIDATOR ] APO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:16:25.070030] [FATAL] [MAIN ] APO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:16:25.070073] [WARN ] [MAIN ] APO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:16:25.070140] [INFO ] [LOCAL ] APO: Starting deletion of primary data. [2025-10-20T17:16:25.070206] [DEBUG] [LOCAL ] APO: No primary data file found to delete. [2025-10-20T17:16:25.070241] [INFO ] [LOCAL ] APO: Starting deletion of derivative data. [2025-10-20T17:16:25.070279] [DEBUG] [LOCAL ] APO: No derivative data file found to delete. [2025-10-20T17:16:25.070336] [INFO ] [MAIN ] Update Progress: 138/2367 symbols completed. APO took 56.6s. Estimated remaining: 4h 23m 60s. Rate: 8.4 symbols/min [2025-10-20T17:16:25.252548] [INFO ] [MAIN ] ARDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:25.279025] [DEBUG] [LOCAL ] ARDT: Loaded 28112 existing records from primary storage. [2025-10-20T17:16:25.279294] [INFO ] [MAIN ] ARDT: Loaded 28112 existing records. [2025-10-20T17:16:25.453447] [WARN ] [MAIN ] ARDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:25.453702] [INFO ] [MAIN ] ARDT: Wiping existing primary data before full refresh. [2025-10-20T17:16:25.453769] [INFO ] [LOCAL ] ARDT: Starting deletion of primary data. [2025-10-20T17:16:25.454222] [DEBUG] [LOCAL ] ARDT: Deleted primary data file. [2025-10-20T17:16:25.454282] [INFO ] [MAIN ] ARDT: Starting full primary refresh. [2025-10-20T17:16:25.828939] [INFO ] [REMOTE ] ARDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:25.851344] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-05. [2025-10-20T17:16:25.988285] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:16:26.651381] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-06. [2025-10-20T17:16:26.714627] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T17:16:27.098730] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-10-20T17:16:27.451410] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-01. [2025-10-20T17:16:27.952249] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T17:16:28.251373] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-07. [2025-10-20T17:16:28.750867] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T17:16:29.051376] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-07. [2025-10-20T17:16:29.576528] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T17:16:29.851399] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-02. [2025-10-20T17:16:30.555309] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:16:30.651346] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-08. [2025-10-20T17:16:30.992374] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T17:16:31.451397] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-06. [2025-10-20T17:16:32.251368] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-08. [2025-10-20T17:16:32.482856] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T17:16:32.647586] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T17:16:33.051391] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-03. [2025-10-20T17:16:33.665319] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T17:16:33.851375] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-09. [2025-10-20T17:16:34.132548] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T17:16:34.651411] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-04. [2025-10-20T17:16:35.012069] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T17:16:35.451421] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-10. [2025-10-20T17:16:35.985328] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:16:36.251434] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-09. [2025-10-20T17:16:36.760424] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:16:36.768453] [INFO ] [REMOTE ] ARCO: Full fetch complete (65147 records). [2025-10-20T17:16:36.768710] [DEBUG] [VALIDATOR ] ARCO: Validating primary data. [2025-10-20T17:16:37.056555] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-07. [2025-10-20T17:16:37.182697] [DEBUG] [VALIDATOR ] ARCO: All expected trading days are present. [2025-10-20T17:16:37.182832] [DEBUG] [VALIDATOR ] ARCO: Primary data validation passed. [2025-10-20T17:16:37.183576] [INFO ] [MAIN ] ARCO: Fetched primary data is valid. [2025-10-20T17:16:37.183622] [DEBUG] [LOCAL ] ARCO: Saving primary data. [2025-10-20T17:16:37.185744] [DEBUG] [LOCAL ] ARCO: Trimmed 65147 records to 44434 (keeping 200 days of data) [2025-10-20T17:16:37.470263] [DEBUG] [LOCAL ] ARCO: Primary data saved (44434 records). [2025-10-20T17:16:37.470501] [INFO ] [MAIN ] ARCO: Full history saved. [2025-10-20T17:16:37.470636] [INFO ] [MAIN ] Update Progress: 139/2367 symbols completed. ARCO took 25.2s. Estimated remaining: 4h 25m 18s. Rate: 8.4 symbols/min [2025-10-20T17:16:37.652667] [INFO ] [MAIN ] ARE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:37.652908] [DEBUG] [LOCAL ] ARE: No existing data file found in primary storage. [2025-10-20T17:16:37.652949] [WARN ] [MAIN ] ARE: No existing data found, performing full refresh instead. [2025-10-20T17:16:37.653075] [INFO ] [MAIN ] ARE: Wiping existing primary data before full refresh. [2025-10-20T17:16:37.653118] [INFO ] [LOCAL ] ARE: Starting deletion of primary data. [2025-10-20T17:16:37.653160] [DEBUG] [LOCAL ] ARE: No primary data file found to delete. [2025-10-20T17:16:37.653194] [INFO ] [MAIN ] ARE: Starting full primary refresh. [2025-10-20T17:16:37.856688] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-05. [2025-10-20T17:16:38.067406] [INFO ] [REMOTE ] ARE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:38.197709] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T17:16:38.312373] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T17:16:38.651432] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-01. [2025-10-20T17:16:38.834176] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T17:16:39.451387] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-10. [2025-10-20T17:16:40.251375] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-06. [2025-10-20T17:16:40.530337] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T17:16:41.051343] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-08. [2025-10-20T17:16:41.421847] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T17:16:41.541000] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T17:16:41.549528] [INFO ] [REMOTE ] AR: Full fetch complete (77737 records). [2025-10-20T17:16:41.549770] [DEBUG] [VALIDATOR ] AR: Validating primary data. [2025-10-20T17:16:41.740069] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:16:41.856511] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-02. [2025-10-20T17:16:41.973450] [DEBUG] [VALIDATOR ] AR: All expected trading days are present. [2025-10-20T17:16:41.973532] [DEBUG] [VALIDATOR ] AR: Primary data validation passed. [2025-10-20T17:16:41.974493] [INFO ] [MAIN ] AR: Fetched primary data is valid. [2025-10-20T17:16:41.974566] [DEBUG] [LOCAL ] AR: Saving primary data. [2025-10-20T17:16:41.976510] [DEBUG] [LOCAL ] AR: Trimmed 77737 records to 53590 (keeping 200 days of data) [2025-10-20T17:16:42.318725] [DEBUG] [LOCAL ] AR: Primary data saved (53590 records). [2025-10-20T17:16:42.318878] [INFO ] [MAIN ] AR: Full history saved. [2025-10-20T17:16:42.319824] [INFO ] [MAIN ] Update Progress: 140/2367 symbols completed. AR took 34.8s. Estimated remaining: 4h 24m 34s. Rate: 8.4 symbols/min [2025-10-20T17:16:42.496190] [INFO ] [MAIN ] ARES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:42.555602] [DEBUG] [LOCAL ] ARES: Loaded 50486 existing records from primary storage. [2025-10-20T17:16:42.555964] [INFO ] [MAIN ] ARES: Loaded 50486 existing records. [2025-10-20T17:16:42.656629] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-07. [2025-10-20T17:16:42.740302] [WARN ] [MAIN ] ARES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:42.740551] [INFO ] [MAIN ] ARES: Wiping existing primary data before full refresh. [2025-10-20T17:16:42.740621] [INFO ] [LOCAL ] ARES: Starting deletion of primary data. [2025-10-20T17:16:42.740900] [DEBUG] [LOCAL ] ARES: Deleted primary data file. [2025-10-20T17:16:42.740960] [INFO ] [MAIN ] ARES: Starting full primary refresh. [2025-10-20T17:16:43.112221] [INFO ] [REMOTE ] ARES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:43.451412] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-09. [2025-10-20T17:16:43.549504] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:16:44.081047] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:16:44.251364] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-08. [2025-10-20T17:16:44.441915] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T17:16:45.051381] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-01. [2025-10-20T17:16:45.446997] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T17:16:45.851380] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-09. [2025-10-20T17:16:46.651385] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-03. [2025-10-20T17:16:46.707023] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:16:46.956439] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T17:16:47.451388] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-02. [2025-10-20T17:16:48.251352] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-10. [2025-10-20T17:16:48.295965] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T17:16:48.749794] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T17:16:49.051375] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-03. [2025-10-20T17:16:49.409248] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:16:49.412440] [INFO ] [REMOTE ] ARDC: Full fetch complete (21441 records). [2025-10-20T17:16:49.412530] [DEBUG] [VALIDATOR ] ARDC: Validating primary data. [2025-10-20T17:16:49.811950] [DEBUG] [VALIDATOR ] ARDC: All expected trading days are present. [2025-10-20T17:16:49.812048] [DEBUG] [VALIDATOR ] ARDC: Primary data validation passed. [2025-10-20T17:16:49.812423] [INFO ] [MAIN ] ARDC: Fetched primary data is valid. [2025-10-20T17:16:49.812496] [DEBUG] [LOCAL ] ARDC: Saving primary data. [2025-10-20T17:16:49.813101] [DEBUG] [LOCAL ] ARDC: Trimmed 21441 records to 15280 (keeping 200 days of data) [2025-10-20T17:16:49.855060] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-04. [2025-10-20T17:16:49.918186] [DEBUG] [LOCAL ] ARDC: Primary data saved (15280 records). [2025-10-20T17:16:49.918551] [INFO ] [MAIN ] ARDC: Full history saved. [2025-10-20T17:16:49.918886] [INFO ] [MAIN ] Update Progress: 141/2367 symbols completed. ARDC took 36.0s. Estimated remaining: 4h 24m 34s. Rate: 8.4 symbols/min [2025-10-20T17:16:50.114579] [INFO ] [MAIN ] ARI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:50.155425] [DEBUG] [LOCAL ] ARI: Loaded 44332 existing records from primary storage. [2025-10-20T17:16:50.155753] [INFO ] [MAIN ] ARI: Loaded 44332 existing records. [2025-10-20T17:16:50.370731] [WARN ] [MAIN ] ARI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:50.370953] [INFO ] [MAIN ] ARI: Wiping existing primary data before full refresh. [2025-10-20T17:16:50.371001] [INFO ] [LOCAL ] ARI: Starting deletion of primary data. [2025-10-20T17:16:50.371277] [DEBUG] [LOCAL ] ARI: Deleted primary data file. [2025-10-20T17:16:50.371334] [INFO ] [MAIN ] ARI: Starting full primary refresh. [2025-10-20T17:16:50.655980] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-10. [2025-10-20T17:16:50.767517] [INFO ] [REMOTE ] ARI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:51.011465] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T17:16:51.451435] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-01. [2025-10-20T17:16:52.021549] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:16:52.026497] [INFO ] [REMOTE ] ARDT: Full fetch complete (41543 records). [2025-10-20T17:16:52.026704] [DEBUG] [VALIDATOR ] ARDT: Validating primary data. [2025-10-20T17:16:52.263695] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-04. [2025-10-20T17:16:52.505023] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-04 (2.65 seconds). [2025-10-20T17:16:52.505582] [DEBUG] [VALIDATOR ] ARDT: All expected trading days are present. [2025-10-20T17:16:52.516830] [DEBUG] [VALIDATOR ] ARDT: Primary data validation passed. [2025-10-20T17:16:52.517849] [INFO ] [MAIN ] ARDT: Fetched primary data is valid. [2025-10-20T17:16:52.517963] [DEBUG] [LOCAL ] ARDT: Saving primary data. [2025-10-20T17:16:52.519715] [DEBUG] [LOCAL ] ARDT: Trimmed 41543 records to 27434 (keeping 200 days of data) [2025-10-20T17:16:52.722361] [DEBUG] [LOCAL ] ARDT: Primary data saved (27434 records). [2025-10-20T17:16:52.724473] [INFO ] [MAIN ] ARDT: Full history saved. [2025-10-20T17:16:52.724340] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:16:52.724677] [INFO ] [MAIN ] Update Progress: 142/2367 symbols completed. ARDT took 27.7s. Estimated remaining: 4h 23m 19s. Rate: 8.4 symbols/min [2025-10-20T17:16:52.926266] [INFO ] [MAIN ] ARLO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:52.970430] [DEBUG] [LOCAL ] ARLO: Loaded 44394 existing records from primary storage. [2025-10-20T17:16:52.970894] [INFO ] [MAIN ] ARLO: Loaded 44394 existing records. [2025-10-20T17:16:53.056597] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-02. [2025-10-20T17:16:53.161990] [WARN ] [MAIN ] ARLO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:53.162303] [INFO ] [MAIN ] ARLO: Wiping existing primary data before full refresh. [2025-10-20T17:16:53.162359] [INFO ] [LOCAL ] ARLO: Starting deletion of primary data. [2025-10-20T17:16:53.162603] [DEBUG] [LOCAL ] ARLO: Deleted primary data file. [2025-10-20T17:16:53.162677] [INFO ] [MAIN ] ARLO: Starting full primary refresh. [2025-10-20T17:16:53.558174] [INFO ] [REMOTE ] ARLO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:53.735408] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:16:53.851369] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-05. [2025-10-20T17:16:54.639865] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:16:54.651333] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-03. [2025-10-20T17:16:55.451369] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-05. [2025-10-20T17:16:55.694477] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T17:16:55.776370] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T17:16:56.251417] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-01. [2025-10-20T17:16:57.051379] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-06. [2025-10-20T17:16:57.416496] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T17:16:57.720160] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T17:16:57.851421] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-06. [2025-10-20T17:16:58.522964] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T17:16:58.651410] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-07. [2025-10-20T17:16:59.451402] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-04. [2025-10-20T17:16:59.544044] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:17:00.251374] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-07. [2025-10-20T17:17:00.476767] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T17:17:01.051489] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-08. [2025-10-20T17:17:01.851403] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-02. [2025-10-20T17:17:01.936561] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-04 (2.49 seconds). [2025-10-20T17:17:02.573861] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-07 (2.32 seconds). [2025-10-20T17:17:02.651446] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-05. [2025-10-20T17:17:03.101464] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T17:17:03.139211] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T17:17:03.451408] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-08. [2025-10-20T17:17:04.251408] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-09. [2025-10-20T17:17:04.411171] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T17:17:05.051430] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-06. [2025-10-20T17:17:05.216770] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T17:17:05.765757] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T17:17:05.851378] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-10. [2025-10-20T17:17:06.550118] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T17:17:06.651403] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-09. [2025-10-20T17:17:07.451366] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-03. [2025-10-20T17:17:07.526788] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T17:17:07.534950] [INFO ] [REMOTE ] ARE: Full fetch complete (73020 records). [2025-10-20T17:17:07.535083] [DEBUG] [VALIDATOR ] ARE: Validating primary data. [2025-10-20T17:17:07.958875] [DEBUG] [VALIDATOR ] ARE: All expected trading days are present. [2025-10-20T17:17:07.958980] [DEBUG] [VALIDATOR ] ARE: Primary data validation passed. [2025-10-20T17:17:07.959808] [INFO ] [MAIN ] ARE: Fetched primary data is valid. [2025-10-20T17:17:07.959908] [DEBUG] [LOCAL ] ARE: Saving primary data. [2025-10-20T17:17:07.961500] [DEBUG] [LOCAL ] ARE: Trimmed 73020 records to 50965 (keeping 200 days of data) [2025-10-20T17:17:08.263458] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-07. [2025-10-20T17:17:08.284516] [DEBUG] [LOCAL ] ARE: Primary data saved (50965 records). [2025-10-20T17:17:08.285931] [INFO ] [MAIN ] ARE: Full history saved. [2025-10-20T17:17:08.286111] [INFO ] [MAIN ] Update Progress: 143/2367 symbols completed. ARE took 30.8s. Estimated remaining: 4h 25m 24s. Rate: 8.4 symbols/min [2025-10-20T17:17:08.472841] [INFO ] [MAIN ] ARMK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:08.504206] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:17:08.555695] [DEBUG] [LOCAL ] ARMK: Loaded 51136 existing records from primary storage. [2025-10-20T17:17:08.556047] [INFO ] [MAIN ] ARMK: Loaded 51136 existing records. [2025-10-20T17:17:08.732809] [WARN ] [MAIN ] ARMK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:08.733019] [INFO ] [MAIN ] ARMK: Wiping existing primary data before full refresh. [2025-10-20T17:17:08.733067] [INFO ] [LOCAL ] ARMK: Starting deletion of primary data. [2025-10-20T17:17:08.733456] [DEBUG] [LOCAL ] ARMK: Deleted primary data file. [2025-10-20T17:17:08.733514] [INFO ] [MAIN ] ARMK: Starting full primary refresh. [2025-10-20T17:17:09.056584] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-10. [2025-10-20T17:17:09.096993] [INFO ] [REMOTE ] ARMK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:09.359716] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T17:17:09.513466] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:17:09.851428] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-04. [2025-10-20T17:17:10.651372] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-08. [2025-10-20T17:17:10.923059] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T17:17:10.931437] [INFO ] [REMOTE ] ARES: Full fetch complete (71606 records). [2025-10-20T17:17:10.931699] [DEBUG] [VALIDATOR ] ARES: Validating primary data. [2025-10-20T17:17:11.247429] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:17:11.374052] [DEBUG] [VALIDATOR ] ARES: All expected trading days are present. [2025-10-20T17:17:11.374138] [DEBUG] [VALIDATOR ] ARES: Primary data validation passed. [2025-10-20T17:17:11.374847] [INFO ] [MAIN ] ARES: Fetched primary data is valid. [2025-10-20T17:17:11.374910] [DEBUG] [LOCAL ] ARES: Saving primary data. [2025-10-20T17:17:11.376300] [DEBUG] [LOCAL ] ARES: Trimmed 71606 records to 49703 (keeping 200 days of data) [2025-10-20T17:17:11.468472] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-05. [2025-10-20T17:17:11.689191] [DEBUG] [LOCAL ] ARES: Primary data saved (49703 records). [2025-10-20T17:17:11.689330] [INFO ] [MAIN ] ARES: Full history saved. [2025-10-20T17:17:11.689964] [INFO ] [MAIN ] Update Progress: 144/2367 symbols completed. ARES took 29.4s. Estimated remaining: 4h 24m 19s. Rate: 8.4 symbols/min [2025-10-20T17:17:11.871468] [INFO ] [MAIN ] AROC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:11.899343] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T17:17:11.950069] [DEBUG] [LOCAL ] AROC: Loaded 49789 existing records from primary storage. [2025-10-20T17:17:11.950405] [INFO ] [MAIN ] AROC: Loaded 49789 existing records. [2025-10-20T17:17:12.128680] [WARN ] [MAIN ] AROC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:12.128939] [INFO ] [MAIN ] AROC: Wiping existing primary data before full refresh. [2025-10-20T17:17:12.129002] [INFO ] [LOCAL ] AROC: Starting deletion of primary data. [2025-10-20T17:17:12.129228] [DEBUG] [LOCAL ] AROC: Deleted primary data file. [2025-10-20T17:17:12.129285] [INFO ] [MAIN ] AROC: Starting full primary refresh. [2025-10-20T17:17:12.256564] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-01. [2025-10-20T17:17:12.498159] [INFO ] [REMOTE ] AROC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:13.020899] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T17:17:13.051340] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-06. [2025-10-20T17:17:13.605380] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T17:17:13.851405] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-09. [2025-10-20T17:17:14.651357] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-02. [2025-10-20T17:17:14.722531] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:17:15.443365] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T17:17:15.452456] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-01. [2025-10-20T17:17:16.251469] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-10. [2025-10-20T17:17:16.304527] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:17:17.051429] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-03. [2025-10-20T17:17:17.233633] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T17:17:17.851464] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-02. [2025-10-20T17:17:18.074792] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T17:17:18.081776] [INFO ] [REMOTE ] ARI: Full fetch complete (62121 records). [2025-10-20T17:17:18.081939] [DEBUG] [VALIDATOR ] ARI: Validating primary data. [2025-10-20T17:17:18.082782] [FATAL] [VALIDATOR ] ARI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:17:18.082883] [FATAL] [MAIN ] ARI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:17:18.082966] [WARN ] [MAIN ] ARI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:17:18.083059] [INFO ] [LOCAL ] ARI: Starting deletion of primary data. [2025-10-20T17:17:18.083139] [DEBUG] [LOCAL ] ARI: No primary data file found to delete. [2025-10-20T17:17:18.083179] [INFO ] [LOCAL ] ARI: Starting deletion of derivative data. [2025-10-20T17:17:18.083377] [DEBUG] [LOCAL ] ARI: Deleted derivative data file. [2025-10-20T17:17:18.083477] [INFO ] [MAIN ] Update Progress: 145/2367 symbols completed. ARI took 28.2s. Estimated remaining: 4h 24m 0s. Rate: 8.4 symbols/min [2025-10-20T17:17:18.263341] [INFO ] [MAIN ] ARR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:18.318396] [DEBUG] [LOCAL ] ARR: Loaded 52880 existing records from primary storage. [2025-10-20T17:17:18.318851] [INFO ] [MAIN ] ARR: Loaded 52880 existing records. [2025-10-20T17:17:18.509944] [WARN ] [MAIN ] ARR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:18.510217] [INFO ] [MAIN ] ARR: Wiping existing primary data before full refresh. [2025-10-20T17:17:18.510267] [INFO ] [LOCAL ] ARR: Starting deletion of primary data. [2025-10-20T17:17:18.510637] [DEBUG] [LOCAL ] ARR: Deleted primary data file. [2025-10-20T17:17:18.510733] [INFO ] [MAIN ] ARR: Starting full primary refresh. [2025-10-20T17:17:18.655629] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-07. [2025-10-20T17:17:18.902803] [INFO ] [REMOTE ] ARR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:18.933070] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T17:17:19.063049] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:17:19.451390] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-03. [2025-10-20T17:17:20.251437] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-01. [2025-10-20T17:17:20.388923] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:17:21.051520] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-08. [2025-10-20T17:17:21.102689] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T17:17:21.851389] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-04. [2025-10-20T17:17:22.272974] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T17:17:22.651341] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-04. [2025-10-20T17:17:22.753841] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T17:17:23.141023] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T17:17:23.451373] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-09. [2025-10-20T17:17:24.238756] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:17:24.251380] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-05. [2025-10-20T17:17:24.740381] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T17:17:25.051417] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-02. [2025-10-20T17:17:25.851480] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-10. [2025-10-20T17:17:26.024730] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:17:26.651389] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-06. [2025-10-20T17:17:26.895606] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T17:17:27.451409] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-03. [2025-10-20T17:17:27.578180] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:17:27.584575] [INFO ] [REMOTE ] ARLO: Full fetch complete (62940 records). [2025-10-20T17:17:27.584775] [DEBUG] [VALIDATOR ] ARLO: Validating primary data. [2025-10-20T17:17:27.585808] [FATAL] [VALIDATOR ] ARLO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:17:27.585906] [FATAL] [MAIN ] ARLO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:17:27.585952] [WARN ] [MAIN ] ARLO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:17:27.585996] [INFO ] [LOCAL ] ARLO: Starting deletion of primary data. [2025-10-20T17:17:27.586061] [DEBUG] [LOCAL ] ARLO: No primary data file found to delete. [2025-10-20T17:17:27.586100] [INFO ] [LOCAL ] ARLO: Starting deletion of derivative data. [2025-10-20T17:17:27.586497] [DEBUG] [LOCAL ] ARLO: Deleted derivative data file. [2025-10-20T17:17:27.587285] [INFO ] [MAIN ] Update Progress: 146/2367 symbols completed. ARLO took 34.9s. Estimated remaining: 4h 24m 29s. Rate: 8.4 symbols/min [2025-10-20T17:17:27.768722] [INFO ] [MAIN ] ARW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:27.804256] [DEBUG] [LOCAL ] ARW: Loaded 33602 existing records from primary storage. [2025-10-20T17:17:27.804525] [INFO ] [MAIN ] ARW: Loaded 33602 existing records. [2025-10-20T17:17:27.987243] [WARN ] [MAIN ] ARW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:27.987803] [INFO ] [MAIN ] ARW: Wiping existing primary data before full refresh. [2025-10-20T17:17:27.988158] [INFO ] [LOCAL ] ARW: Starting deletion of primary data. [2025-10-20T17:17:27.988822] [DEBUG] [LOCAL ] ARW: Deleted primary data file. [2025-10-20T17:17:27.989173] [INFO ] [MAIN ] ARW: Starting full primary refresh. [2025-10-20T17:17:28.242175] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T17:17:28.267436] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-05. [2025-10-20T17:17:28.450474] [INFO ] [REMOTE ] ARW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:29.051396] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-01. [2025-10-20T17:17:29.303418] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T17:17:29.851407] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-04. [2025-10-20T17:17:30.038296] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:17:30.299271] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T17:17:30.651402] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-06. [2025-10-20T17:17:31.451404] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-02. [2025-10-20T17:17:31.843717] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T17:17:31.905273] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T17:17:32.251377] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-07. [2025-10-20T17:17:32.804395] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T17:17:33.051383] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-05. [2025-10-20T17:17:33.851360] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-07. [2025-10-20T17:17:34.026743] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T17:17:34.651423] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-03. [2025-10-20T17:17:35.214891] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-05 (2.16 seconds). [2025-10-20T17:17:35.290269] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:17:35.451369] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-08. [2025-10-20T17:17:35.797592] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T17:17:36.251411] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-06. [2025-10-20T17:17:36.923888] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T17:17:37.051369] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-09. [2025-10-20T17:17:37.851374] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-04. [2025-10-20T17:17:37.904850] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T17:17:38.651358] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-07. [2025-10-20T17:17:38.698768] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:17:39.430537] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:17:39.451339] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-05. [2025-10-20T17:17:40.251394] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-10. [2025-10-20T17:17:40.912514] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T17:17:41.053570] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-08. [2025-10-20T17:17:41.058897] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:17:41.851367] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-08. [2025-10-20T17:17:42.170561] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:17:42.179263] [INFO ] [REMOTE ] ARMK: Full fetch complete (73006 records). [2025-10-20T17:17:42.179523] [DEBUG] [VALIDATOR ] ARMK: Validating primary data. [2025-10-20T17:17:42.670691] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-06. [2025-10-20T17:17:42.671134] [DEBUG] [VALIDATOR ] ARMK: All expected trading days are present. [2025-10-20T17:17:42.671464] [DEBUG] [VALIDATOR ] ARMK: Primary data validation passed. [2025-10-20T17:17:42.672576] [INFO ] [MAIN ] ARMK: Fetched primary data is valid. [2025-10-20T17:17:42.673774] [DEBUG] [LOCAL ] ARMK: Saving primary data. [2025-10-20T17:17:42.675631] [DEBUG] [LOCAL ] ARMK: Trimmed 73006 records to 50421 (keeping 200 days of data) [2025-10-20T17:17:42.855171] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T17:17:43.047450] [DEBUG] [LOCAL ] ARMK: Primary data saved (50421 records). [2025-10-20T17:17:43.047663] [INFO ] [MAIN ] ARMK: Full history saved. [2025-10-20T17:17:43.047772] [INFO ] [MAIN ] Update Progress: 147/2367 symbols completed. ARMK took 34.8s. Estimated remaining: 4h 26m 28s. Rate: 8.3 symbols/min [2025-10-20T17:17:43.245371] [INFO ] [MAIN ] ARX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:43.245549] [DEBUG] [LOCAL ] ARX: No existing data file found in primary storage. [2025-10-20T17:17:43.245586] [WARN ] [MAIN ] ARX: No existing data found, performing full refresh instead. [2025-10-20T17:17:43.245730] [INFO ] [MAIN ] ARX: Wiping existing primary data before full refresh. [2025-10-20T17:17:43.245775] [INFO ] [LOCAL ] ARX: Starting deletion of primary data. [2025-10-20T17:17:43.245815] [DEBUG] [LOCAL ] ARX: No primary data file found to delete. [2025-10-20T17:17:43.245846] [INFO ] [MAIN ] ARX: Starting full primary refresh. [2025-10-20T17:17:43.458720] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-09. [2025-10-20T17:17:43.613773] [INFO ] [REMOTE ] ARX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:43.657609] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T17:17:43.697560] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T17:17:44.251378] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-09. [2025-10-20T17:17:45.051376] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-01. [2025-10-20T17:17:45.155734] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:17:45.259695] [DEBUG] [REMOTE ] ARX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:17:45.260040] [WARN ] [REMOTE ] ARX: No data returned for month 2025-01 [2025-10-20T17:17:45.260145] [FATAL] [REMOTE ] ARX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:17:45.260231] [FATAL] [MAIN ] ARX: Failed to fetch primary data. [2025-10-20T17:17:45.260310] [WARN ] [MAIN ] ARX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:17:45.260399] [INFO ] [LOCAL ] ARX: Starting deletion of primary data. [2025-10-20T17:17:45.260506] [DEBUG] [LOCAL ] ARX: No primary data file found to delete. [2025-10-20T17:17:45.260588] [INFO ] [LOCAL ] ARX: Starting deletion of derivative data. [2025-10-20T17:17:45.260671] [DEBUG] [LOCAL ] ARX: No derivative data file found to delete. [2025-10-20T17:17:45.260746] [INFO ] [MAIN ] Update Progress: 148/2367 symbols completed. ARX took 2.2s. Estimated remaining: 4h 25m 6s. Rate: 8.4 symbols/min [2025-10-20T17:17:45.484214] [INFO ] [MAIN ] AS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:45.540018] [DEBUG] [LOCAL ] AS: Loaded 53954 existing records from primary storage. [2025-10-20T17:17:45.540515] [INFO ] [MAIN ] AS: Loaded 53954 existing records. [2025-10-20T17:17:45.732790] [WARN ] [MAIN ] AS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:45.733091] [INFO ] [MAIN ] AS: Wiping existing primary data before full refresh. [2025-10-20T17:17:45.733156] [INFO ] [LOCAL ] AS: Starting deletion of primary data. [2025-10-20T17:17:45.733692] [DEBUG] [LOCAL ] AS: Deleted primary data file. [2025-10-20T17:17:45.733771] [INFO ] [MAIN ] AS: Starting full primary refresh. [2025-10-20T17:17:45.856790] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-10. [2025-10-20T17:17:46.176104] [INFO ] [REMOTE ] AS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:46.393575] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-10-20T17:17:46.651424] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-07. [2025-10-20T17:17:47.451422] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-10. [2025-10-20T17:17:47.654270] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T17:17:47.663527] [INFO ] [REMOTE ] AROC: Full fetch complete (71331 records). [2025-10-20T17:17:47.663786] [DEBUG] [VALIDATOR ] AROC: Validating primary data. [2025-10-20T17:17:48.030582] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:17:48.119571] [DEBUG] [VALIDATOR ] AROC: All expected trading days are present. [2025-10-20T17:17:48.119691] [DEBUG] [VALIDATOR ] AROC: Primary data validation passed. [2025-10-20T17:17:48.120420] [INFO ] [MAIN ] AROC: Fetched primary data is valid. [2025-10-20T17:17:48.120465] [DEBUG] [LOCAL ] AROC: Saving primary data. [2025-10-20T17:17:48.121898] [DEBUG] [LOCAL ] AROC: Trimmed 71331 records to 49092 (keeping 200 days of data) [2025-10-20T17:17:48.295332] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-01. [2025-10-20T17:17:48.420846] [DEBUG] [LOCAL ] AROC: Primary data saved (49092 records). [2025-10-20T17:17:48.420991] [INFO ] [MAIN ] AROC: Full history saved. [2025-10-20T17:17:48.421699] [INFO ] [MAIN ] Update Progress: 149/2367 symbols completed. AROC took 36.7s. Estimated remaining: 4h 23m 59s. Rate: 8.4 symbols/min [2025-10-20T17:17:48.597544] [INFO ] [MAIN ] ASA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:48.615873] [DEBUG] [LOCAL ] ASA: Loaded 15654 existing records from primary storage. [2025-10-20T17:17:48.616177] [INFO ] [MAIN ] ASA: Loaded 15654 existing records. [2025-10-20T17:17:48.794307] [WARN ] [MAIN ] ASA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:48.794547] [INFO ] [MAIN ] ASA: Wiping existing primary data before full refresh. [2025-10-20T17:17:48.794596] [INFO ] [LOCAL ] ASA: Starting deletion of primary data. [2025-10-20T17:17:48.794773] [DEBUG] [LOCAL ] ASA: Deleted primary data file. [2025-10-20T17:17:48.794830] [INFO ] [MAIN ] ASA: Starting full primary refresh. [2025-10-20T17:17:49.056507] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-08. [2025-10-20T17:17:49.165504] [INFO ] [REMOTE ] ASA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:49.461321] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T17:17:49.471525] [INFO ] [REMOTE ] ARR: Full fetch complete (75097 records). [2025-10-20T17:17:49.471769] [DEBUG] [VALIDATOR ] ARR: Validating primary data. [2025-10-20T17:17:49.472862] [FATAL] [VALIDATOR ] ARR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:17:49.473002] [FATAL] [MAIN ] ARR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:17:49.473059] [WARN ] [MAIN ] ARR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:17:49.473139] [INFO ] [LOCAL ] ARR: Starting deletion of primary data. [2025-10-20T17:17:49.473223] [DEBUG] [LOCAL ] ARR: No primary data file found to delete. [2025-10-20T17:17:49.473273] [INFO ] [LOCAL ] ARR: Starting deletion of derivative data. [2025-10-20T17:17:49.473482] [DEBUG] [LOCAL ] ARR: Deleted derivative data file. [2025-10-20T17:17:49.473598] [INFO ] [MAIN ] Update Progress: 150/2367 symbols completed. ARR took 31.4s. Estimated remaining: 4h 22m 22s. Rate: 8.5 symbols/min [2025-10-20T17:17:49.672432] [INFO ] [MAIN ] ASAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:49.727369] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T17:17:49.754531] [DEBUG] [LOCAL ] ASAN: Loaded 53507 existing records from primary storage. [2025-10-20T17:17:49.754923] [INFO ] [MAIN ] ASAN: Loaded 53507 existing records. [2025-10-20T17:17:49.856512] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-01. [2025-10-20T17:17:49.938983] [WARN ] [MAIN ] ASAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:49.939181] [INFO ] [MAIN ] ASAN: Wiping existing primary data before full refresh. [2025-10-20T17:17:49.939228] [INFO ] [LOCAL ] ASAN: Starting deletion of primary data. [2025-10-20T17:17:49.939611] [DEBUG] [LOCAL ] ASAN: Deleted primary data file. [2025-10-20T17:17:49.939679] [INFO ] [MAIN ] ASAN: Starting full primary refresh. [2025-10-20T17:17:50.306268] [INFO ] [REMOTE ] ASAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:50.418492] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T17:17:50.584578] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T17:17:50.651473] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-01. [2025-10-20T17:17:51.451407] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-02. [2025-10-20T17:17:52.251361] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-09. [2025-10-20T17:17:52.602062] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T17:17:53.051351] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-02. [2025-10-20T17:17:53.161739] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T17:17:53.821927] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T17:17:53.851350] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-10. [2025-10-20T17:17:54.651356] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-03. [2025-10-20T17:17:54.972837] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-10-20T17:17:55.065788] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T17:17:55.071320] [INFO ] [REMOTE ] ARW: Full fetch complete (48613 records). [2025-10-20T17:17:55.071465] [DEBUG] [VALIDATOR ] ARW: Validating primary data. [2025-10-20T17:17:55.457727] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-03. [2025-10-20T17:17:55.463854] [DEBUG] [VALIDATOR ] ARW: All expected trading days are present. [2025-10-20T17:17:55.464891] [DEBUG] [VALIDATOR ] ARW: Primary data validation passed. [2025-10-20T17:17:55.465510] [INFO ] [MAIN ] ARW: Fetched primary data is valid. [2025-10-20T17:17:55.465584] [DEBUG] [LOCAL ] ARW: Saving primary data. [2025-10-20T17:17:55.467012] [DEBUG] [LOCAL ] ARW: Trimmed 48613 records to 33236 (keeping 200 days of data) [2025-10-20T17:17:55.664346] [DEBUG] [LOCAL ] ARW: Primary data saved (33236 records). [2025-10-20T17:17:55.664477] [INFO ] [MAIN ] ARW: Full history saved. [2025-10-20T17:17:55.664937] [INFO ] [MAIN ] Update Progress: 151/2367 symbols completed. ARW took 28.1s. Estimated remaining: 4h 22m 1s. Rate: 8.5 symbols/min [2025-10-20T17:17:55.836873] [INFO ] [MAIN ] ASB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:55.837071] [DEBUG] [LOCAL ] ASB: No existing data file found in primary storage. [2025-10-20T17:17:55.837110] [WARN ] [MAIN ] ASB: No existing data found, performing full refresh instead. [2025-10-20T17:17:55.837249] [INFO ] [MAIN ] ASB: Wiping existing primary data before full refresh. [2025-10-20T17:17:55.837318] [INFO ] [LOCAL ] ASB: Starting deletion of primary data. [2025-10-20T17:17:55.837384] [DEBUG] [LOCAL ] ASB: No primary data file found to delete. [2025-10-20T17:17:55.837430] [INFO ] [MAIN ] ASB: Starting full primary refresh. [2025-10-20T17:17:56.211862] [INFO ] [REMOTE ] ASB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:56.251516] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:17:56.261807] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-02. [2025-10-20T17:17:57.051927] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-04. [2025-10-20T17:17:57.056417] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:17:57.081900] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T17:17:57.851398] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-03. [2025-10-20T17:17:58.651411] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-01. [2025-10-20T17:17:58.740825] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T17:17:58.844498] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:17:59.451397] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-04. [2025-10-20T17:18:00.185757] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T17:18:00.251340] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-02. [2025-10-20T17:18:01.051441] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-04. [2025-10-20T17:18:01.818123] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-04 (2.37 seconds). [2025-10-20T17:18:01.851628] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-05. [2025-10-20T17:18:01.903807] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:18:01.945583] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T17:18:02.651416] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-05. [2025-10-20T17:18:03.331030] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T17:18:03.451352] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-06. [2025-10-20T17:18:03.573084] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:18:04.005852] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T17:18:04.251394] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-05. [2025-10-20T17:18:05.051366] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-07. [2025-10-20T17:18:05.851352] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-03. [2025-10-20T17:18:05.998829] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:18:06.032748] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T17:18:06.651344] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-06. [2025-10-20T17:18:07.451351] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-06. [2025-10-20T17:18:07.583903] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T17:18:08.251423] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-08. [2025-10-20T17:18:08.312356] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:18:08.950709] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T17:18:09.051351] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-07. [2025-10-20T17:18:09.330067] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T17:18:09.851373] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-04. [2025-10-20T17:18:10.651401] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-07. [2025-10-20T17:18:10.983329] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T17:18:11.163094] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T17:18:11.451428] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-08. [2025-10-20T17:18:12.251358] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-05. [2025-10-20T17:18:12.698092] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-10-20T17:18:13.051354] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-08. [2025-10-20T17:18:13.336292] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T17:18:13.801471] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T17:18:13.851345] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-06. [2025-10-20T17:18:14.651429] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-09. [2025-10-20T17:18:14.893713] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T17:18:14.990194] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:18:15.451413] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-09. [2025-10-20T17:18:16.251404] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-09. [2025-10-20T17:18:16.417722] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:18:17.051363] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-10. [2025-10-20T17:18:17.268324] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T17:18:17.601590] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:18:17.851387] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-07. [2025-10-20T17:18:18.651368] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-10. [2025-10-20T17:18:19.148257] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:18:19.156053] [INFO ] [REMOTE ] AS: Full fetch complete (76526 records). [2025-10-20T17:18:19.156207] [DEBUG] [VALIDATOR ] AS: Validating primary data. [2025-10-20T17:18:19.157043] [FATAL] [VALIDATOR ] AS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:18:19.157135] [FATAL] [MAIN ] AS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:18:19.157178] [WARN ] [MAIN ] AS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:18:19.157217] [INFO ] [LOCAL ] AS: Starting deletion of primary data. [2025-10-20T17:18:19.157292] [DEBUG] [LOCAL ] AS: No primary data file found to delete. [2025-10-20T17:18:19.157329] [INFO ] [LOCAL ] AS: Starting deletion of derivative data. [2025-10-20T17:18:19.157483] [DEBUG] [LOCAL ] AS: Deleted derivative data file. [2025-10-20T17:18:19.157573] [INFO ] [MAIN ] Update Progress: 152/2367 symbols completed. AS took 33.9s. Estimated remaining: 4h 25m 53s. Rate: 8.3 symbols/min [2025-10-20T17:18:19.355671] [INFO ] [MAIN ] ASBA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:19.361912] [DEBUG] [LOCAL ] ASBA: Loaded 3617 existing records from primary storage. [2025-10-20T17:18:19.362128] [INFO ] [MAIN ] ASBA: Loaded 3617 existing records. [2025-10-20T17:18:19.458377] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-10. [2025-10-20T17:18:19.545095] [WARN ] [MAIN ] ASBA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:19.545365] [INFO ] [MAIN ] ASBA: Wiping existing primary data before full refresh. [2025-10-20T17:18:19.545531] [INFO ] [LOCAL ] ASBA: Starting deletion of primary data. [2025-10-20T17:18:19.545781] [DEBUG] [LOCAL ] ASBA: Deleted primary data file. [2025-10-20T17:18:19.545918] [INFO ] [MAIN ] ASBA: Starting full primary refresh. [2025-10-20T17:18:19.770572] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T17:18:19.956980] [INFO ] [REMOTE ] ASBA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:20.251377] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-08. [2025-10-20T17:18:20.507864] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:18:20.512425] [INFO ] [REMOTE ] ASA: Full fetch complete (20356 records). [2025-10-20T17:18:20.512515] [DEBUG] [VALIDATOR ] ASA: Validating primary data. [2025-10-20T17:18:20.512987] [FATAL] [VALIDATOR ] ASA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:18:20.513073] [FATAL] [MAIN ] ASA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:18:20.513115] [WARN ] [MAIN ] ASA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:18:20.513152] [INFO ] [LOCAL ] ASA: Starting deletion of primary data. [2025-10-20T17:18:20.513213] [DEBUG] [LOCAL ] ASA: No primary data file found to delete. [2025-10-20T17:18:20.513271] [INFO ] [LOCAL ] ASA: Starting deletion of derivative data. [2025-10-20T17:18:20.513423] [DEBUG] [LOCAL ] ASA: Deleted derivative data file. [2025-10-20T17:18:20.513509] [INFO ] [MAIN ] Update Progress: 153/2367 symbols completed. ASA took 32.1s. Estimated remaining: 4h 24m 21s. Rate: 8.4 symbols/min [2025-10-20T17:18:20.687525] [INFO ] [MAIN ] ASC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:20.722219] [DEBUG] [LOCAL ] ASC: Loaded 37898 existing records from primary storage. [2025-10-20T17:18:20.722521] [INFO ] [MAIN ] ASC: Loaded 37898 existing records. [2025-10-20T17:18:20.909256] [WARN ] [MAIN ] ASC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:20.909522] [INFO ] [MAIN ] ASC: Wiping existing primary data before full refresh. [2025-10-20T17:18:20.909614] [INFO ] [LOCAL ] ASC: Starting deletion of primary data. [2025-10-20T17:18:20.909943] [DEBUG] [LOCAL ] ASC: Deleted primary data file. [2025-10-20T17:18:20.910008] [INFO ] [MAIN ] ASC: Starting full primary refresh. [2025-10-20T17:18:21.031740] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-10 (2.38 seconds). [2025-10-20T17:18:21.058223] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-01. [2025-10-20T17:18:21.068974] [INFO ] [REMOTE ] ASAN: Full fetch complete (76635 records). [2025-10-20T17:18:21.095020] [DEBUG] [VALIDATOR ] ASAN: Validating primary data. [2025-10-20T17:18:21.588019] [INFO ] [REMOTE ] ASC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:21.750170] [DEBUG] [VALIDATOR ] ASAN: All expected trading days are present. [2025-10-20T17:18:21.750255] [DEBUG] [VALIDATOR ] ASAN: Primary data validation passed. [2025-10-20T17:18:21.750965] [INFO ] [MAIN ] ASAN: Fetched primary data is valid. [2025-10-20T17:18:21.751030] [DEBUG] [LOCAL ] ASAN: Saving primary data. [2025-10-20T17:18:21.752607] [DEBUG] [LOCAL ] ASAN: Trimmed 76635 records to 52732 (keeping 200 days of data) [2025-10-20T17:18:21.854714] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-01. [2025-10-20T17:18:21.975083] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:18:22.094252] [DEBUG] [LOCAL ] ASAN: Primary data saved (52732 records). [2025-10-20T17:18:22.094492] [INFO ] [MAIN ] ASAN: Full history saved. [2025-10-20T17:18:22.094583] [INFO ] [MAIN ] Update Progress: 154/2367 symbols completed. ASAN took 32.6s. Estimated remaining: 4h 22m 54s. Rate: 8.4 symbols/min [2025-10-20T17:18:22.263329] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T17:18:22.336748] [INFO ] [MAIN ] ASG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:22.354585] [DEBUG] [LOCAL ] ASG: Loaded 19812 existing records from primary storage. [2025-10-20T17:18:22.354867] [INFO ] [MAIN ] ASG: Loaded 19812 existing records. [2025-10-20T17:18:22.541146] [WARN ] [MAIN ] ASG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:22.541371] [INFO ] [MAIN ] ASG: Wiping existing primary data before full refresh. [2025-10-20T17:18:22.541420] [INFO ] [LOCAL ] ASG: Starting deletion of primary data. [2025-10-20T17:18:22.541686] [DEBUG] [LOCAL ] ASG: Deleted primary data file. [2025-10-20T17:18:22.541746] [INFO ] [MAIN ] ASG: Starting full primary refresh. [2025-10-20T17:18:22.656550] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-02. [2025-10-20T17:18:22.964842] [INFO ] [REMOTE ] ASG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:23.173088] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T17:18:23.451373] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-01. [2025-10-20T17:18:23.716023] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:18:24.251437] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-09. [2025-10-20T17:18:24.430542] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T17:18:25.051366] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-02. [2025-10-20T17:18:25.554155] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T17:18:25.851384] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-02. [2025-10-20T17:18:26.082742] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T17:18:26.651367] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-03. [2025-10-20T17:18:27.451413] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-10. [2025-10-20T17:18:27.500683] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:18:27.538375] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T17:18:28.251393] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-03. [2025-10-20T17:18:29.051427] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-03. [2025-10-20T17:18:29.177611] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:18:29.184974] [INFO ] [REMOTE ] ASB: Full fetch complete (71771 records). [2025-10-20T17:18:29.185115] [DEBUG] [VALIDATOR ] ASB: Validating primary data. [2025-10-20T17:18:29.185888] [FATAL] [VALIDATOR ] ASB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:18:29.185974] [FATAL] [MAIN ] ASB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:18:29.186018] [WARN ] [MAIN ] ASB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:18:29.186057] [INFO ] [LOCAL ] ASB: Starting deletion of primary data. [2025-10-20T17:18:29.186112] [DEBUG] [LOCAL ] ASB: No primary data file found to delete. [2025-10-20T17:18:29.186148] [INFO ] [LOCAL ] ASB: Starting deletion of derivative data. [2025-10-20T17:18:29.186188] [DEBUG] [LOCAL ] ASB: No derivative data file found to delete. [2025-10-20T17:18:29.186243] [INFO ] [MAIN ] Update Progress: 155/2367 symbols completed. ASB took 33.5s. Estimated remaining: 4h 22m 46s. Rate: 8.4 symbols/min [2025-10-20T17:18:29.362001] [INFO ] [MAIN ] ASGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:29.385215] [DEBUG] [LOCAL ] ASGI: Loaded 23311 existing records from primary storage. [2025-10-20T17:18:29.385494] [INFO ] [MAIN ] ASGI: Loaded 23311 existing records. [2025-10-20T17:18:29.576463] [WARN ] [MAIN ] ASGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:29.580945] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-10-20T17:18:29.582968] [INFO ] [MAIN ] ASGI: Wiping existing primary data before full refresh. [2025-10-20T17:18:29.583150] [INFO ] [LOCAL ] ASGI: Starting deletion of primary data. [2025-10-20T17:18:29.583352] [DEBUG] [LOCAL ] ASGI: Deleted primary data file. [2025-10-20T17:18:29.583392] [INFO ] [MAIN ] ASGI: Starting full primary refresh. [2025-10-20T17:18:29.860039] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-04. [2025-10-20T17:18:29.956926] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:18:29.997662] [INFO ] [REMOTE ] ASGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:30.307764] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T17:18:30.651401] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-05. [2025-10-20T17:18:31.126490] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T17:18:31.451370] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-06. [2025-10-20T17:18:32.021584] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T17:18:32.251373] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-04. [2025-10-20T17:18:33.051419] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-01. [2025-10-20T17:18:33.558986] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T17:18:33.851388] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-04. [2025-10-20T17:18:34.300443] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T17:18:34.651366] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-02. [2025-10-20T17:18:35.151795] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T17:18:35.371631] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T17:18:35.451332] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-03. [2025-10-20T17:18:36.251351] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-05. [2025-10-20T17:18:36.405571] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:18:37.051393] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-05. [2025-10-20T17:18:37.708506] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T17:18:37.851374] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-06. [2025-10-20T17:18:38.523915] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:18:38.651362] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-04. [2025-10-20T17:18:38.858988] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:18:39.454298] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-07. [2025-10-20T17:18:39.459090] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T17:18:40.251371] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-06. [2025-10-20T17:18:40.509394] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T17:18:41.051385] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-05. [2025-10-20T17:18:41.727007] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T17:18:41.851360] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-07. [2025-10-20T17:18:41.880022] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T17:18:42.503150] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T17:18:42.651361] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-08. [2025-10-20T17:18:43.061804] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-10-20T17:18:43.451416] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-08. [2025-10-20T17:18:44.251356] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-07. [2025-10-20T17:18:44.429481] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T17:18:45.051361] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-09. [2025-10-20T17:18:45.495722] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T17:18:45.851348] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-06. [2025-10-20T17:18:46.253334] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T17:18:46.651425] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-09. [2025-10-20T17:18:46.784186] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T17:18:47.451402] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-08. [2025-10-20T17:18:47.584096] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T17:18:48.251429] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-10. [2025-10-20T17:18:48.840031] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:18:49.051403] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-10. [2025-10-20T17:18:49.251913] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:18:49.254987] [INFO ] [REMOTE ] ASG: Full fetch complete (30907 records). [2025-10-20T17:18:49.255081] [DEBUG] [VALIDATOR ] ASG: Validating primary data. [2025-10-20T17:18:49.617435] [DEBUG] [VALIDATOR ] ASG: All expected trading days are present. [2025-10-20T17:18:49.617513] [DEBUG] [VALIDATOR ] ASG: Primary data validation passed. [2025-10-20T17:18:49.617917] [INFO ] [MAIN ] ASG: Fetched primary data is valid. [2025-10-20T17:18:49.617979] [DEBUG] [LOCAL ] ASG: Saving primary data. [2025-10-20T17:18:49.618718] [DEBUG] [LOCAL ] ASG: Trimmed 30907 records to 19504 (keeping 200 days of data) [2025-10-20T17:18:49.709326] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T17:18:49.711722] [INFO ] [REMOTE ] ASBA: Full fetch complete (5279 records). [2025-10-20T17:18:49.712296] [DEBUG] [VALIDATOR ] ASBA: Validating primary data. [2025-10-20T17:18:49.831967] [DEBUG] [LOCAL ] ASG: Primary data saved (19504 records). [2025-10-20T17:18:49.834537] [INFO ] [MAIN ] ASG: Full history saved. [2025-10-20T17:18:49.834628] [INFO ] [MAIN ] Update Progress: 156/2367 symbols completed. ASG took 27.7s. Estimated remaining: 4h 25m 51s. Rate: 8.3 symbols/min [2025-10-20T17:18:49.860304] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-07. [2025-10-20T17:18:50.206240] [INFO ] [MAIN ] ASGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:50.272887] [DEBUG] [VALIDATOR ] ASBA: All expected trading days are present. [2025-10-20T17:18:50.272968] [DEBUG] [VALIDATOR ] ASBA: Primary data validation passed. [2025-10-20T17:18:50.273121] [INFO ] [MAIN ] ASBA: Fetched primary data is valid. [2025-10-20T17:18:50.273159] [DEBUG] [LOCAL ] ASBA: Saving primary data. [2025-10-20T17:18:50.273516] [DEBUG] [LOCAL ] ASBA: Trimmed 5279 records to 3520 (keeping 200 days of data) [2025-10-20T17:18:50.306483] [DEBUG] [LOCAL ] ASBA: Primary data saved (3520 records). [2025-10-20T17:18:50.306585] [INFO ] [MAIN ] ASBA: Full history saved. [2025-10-20T17:18:50.306690] [INFO ] [MAIN ] Update Progress: 157/2367 symbols completed. ASBA took 31.1s. Estimated remaining: 4h 24m 9s. Rate: 8.4 symbols/min [2025-10-20T17:18:50.320832] [DEBUG] [LOCAL ] ASGN: Loaded 37201 existing records from primary storage. [2025-10-20T17:18:50.321111] [INFO ] [MAIN ] ASGN: Loaded 37201 existing records. [2025-10-20T17:18:50.517620] [WARN ] [MAIN ] ASGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:50.517940] [INFO ] [MAIN ] ASGN: Wiping existing primary data before full refresh. [2025-10-20T17:18:50.517994] [INFO ] [LOCAL ] ASGN: Starting deletion of primary data. [2025-10-20T17:18:50.518227] [DEBUG] [LOCAL ] ASGN: Deleted primary data file. [2025-10-20T17:18:50.518270] [INFO ] [MAIN ] ASGN: Starting full primary refresh. [2025-10-20T17:18:50.656562] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-09. [2025-10-20T17:18:50.955941] [INFO ] [REMOTE ] ASGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:51.024974] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:18:51.451385] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-08. [2025-10-20T17:18:52.120676] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:18:52.251370] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-01. [2025-10-20T17:18:52.562788] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:18:53.051428] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-09. [2025-10-20T17:18:53.776805] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T17:18:53.851360] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-02. [2025-10-20T17:18:54.161410] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T17:18:54.651394] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-10. [2025-10-20T17:18:55.250577] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T17:18:55.451239] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T17:18:55.451427] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-10. [2025-10-20T17:18:55.456075] [INFO ] [REMOTE ] ASGI: Full fetch complete (33240 records). [2025-10-20T17:18:55.456145] [DEBUG] [VALIDATOR ] ASGI: Validating primary data. [2025-10-20T17:18:55.830933] [DEBUG] [VALIDATOR ] ASGI: All expected trading days are present. [2025-10-20T17:18:55.831056] [DEBUG] [VALIDATOR ] ASGI: Primary data validation passed. [2025-10-20T17:18:55.831493] [INFO ] [MAIN ] ASGI: Fetched primary data is valid. [2025-10-20T17:18:55.831535] [DEBUG] [LOCAL ] ASGI: Saving primary data. [2025-10-20T17:18:55.832341] [DEBUG] [LOCAL ] ASGI: Trimmed 33240 records to 23006 (keeping 200 days of data) [2025-10-20T17:18:55.968375] [DEBUG] [LOCAL ] ASGI: Primary data saved (23006 records). [2025-10-20T17:18:55.968549] [INFO ] [MAIN ] ASGI: Full history saved. [2025-10-20T17:18:55.968631] [INFO ] [MAIN ] Update Progress: 158/2367 symbols completed. ASGI took 26.8s. Estimated remaining: 4h 23m 40s. Rate: 8.4 symbols/min [2025-10-20T17:18:56.251463] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-03. [2025-10-20T17:18:57.089065] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T17:18:57.094378] [INFO ] [REMOTE ] ASC: Full fetch complete (55853 records). [2025-10-20T17:18:57.094480] [DEBUG] [VALIDATOR ] ASC: Validating primary data. [2025-10-20T17:18:57.095235] [FATAL] [VALIDATOR ] ASC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:18:57.095326] [FATAL] [MAIN ] ASC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:18:57.095394] [WARN ] [MAIN ] ASC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:18:57.095474] [INFO ] [LOCAL ] ASC: Starting deletion of primary data. [2025-10-20T17:18:57.095531] [DEBUG] [LOCAL ] ASC: No primary data file found to delete. [2025-10-20T17:18:57.095569] [INFO ] [LOCAL ] ASC: Starting deletion of derivative data. [2025-10-20T17:18:57.095739] [DEBUG] [LOCAL ] ASC: Deleted derivative data file. [2025-10-20T17:18:57.096340] [INFO ] [MAIN ] Update Progress: 159/2367 symbols completed. ASC took 36.6s. Estimated remaining: 4h 22m 9s. Rate: 8.4 symbols/min [2025-10-20T17:18:57.753347] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T17:18:57.761349] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-04. [2025-10-20T17:18:58.865237] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T17:18:58.877781] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-05. [2025-10-20T17:19:00.294964] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T17:19:00.302078] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-06. [2025-10-20T17:19:01.994602] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:19:02.002441] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-07. [2025-10-20T17:19:03.871625] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T17:19:03.881138] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-08. [2025-10-20T17:19:06.577323] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-08 (2.70 seconds). [2025-10-20T17:19:06.585703] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-09. [2025-10-20T17:19:08.265102] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:19:08.273997] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-10. [2025-10-20T17:19:09.889024] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T17:19:09.895211] [INFO ] [REMOTE ] ASGN: Full fetch complete (53460 records). [2025-10-20T17:19:09.895312] [DEBUG] [VALIDATOR ] ASGN: Validating primary data. [2025-10-20T17:19:10.293148] [DEBUG] [VALIDATOR ] ASGN: All expected trading days are present. [2025-10-20T17:19:10.293246] [DEBUG] [VALIDATOR ] ASGN: Primary data validation passed. [2025-10-20T17:19:10.293863] [INFO ] [MAIN ] ASGN: Fetched primary data is valid. [2025-10-20T17:19:10.293925] [DEBUG] [LOCAL ] ASGN: Saving primary data. [2025-10-20T17:19:10.295181] [DEBUG] [LOCAL ] ASGN: Trimmed 53460 records to 36724 (keeping 200 days of data) [2025-10-20T17:19:10.503123] [DEBUG] [LOCAL ] ASGN: Primary data saved (36724 records). [2025-10-20T17:19:10.503260] [INFO ] [MAIN ] ASGN: Full history saved. [2025-10-20T17:19:10.503341] [INFO ] [MAIN ] Update Progress: 160/2367 symbols completed. ASGN took 20.7s. Estimated remaining: 4h 23m 29s. Rate: 8.4 symbols/min [2025-10-20T17:19:11.127178] [INFO ] [MAIN ] ASR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:11.206121] [INFO ] [MAIN ] ASH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:11.209487] [INFO ] [MAIN ] ASPN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:11.233857] [INFO ] [MAIN ] ASIX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:11.244419] [DEBUG] [LOCAL ] ASR: Loaded 9527 existing records from primary storage. [2025-10-20T17:19:11.244851] [INFO ] [MAIN ] ASR: Loaded 9527 existing records. [2025-10-20T17:19:11.417133] [DEBUG] [LOCAL ] ASIX: Loaded 23435 existing records from primary storage. [2025-10-20T17:19:11.447091] [INFO ] [MAIN ] ASIX: Loaded 23435 existing records. [2025-10-20T17:19:11.492504] [WARN ] [MAIN ] ASR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:11.522997] [INFO ] [MAIN ] ASR: Wiping existing primary data before full refresh. [2025-10-20T17:19:11.556247] [DEBUG] [LOCAL ] ASH: Loaded 38148 existing records from primary storage. [2025-10-20T17:19:11.550459] [INFO ] [LOCAL ] ASR: Starting deletion of primary data. [2025-10-20T17:19:11.561764] [DEBUG] [LOCAL ] ASR: Deleted primary data file. [2025-10-20T17:19:11.579575] [INFO ] [MAIN ] ASR: Starting full primary refresh. [2025-10-20T17:19:11.620688] [INFO ] [MAIN ] ASH: Loaded 38148 existing records. [2025-10-20T17:19:11.727926] [DEBUG] [LOCAL ] ASPN: Loaded 50648 existing records from primary storage. [2025-10-20T17:19:11.862029] [INFO ] [MAIN ] ASPN: Loaded 50648 existing records. [2025-10-20T17:19:12.035289] [WARN ] [MAIN ] ASIX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:12.134912] [WARN ] [MAIN ] ASH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:12.142002] [INFO ] [MAIN ] ASIX: Wiping existing primary data before full refresh. [2025-10-20T17:19:12.194484] [INFO ] [LOCAL ] ASIX: Starting deletion of primary data. [2025-10-20T17:19:12.199720] [INFO ] [MAIN ] ASH: Wiping existing primary data before full refresh. [2025-10-20T17:19:12.257732] [INFO ] [LOCAL ] ASH: Starting deletion of primary data. [2025-10-20T17:19:12.278475] [DEBUG] [LOCAL ] ASIX: Deleted primary data file. [2025-10-20T17:19:12.288947] [INFO ] [MAIN ] ASIX: Starting full primary refresh. [2025-10-20T17:19:12.397266] [DEBUG] [LOCAL ] ASH: Deleted primary data file. [2025-10-20T17:19:12.413144] [INFO ] [MAIN ] ASH: Starting full primary refresh. [2025-10-20T17:19:12.391960] [WARN ] [MAIN ] ASPN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:12.511561] [INFO ] [MAIN ] ASPN: Wiping existing primary data before full refresh. [2025-10-20T17:19:12.650491] [INFO ] [LOCAL ] ASPN: Starting deletion of primary data. [2025-10-20T17:19:12.659541] [INFO ] [REMOTE ] ASR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:12.699669] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-01. [2025-10-20T17:19:12.699752] [DEBUG] [LOCAL ] ASPN: Deleted primary data file. [2025-10-20T17:19:12.743822] [INFO ] [MAIN ] ASPN: Starting full primary refresh. [2025-10-20T17:19:13.424722] [INFO ] [REMOTE ] ASIX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:13.444115] [INFO ] [REMOTE ] ASH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:13.468927] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-01. [2025-10-20T17:19:13.495331] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-01. [2025-10-20T17:19:13.599454] [INFO ] [REMOTE ] ASPN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:13.602206] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-01. [2025-10-20T17:19:13.605764] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:19:13.607766] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-02. [2025-10-20T17:19:14.260332] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T17:19:14.264345] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-02. [2025-10-20T17:19:14.351464] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T17:19:14.353750] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-03. [2025-10-20T17:19:14.725010] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T17:19:14.733326] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-02. [2025-10-20T17:19:14.889596] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T17:19:14.891876] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-04. [2025-10-20T17:19:15.210800] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T17:19:15.214395] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-03. [2025-10-20T17:19:15.454939] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T17:19:15.458591] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-05. [2025-10-20T17:19:15.501896] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T17:19:15.512683] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-02. [2025-10-20T17:19:15.849435] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:19:15.857397] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-03. [2025-10-20T17:19:15.872609] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T17:19:15.877525] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-04. [2025-10-20T17:19:15.946345] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T17:19:15.950128] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-06. [2025-10-20T17:19:16.736940] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:19:16.744351] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-05. [2025-10-20T17:19:16.784134] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T17:19:16.787578] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-07. [2025-10-20T17:19:17.052402] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T17:19:17.062142] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-04. [2025-10-20T17:19:17.265660] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T17:19:17.275631] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-03. [2025-10-20T17:19:17.387988] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T17:19:17.552819] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T17:19:17.851376] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-06. [2025-10-20T17:19:18.459590] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:19:18.493075] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:19:18.657741] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-05. [2025-10-20T17:19:18.663377] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:19:19.451387] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-04. [2025-10-20T17:19:19.946082] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T17:19:20.251370] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-08. [2025-10-20T17:19:21.045281] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T17:19:21.051320] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-09. [2025-10-20T17:19:21.173635] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T17:19:21.851407] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-06. [2025-10-20T17:19:21.883619] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T17:19:22.651376] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-07. [2025-10-20T17:19:22.844874] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T17:19:23.451342] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-10. [2025-10-20T17:19:23.855412] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T17:19:24.251371] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-05. [2025-10-20T17:19:24.424346] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T17:19:24.425873] [INFO ] [REMOTE ] ASR: Full fetch complete (12384 records). [2025-10-20T17:19:24.425942] [DEBUG] [VALIDATOR ] ASR: Validating primary data. [2025-10-20T17:19:24.426323] [FATAL] [VALIDATOR ] ASR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:19:24.426405] [FATAL] [MAIN ] ASR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:19:24.426448] [WARN ] [MAIN ] ASR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:19:24.426486] [INFO ] [LOCAL ] ASR: Starting deletion of primary data. [2025-10-20T17:19:24.426538] [DEBUG] [LOCAL ] ASR: No primary data file found to delete. [2025-10-20T17:19:24.426580] [INFO ] [LOCAL ] ASR: Starting deletion of derivative data. [2025-10-20T17:19:24.426769] [DEBUG] [LOCAL ] ASR: Deleted derivative data file. [2025-10-20T17:19:24.426852] [INFO ] [MAIN ] Update Progress: 161/2367 symbols completed. ASR took 13.9s. Estimated remaining: 4h 24m 54s. Rate: 8.3 symbols/min [2025-10-20T17:19:24.594060] [INFO ] [MAIN ] ASX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:24.642752] [DEBUG] [LOCAL ] ASX: Loaded 53653 existing records from primary storage. [2025-10-20T17:19:24.643057] [INFO ] [MAIN ] ASX: Loaded 53653 existing records. [2025-10-20T17:19:24.822689] [WARN ] [MAIN ] ASX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:24.822918] [INFO ] [MAIN ] ASX: Wiping existing primary data before full refresh. [2025-10-20T17:19:24.822977] [INFO ] [LOCAL ] ASX: Starting deletion of primary data. [2025-10-20T17:19:24.823210] [DEBUG] [LOCAL ] ASX: Deleted primary data file. [2025-10-20T17:19:24.823267] [INFO ] [MAIN ] ASX: Starting full primary refresh. [2025-10-20T17:19:25.062563] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-07. [2025-10-20T17:19:25.179127] [INFO ] [REMOTE ] ASX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:25.851419] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-08. [2025-10-20T17:19:25.994362] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:19:26.599013] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:19:26.651420] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-06. [2025-10-20T17:19:27.451371] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-08. [2025-10-20T17:19:27.468518] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T17:19:28.251387] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-01. [2025-10-20T17:19:28.634779] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-10-20T17:19:28.894318] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T17:19:29.051369] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-09. [2025-10-20T17:19:29.856873] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-07. [2025-10-20T17:19:29.875510] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T17:19:30.414995] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T17:19:30.651370] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-09. [2025-10-20T17:19:31.357923] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:19:31.451396] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-08. [2025-10-20T17:19:31.705895] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T17:19:32.251375] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-10. [2025-10-20T17:19:32.848065] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:19:33.051332] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-09. [2025-10-20T17:19:33.113605] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T17:19:33.117408] [INFO ] [REMOTE ] ASIX: Full fetch complete (31091 records). [2025-10-20T17:19:33.117477] [DEBUG] [VALIDATOR ] ASIX: Validating primary data. [2025-10-20T17:19:33.477755] [DEBUG] [VALIDATOR ] ASIX: All expected trading days are present. [2025-10-20T17:19:33.477830] [DEBUG] [VALIDATOR ] ASIX: Primary data validation passed. [2025-10-20T17:19:33.478216] [INFO ] [MAIN ] ASIX: Fetched primary data is valid. [2025-10-20T17:19:33.478254] [DEBUG] [LOCAL ] ASIX: Saving primary data. [2025-10-20T17:19:33.479016] [DEBUG] [LOCAL ] ASIX: Trimmed 31091 records to 23205 (keeping 200 days of data) [2025-10-20T17:19:33.610832] [DEBUG] [LOCAL ] ASIX: Primary data saved (23205 records). [2025-10-20T17:19:33.610961] [INFO ] [MAIN ] ASIX: Full history saved. [2025-10-20T17:19:33.611520] [INFO ] [MAIN ] Update Progress: 162/2367 symbols completed. ASIX took 23.1s. Estimated remaining: 4h 25m 14s. Rate: 8.3 symbols/min [2025-10-20T17:19:33.781162] [INFO ] [MAIN ] ATEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:33.817993] [DEBUG] [LOCAL ] ATEN: Loaded 38815 existing records from primary storage. [2025-10-20T17:19:33.818309] [INFO ] [MAIN ] ATEN: Loaded 38815 existing records. [2025-10-20T17:19:33.860303] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-02. [2025-10-20T17:19:33.997452] [WARN ] [MAIN ] ATEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:33.997693] [INFO ] [MAIN ] ATEN: Wiping existing primary data before full refresh. [2025-10-20T17:19:33.997749] [INFO ] [LOCAL ] ATEN: Starting deletion of primary data. [2025-10-20T17:19:33.997995] [DEBUG] [LOCAL ] ATEN: Deleted primary data file. [2025-10-20T17:19:33.998033] [INFO ] [MAIN ] ATEN: Starting full primary refresh. [2025-10-20T17:19:34.344449] [INFO ] [REMOTE ] ATEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:34.651348] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-10. [2025-10-20T17:19:34.830256] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:19:35.460468] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-10. [2025-10-20T17:19:35.467728] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T17:19:36.150004] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T17:19:36.155919] [INFO ] [REMOTE ] ASH: Full fetch complete (55741 records). [2025-10-20T17:19:36.156019] [DEBUG] [VALIDATOR ] ASH: Validating primary data. [2025-10-20T17:19:36.256470] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-01. [2025-10-20T17:19:36.546668] [DEBUG] [VALIDATOR ] ASH: All expected trading days are present. [2025-10-20T17:19:36.546754] [DEBUG] [VALIDATOR ] ASH: Primary data validation passed. [2025-10-20T17:19:36.547321] [INFO ] [MAIN ] ASH: Fetched primary data is valid. [2025-10-20T17:19:36.547360] [DEBUG] [LOCAL ] ASH: Saving primary data. [2025-10-20T17:19:36.548460] [DEBUG] [LOCAL ] ASH: Trimmed 55741 records to 37563 (keeping 200 days of data) [2025-10-20T17:19:36.772757] [DEBUG] [LOCAL ] ASH: Primary data saved (37563 records). [2025-10-20T17:19:36.772904] [INFO ] [MAIN ] ASH: Full history saved. [2025-10-20T17:19:36.772982] [INFO ] [MAIN ] Update Progress: 163/2367 symbols completed. ASH took 26.3s. Estimated remaining: 4h 24m 12s. Rate: 8.3 symbols/min [2025-10-20T17:19:36.943000] [INFO ] [MAIN ] ATGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:36.975349] [DEBUG] [LOCAL ] ATGE: Loaded 31377 existing records from primary storage. [2025-10-20T17:19:36.975735] [INFO ] [MAIN ] ATGE: Loaded 31377 existing records. [2025-10-20T17:19:37.052639] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-03. [2025-10-20T17:19:37.168103] [WARN ] [MAIN ] ATGE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:37.191616] [INFO ] [MAIN ] ATGE: Wiping existing primary data before full refresh. [2025-10-20T17:19:37.191782] [INFO ] [LOCAL ] ATGE: Starting deletion of primary data. [2025-10-20T17:19:37.193305] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:19:37.193353] [DEBUG] [LOCAL ] ATGE: Deleted primary data file. [2025-10-20T17:19:37.200905] [INFO ] [MAIN ] ATGE: Starting full primary refresh. [2025-10-20T17:19:37.206477] [INFO ] [REMOTE ] ASPN: Full fetch complete (72615 records). [2025-10-20T17:19:37.217621] [DEBUG] [VALIDATOR ] ASPN: Validating primary data. [2025-10-20T17:19:38.026864] [DEBUG] [VALIDATOR ] ASPN: All expected trading days are present. [2025-10-20T17:19:38.050832] [DEBUG] [VALIDATOR ] ASPN: Primary data validation passed. [2025-10-20T17:19:38.051666] [INFO ] [MAIN ] ASPN: Fetched primary data is valid. [2025-10-20T17:19:38.050572] [INFO ] [REMOTE ] ATGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:38.051926] [DEBUG] [LOCAL ] ASPN: Saving primary data. [2025-10-20T17:19:38.052345] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-01. [2025-10-20T17:19:38.076796] [DEBUG] [LOCAL ] ASPN: Trimmed 72615 records to 49915 (keeping 200 days of data) [2025-10-20T17:19:38.084970] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:19:38.371492] [DEBUG] [LOCAL ] ASPN: Primary data saved (49915 records). [2025-10-20T17:19:38.371675] [INFO ] [MAIN ] ASPN: Full history saved. [2025-10-20T17:19:38.371775] [INFO ] [MAIN ] Update Progress: 164/2367 symbols completed. ASPN took 27.9s. Estimated remaining: 4h 22m 50s. Rate: 8.4 symbols/min [2025-10-20T17:19:38.550061] [INFO ] [MAIN ] ATHM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:38.550326] [DEBUG] [LOCAL ] ATHM: No existing data file found in primary storage. [2025-10-20T17:19:38.550476] [WARN ] [MAIN ] ATHM: No existing data found, performing full refresh instead. [2025-10-20T17:19:38.550888] [INFO ] [MAIN ] ATHM: Wiping existing primary data before full refresh. [2025-10-20T17:19:38.551050] [INFO ] [LOCAL ] ATHM: Starting deletion of primary data. [2025-10-20T17:19:38.551202] [DEBUG] [LOCAL ] ATHM: No primary data file found to delete. [2025-10-20T17:19:38.551268] [INFO ] [MAIN ] ATHM: Starting full primary refresh. [2025-10-20T17:19:38.656678] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-02. [2025-10-20T17:19:38.905726] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T17:19:38.990356] [INFO ] [REMOTE ] ATHM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:39.451373] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-04. [2025-10-20T17:19:39.626227] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:19:40.180953] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:19:40.251381] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-02. [2025-10-20T17:19:41.003437] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:19:41.051404] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-01. [2025-10-20T17:19:41.637335] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T17:19:41.851366] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-03. [2025-10-20T17:19:42.613279] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:19:42.651367] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-02. [2025-10-20T17:19:43.272339] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:19:43.451427] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-05. [2025-10-20T17:19:43.751245] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:19:44.251443] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-03. [2025-10-20T17:19:45.051355] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-03. [2025-10-20T17:19:45.552450] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-05 (2.10 seconds). [2025-10-20T17:19:45.621583] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:19:45.851422] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-04. [2025-10-20T17:19:46.538818] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T17:19:46.651382] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-04. [2025-10-20T17:19:47.013748] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T17:19:47.451456] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-04. [2025-10-20T17:19:47.869574] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T17:19:48.251460] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-05. [2025-10-20T17:19:49.029357] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:19:49.051334] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-05. [2025-10-20T17:19:49.721701] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:19:49.851371] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-05. [2025-10-20T17:19:50.519934] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:19:50.651369] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-06. [2025-10-20T17:19:51.385305] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T17:19:51.451358] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-06. [2025-10-20T17:19:51.867334] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T17:19:52.251410] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-07. [2025-10-20T17:19:52.402933] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T17:19:53.051364] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-06. [2025-10-20T17:19:53.575245] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T17:19:53.851367] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-06. [2025-10-20T17:19:54.651474] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-07. [2025-10-20T17:19:54.899017] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T17:19:55.255090] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T17:19:55.451384] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-07. [2025-10-20T17:19:55.978849] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T17:19:56.251413] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-08. [2025-10-20T17:19:56.893831] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:19:57.051351] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-08. [2025-10-20T17:19:57.584701] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T17:19:57.851398] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-08. [2025-10-20T17:19:58.429380] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T17:19:58.651428] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-07. [2025-10-20T17:19:59.337392] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T17:19:59.451385] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-09. [2025-10-20T17:20:00.251373] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-09. [2025-10-20T17:20:00.538469] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T17:20:00.612154] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T17:20:00.967542] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T17:20:01.051369] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-09. [2025-10-20T17:20:01.851379] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-08. [2025-10-20T17:20:02.545371] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T17:20:02.651406] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-10. [2025-10-20T17:20:03.451444] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-10. [2025-10-20T17:20:03.622550] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T17:20:04.110966] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T17:20:04.117142] [INFO ] [REMOTE ] ATEN: Full fetch complete (56825 records). [2025-10-20T17:20:04.117277] [DEBUG] [VALIDATOR ] ATEN: Validating primary data. [2025-10-20T17:20:04.256618] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-10. [2025-10-20T17:20:04.517594] [DEBUG] [VALIDATOR ] ATEN: All expected trading days are present. [2025-10-20T17:20:04.518743] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T17:20:04.518172] [DEBUG] [VALIDATOR ] ATEN: Primary data validation passed. [2025-10-20T17:20:04.523469] [INFO ] [MAIN ] ATEN: Fetched primary data is valid. [2025-10-20T17:20:04.523869] [DEBUG] [LOCAL ] ATEN: Saving primary data. [2025-10-20T17:20:04.524390] [INFO ] [REMOTE ] ATHM: Full fetch complete (49799 records). [2025-10-20T17:20:04.524710] [DEBUG] [VALIDATOR ] ATHM: Validating primary data. [2025-10-20T17:20:04.584784] [DEBUG] [LOCAL ] ATEN: Trimmed 56825 records to 38022 (keeping 200 days of data) [2025-10-20T17:20:05.077847] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-09. [2025-10-20T17:20:05.185081] [DEBUG] [VALIDATOR ] ATHM: All expected trading days are present. [2025-10-20T17:20:05.193922] [DEBUG] [VALIDATOR ] ATHM: Primary data validation passed. [2025-10-20T17:20:05.226982] [INFO ] [MAIN ] ATHM: Fetched primary data is valid. [2025-10-20T17:20:05.238354] [DEBUG] [LOCAL ] ATHM: Saving primary data. [2025-10-20T17:20:05.253330] [DEBUG] [LOCAL ] ATHM: Trimmed 49799 records to 31516 (keeping 200 days of data) [2025-10-20T17:20:05.256830] [DEBUG] [LOCAL ] ATEN: Primary data saved (38022 records). [2025-10-20T17:20:05.257153] [INFO ] [MAIN ] ATEN: Full history saved. [2025-10-20T17:20:05.257400] [INFO ] [MAIN ] Update Progress: 165/2367 symbols completed. ATEN took 31.6s. Estimated remaining: 4h 27m 6s. Rate: 8.2 symbols/min [2025-10-20T17:20:05.599199] [INFO ] [MAIN ] ATHS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:05.629141] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:20:05.635639] [INFO ] [REMOTE ] ATGE: Full fetch complete (45740 records). [2025-10-20T17:20:05.636580] [DEBUG] [VALIDATOR ] ATGE: Validating primary data. [2025-10-20T17:20:05.638133] [FATAL] [VALIDATOR ] ATGE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:20:05.641187] [FATAL] [MAIN ] ATGE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:20:05.642396] [WARN ] [MAIN ] ATGE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:20:05.648965] [INFO ] [LOCAL ] ATGE: Starting deletion of primary data. [2025-10-20T17:20:05.655686] [DEBUG] [LOCAL ] ATHS: Loaded 6713 existing records from primary storage. [2025-10-20T17:20:05.657615] [DEBUG] [LOCAL ] ATGE: No primary data file found to delete. [2025-10-20T17:20:05.662143] [INFO ] [MAIN ] ATHS: Loaded 6713 existing records. [2025-10-20T17:20:05.657897] [INFO ] [LOCAL ] ATGE: Starting deletion of derivative data. [2025-10-20T17:20:05.674036] [DEBUG] [LOCAL ] ATHM: Primary data saved (31516 records). [2025-10-20T17:20:05.690604] [INFO ] [MAIN ] ATHM: Full history saved. [2025-10-20T17:20:05.713076] [INFO ] [MAIN ] Update Progress: 166/2367 symbols completed. ATHM took 27.3s. Estimated remaining: 4h 25m 28s. Rate: 8.3 symbols/min [2025-10-20T17:20:05.706275] [DEBUG] [LOCAL ] ATGE: Deleted derivative data file. [2025-10-20T17:20:05.758956] [INFO ] [MAIN ] Update Progress: 167/2367 symbols completed. ATGE took 29.0s. Estimated remaining: 4h 23m 46s. Rate: 8.3 symbols/min [2025-10-20T17:20:06.089866] [WARN ] [MAIN ] ATHS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:06.109299] [INFO ] [MAIN ] ATHS: Wiping existing primary data before full refresh. [2025-10-20T17:20:06.119778] [INFO ] [LOCAL ] ATHS: Starting deletion of primary data. [2025-10-20T17:20:06.125484] [DEBUG] [LOCAL ] ATHS: Deleted primary data file. [2025-10-20T17:20:06.136009] [INFO ] [MAIN ] ATHS: Starting full primary refresh. [2025-10-20T17:20:06.227417] [INFO ] [MAIN ] ATI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:06.347923] [INFO ] [MAIN ] ATKR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:06.636451] [DEBUG] [LOCAL ] ATI: Loaded 52088 existing records from primary storage. [2025-10-20T17:20:06.652094] [INFO ] [MAIN ] ATI: Loaded 52088 existing records. [2025-10-20T17:20:06.706829] [DEBUG] [LOCAL ] ATKR: Loaded 35684 existing records from primary storage. [2025-10-20T17:20:06.782743] [INFO ] [MAIN ] ATKR: Loaded 35684 existing records. [2025-10-20T17:20:06.937570] [INFO ] [REMOTE ] ATHS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:07.018020] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-01. [2025-10-20T17:20:07.100147] [WARN ] [MAIN ] ATI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:07.124319] [INFO ] [MAIN ] ATI: Wiping existing primary data before full refresh. [2025-10-20T17:20:07.147341] [WARN ] [MAIN ] ATKR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:07.147477] [INFO ] [LOCAL ] ATI: Starting deletion of primary data. [2025-10-20T17:20:07.148164] [INFO ] [MAIN ] ATKR: Wiping existing primary data before full refresh. [2025-10-20T17:20:07.148333] [INFO ] [LOCAL ] ATKR: Starting deletion of primary data. [2025-10-20T17:20:07.148457] [DEBUG] [LOCAL ] ATI: Deleted primary data file. [2025-10-20T17:20:07.148610] [INFO ] [MAIN ] ATI: Starting full primary refresh. [2025-10-20T17:20:07.161149] [DEBUG] [LOCAL ] ATKR: Deleted primary data file. [2025-10-20T17:20:07.161372] [INFO ] [MAIN ] ATKR: Starting full primary refresh. [2025-10-20T17:20:07.566354] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-09 (2.49 seconds). [2025-10-20T17:20:07.717267] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-10. [2025-10-20T17:20:07.868505] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T17:20:07.905336] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-02. [2025-10-20T17:20:07.977284] [INFO ] [REMOTE ] ATKR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:08.020721] [INFO ] [REMOTE ] ATI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:08.251435] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-01. [2025-10-20T17:20:08.491908] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T17:20:09.051416] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-03. [2025-10-20T17:20:09.580830] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T17:20:09.844006] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T17:20:09.851336] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-04. [2025-10-20T17:20:10.184903] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-10 (2.47 seconds). [2025-10-20T17:20:10.192841] [INFO ] [REMOTE ] ASX: Full fetch complete (76790 records). [2025-10-20T17:20:10.192990] [DEBUG] [VALIDATOR ] ASX: Validating primary data. [2025-10-20T17:20:10.499135] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:20:10.597121] [DEBUG] [VALIDATOR ] ASX: All expected trading days are present. [2025-10-20T17:20:10.597204] [DEBUG] [VALIDATOR ] ASX: Primary data validation passed. [2025-10-20T17:20:10.597964] [INFO ] [MAIN ] ASX: Fetched primary data is valid. [2025-10-20T17:20:10.598029] [DEBUG] [LOCAL ] ASX: Saving primary data. [2025-10-20T17:20:10.599872] [DEBUG] [LOCAL ] ASX: Trimmed 76790 records to 52882 (keeping 200 days of data) [2025-10-20T17:20:10.658739] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-02. [2025-10-20T17:20:10.907892] [DEBUG] [LOCAL ] ASX: Primary data saved (52882 records). [2025-10-20T17:20:10.908029] [INFO ] [MAIN ] ASX: Full history saved. [2025-10-20T17:20:10.908108] [INFO ] [MAIN ] Update Progress: 168/2367 symbols completed. ASX took 46.5s. Estimated remaining: 4h 23m 12s. Rate: 8.4 symbols/min [2025-10-20T17:20:11.074502] [INFO ] [MAIN ] ATMU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:11.108585] [DEBUG] [LOCAL ] ATMU: Loaded 39475 existing records from primary storage. [2025-10-20T17:20:11.108904] [INFO ] [MAIN ] ATMU: Loaded 39475 existing records. [2025-10-20T17:20:11.287073] [WARN ] [MAIN ] ATMU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:11.287299] [INFO ] [MAIN ] ATMU: Wiping existing primary data before full refresh. [2025-10-20T17:20:11.287345] [INFO ] [LOCAL ] ATMU: Starting deletion of primary data. [2025-10-20T17:20:11.287590] [DEBUG] [LOCAL ] ATMU: Deleted primary data file. [2025-10-20T17:20:11.287680] [INFO ] [MAIN ] ATMU: Starting full primary refresh. [2025-10-20T17:20:11.460322] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-01. [2025-10-20T17:20:11.641600] [INFO ] [REMOTE ] ATMU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:12.251391] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-01. [2025-10-20T17:20:12.316841] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:20:13.051387] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-05. [2025-10-20T17:20:13.650732] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T17:20:13.851462] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-06. [2025-10-20T17:20:14.485600] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T17:20:14.574099] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-01 (3.11 seconds). [2025-10-20T17:20:14.651482] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-07. [2025-10-20T17:20:14.939857] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-01 (2.69 seconds). [2025-10-20T17:20:15.356202] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T17:20:15.451396] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-03. [2025-10-20T17:20:16.251397] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-08. [2025-10-20T17:20:16.722219] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-10-20T17:20:16.811952] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T17:20:17.051477] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-02. [2025-10-20T17:20:17.851416] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-02. [2025-10-20T17:20:18.467611] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:20:18.651371] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-03. [2025-10-20T17:20:19.449371] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:20:19.457668] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-09. [2025-10-20T17:20:19.929012] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T17:20:20.170853] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:20:20.251404] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-10. [2025-10-20T17:20:20.929466] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T17:20:20.930879] [INFO ] [REMOTE ] ATHS: Full fetch complete (9606 records). [2025-10-20T17:20:20.930970] [DEBUG] [VALIDATOR ] ATHS: Validating primary data. [2025-10-20T17:20:21.056512] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-03. [2025-10-20T17:20:21.327832] [DEBUG] [VALIDATOR ] ATHS: All expected trading days are present. [2025-10-20T17:20:21.327913] [DEBUG] [VALIDATOR ] ATHS: Primary data validation passed. [2025-10-20T17:20:21.328147] [INFO ] [MAIN ] ATHS: Fetched primary data is valid. [2025-10-20T17:20:21.328210] [DEBUG] [LOCAL ] ATHS: Saving primary data. [2025-10-20T17:20:21.328701] [DEBUG] [LOCAL ] ATHS: Trimmed 9606 records to 6572 (keeping 200 days of data) [2025-10-20T17:20:21.367354] [DEBUG] [LOCAL ] ATHS: Primary data saved (6572 records). [2025-10-20T17:20:21.367480] [INFO ] [MAIN ] ATHS: Full history saved. [2025-10-20T17:20:21.367559] [INFO ] [MAIN ] Update Progress: 169/2367 symbols completed. ATHS took 16.1s. Estimated remaining: 4h 23m 48s. Rate: 8.3 symbols/min [2025-10-20T17:20:21.536406] [INFO ] [MAIN ] ATO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:21.577300] [DEBUG] [LOCAL ] ATO: Loaded 42359 existing records from primary storage. [2025-10-20T17:20:21.577596] [INFO ] [MAIN ] ATO: Loaded 42359 existing records. [2025-10-20T17:20:21.765013] [WARN ] [MAIN ] ATO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:21.765230] [INFO ] [MAIN ] ATO: Wiping existing primary data before full refresh. [2025-10-20T17:20:21.765277] [INFO ] [LOCAL ] ATO: Starting deletion of primary data. [2025-10-20T17:20:21.765620] [DEBUG] [LOCAL ] ATO: Deleted primary data file. [2025-10-20T17:20:21.765715] [INFO ] [MAIN ] ATO: Starting full primary refresh. [2025-10-20T17:20:21.856457] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-04. [2025-10-20T17:20:22.123621] [INFO ] [REMOTE ] ATO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:22.656830] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-04. [2025-10-20T17:20:22.659470] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:20:23.415784] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T17:20:23.451367] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-01. [2025-10-20T17:20:24.038799] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:20:24.251419] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-04. [2025-10-20T17:20:25.051379] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-05. [2025-10-20T17:20:25.122672] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T17:20:25.755700] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T17:20:25.851382] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-02. [2025-10-20T17:20:26.651414] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-05. [2025-10-20T17:20:26.816852] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:20:27.278918] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T17:20:27.451388] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-05. [2025-10-20T17:20:27.765436] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T17:20:28.251389] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-06. [2025-10-20T17:20:28.797221] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T17:20:29.051412] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-06. [2025-10-20T17:20:29.860913] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-03. [2025-10-20T17:20:29.867597] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:20:30.321921] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T17:20:30.651385] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-06. [2025-10-20T17:20:31.451370] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-07. [2025-10-20T17:20:31.529396] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:20:31.610100] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T17:20:32.251422] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-07. [2025-10-20T17:20:32.886150] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T17:20:33.051375] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-08. [2025-10-20T17:20:33.734051] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:20:33.851396] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-07. [2025-10-20T17:20:34.627549] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:20:34.651380] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-08. [2025-10-20T17:20:35.331849] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:20:35.451374] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-08. [2025-10-20T17:20:36.074624] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T17:20:36.251369] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-09. [2025-10-20T17:20:37.056501] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-09. [2025-10-20T17:20:37.070870] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T17:20:37.851386] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-09. [2025-10-20T17:20:37.988587] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:20:38.292077] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T17:20:38.651369] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-04. [2025-10-20T17:20:39.285061] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:20:39.451374] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-10. [2025-10-20T17:20:40.098893] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T17:20:40.251409] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-10. [2025-10-20T17:20:41.027299] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:20:41.033287] [INFO ] [REMOTE ] ATMU: Full fetch complete (56862 records). [2025-10-20T17:20:41.033442] [DEBUG] [VALIDATOR ] ATMU: Validating primary data. [2025-10-20T17:20:41.056544] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-10. [2025-10-20T17:20:41.450390] [DEBUG] [VALIDATOR ] ATMU: All expected trading days are present. [2025-10-20T17:20:41.450486] [DEBUG] [VALIDATOR ] ATMU: Primary data validation passed. [2025-10-20T17:20:41.451197] [INFO ] [MAIN ] ATMU: Fetched primary data is valid. [2025-10-20T17:20:41.451265] [DEBUG] [LOCAL ] ATMU: Saving primary data. [2025-10-20T17:20:41.452571] [DEBUG] [LOCAL ] ATMU: Trimmed 56862 records to 38808 (keeping 200 days of data) [2025-10-20T17:20:41.693339] [DEBUG] [LOCAL ] ATMU: Primary data saved (38808 records). [2025-10-20T17:20:41.693511] [INFO ] [MAIN ] ATMU: Full history saved. [2025-10-20T17:20:41.693603] [INFO ] [MAIN ] Update Progress: 170/2367 symbols completed. ATMU took 30.8s. Estimated remaining: 4h 26m 30s. Rate: 8.2 symbols/min [2025-10-20T17:20:41.852574] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-05. [2025-10-20T17:20:41.912814] [INFO ] [MAIN ] ATR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:41.942782] [DEBUG] [LOCAL ] ATR: Loaded 30974 existing records from primary storage. [2025-10-20T17:20:41.943056] [INFO ] [MAIN ] ATR: Loaded 30974 existing records. [2025-10-20T17:20:42.123634] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T17:20:42.180506] [WARN ] [MAIN ] ATR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:42.180754] [INFO ] [REMOTE ] ATI: Full fetch complete (74018 records). [2025-10-20T17:20:42.181115] [INFO ] [MAIN ] ATR: Wiping existing primary data before full refresh. [2025-10-20T17:20:42.181200] [INFO ] [LOCAL ] ATR: Starting deletion of primary data. [2025-10-20T17:20:42.181381] [DEBUG] [VALIDATOR ] ATI: Validating primary data. [2025-10-20T17:20:42.181681] [DEBUG] [LOCAL ] ATR: Deleted primary data file. [2025-10-20T17:20:42.181757] [INFO ] [MAIN ] ATR: Starting full primary refresh. [2025-10-20T17:20:43.125195] [INFO ] [REMOTE ] ATR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:43.120444] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T17:20:43.157292] [DEBUG] [VALIDATOR ] ATI: All expected trading days are present. [2025-10-20T17:20:43.157011] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-01. [2025-10-20T17:20:43.157447] [DEBUG] [VALIDATOR ] ATI: Primary data validation passed. [2025-10-20T17:20:43.159603] [INFO ] [MAIN ] ATI: Fetched primary data is valid. [2025-10-20T17:20:43.159989] [DEBUG] [LOCAL ] ATI: Saving primary data. [2025-10-20T17:20:43.160557] [INFO ] [REMOTE ] ATKR: Full fetch complete (53302 records). [2025-10-20T17:20:43.161317] [DEBUG] [VALIDATOR ] ATKR: Validating primary data. [2025-10-20T17:20:43.184527] [DEBUG] [LOCAL ] ATI: Trimmed 74018 records to 51311 (keeping 200 days of data) [2025-10-20T17:20:43.875939] [DEBUG] [VALIDATOR ] ATKR: All expected trading days are present. [2025-10-20T17:20:43.889040] [DEBUG] [VALIDATOR ] ATKR: Primary data validation passed. [2025-10-20T17:20:43.894906] [INFO ] [MAIN ] ATKR: Fetched primary data is valid. [2025-10-20T17:20:43.895035] [DEBUG] [LOCAL ] ATKR: Saving primary data. [2025-10-20T17:20:43.896668] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T17:20:43.960960] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-06. [2025-10-20T17:20:43.965835] [DEBUG] [LOCAL ] ATKR: Trimmed 53302 records to 35028 (keeping 200 days of data) [2025-10-20T17:20:44.039259] [DEBUG] [LOCAL ] ATI: Primary data saved (51311 records). [2025-10-20T17:20:44.049951] [INFO ] [MAIN ] ATI: Full history saved. [2025-10-20T17:20:44.077074] [INFO ] [MAIN ] Update Progress: 171/2367 symbols completed. ATI took 38.3s. Estimated remaining: 4h 25m 20s. Rate: 8.3 symbols/min [2025-10-20T17:20:44.352749] [DEBUG] [LOCAL ] ATKR: Primary data saved (35028 records). [2025-10-20T17:20:44.385593] [INFO ] [MAIN ] ATKR: Full history saved. [2025-10-20T17:20:44.412998] [INFO ] [MAIN ] ATS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:44.413263] [DEBUG] [LOCAL ] ATS: No existing data file found in primary storage. [2025-10-20T17:20:44.409471] [INFO ] [MAIN ] Update Progress: 172/2367 symbols completed. ATKR took 38.6s. Estimated remaining: 4h 23m 44s. Rate: 8.3 symbols/min [2025-10-20T17:20:44.418811] [WARN ] [MAIN ] ATS: No existing data found, performing full refresh instead. [2025-10-20T17:20:44.448900] [INFO ] [MAIN ] ATS: Wiping existing primary data before full refresh. [2025-10-20T17:20:44.477075] [INFO ] [LOCAL ] ATS: Starting deletion of primary data. [2025-10-20T17:20:44.528799] [DEBUG] [LOCAL ] ATS: No primary data file found to delete. [2025-10-20T17:20:44.529561] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T17:20:44.547181] [INFO ] [MAIN ] ATS: Starting full primary refresh. [2025-10-20T17:20:44.553554] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-02. [2025-10-20T17:20:44.778290] [INFO ] [MAIN ] ATUS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:44.788925] [DEBUG] [LOCAL ] ATUS: No existing data file found in primary storage. [2025-10-20T17:20:44.794182] [WARN ] [MAIN ] ATUS: No existing data found, performing full refresh instead. [2025-10-20T17:20:44.830616] [INFO ] [MAIN ] ATUS: Wiping existing primary data before full refresh. [2025-10-20T17:20:44.850672] [INFO ] [LOCAL ] ATUS: Starting deletion of primary data. [2025-10-20T17:20:44.930768] [DEBUG] [LOCAL ] ATUS: No primary data file found to delete. [2025-10-20T17:20:44.944874] [INFO ] [MAIN ] ATUS: Starting full primary refresh. [2025-10-20T17:20:45.182727] [INFO ] [REMOTE ] ATS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:45.230372] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-01. [2025-10-20T17:20:45.455365] [INFO ] [REMOTE ] ATUS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:45.481567] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:20:45.838558] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T17:20:45.851344] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-03. [2025-10-20T17:20:46.138887] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:20:46.651354] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-02. [2025-10-20T17:20:47.220054] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:20:47.236828] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T17:20:47.451385] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-04. [2025-10-20T17:20:48.251452] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-01. [2025-10-20T17:20:48.710347] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T17:20:49.051381] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-07. [2025-10-20T17:20:49.637554] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:20:49.851376] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-02. [2025-10-20T17:20:50.625117] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T17:20:50.651329] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-08. [2025-10-20T17:20:51.116043] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T17:20:51.451382] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-05. [2025-10-20T17:20:51.909140] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:20:52.251377] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-09. [2025-10-20T17:20:52.775528] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T17:20:53.051345] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-06. [2025-10-20T17:20:53.850037] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:20:53.859177] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-03. [2025-10-20T17:20:54.332058] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T17:20:54.546931] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T17:20:54.651384] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-10. [2025-10-20T17:20:55.451415] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-03. [2025-10-20T17:20:56.251368] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-04. [2025-10-20T17:20:56.472379] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T17:20:56.478670] [INFO ] [REMOTE ] ATO: Full fetch complete (60745 records). [2025-10-20T17:20:56.478821] [DEBUG] [VALIDATOR ] ATO: Validating primary data. [2025-10-20T17:20:56.872551] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:20:56.917862] [DEBUG] [VALIDATOR ] ATO: All expected trading days are present. [2025-10-20T17:20:56.917950] [DEBUG] [VALIDATOR ] ATO: Primary data validation passed. [2025-10-20T17:20:56.918700] [INFO ] [MAIN ] ATO: Fetched primary data is valid. [2025-10-20T17:20:56.918769] [DEBUG] [LOCAL ] ATO: Saving primary data. [2025-10-20T17:20:56.920064] [DEBUG] [LOCAL ] ATO: Trimmed 60745 records to 41744 (keeping 200 days of data) [2025-10-20T17:20:57.051387] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-04. [2025-10-20T17:20:57.191884] [DEBUG] [LOCAL ] ATO: Primary data saved (41744 records). [2025-10-20T17:20:57.192041] [INFO ] [MAIN ] ATO: Full history saved. [2025-10-20T17:20:57.192811] [INFO ] [MAIN ] Update Progress: 173/2367 symbols completed. ATO took 35.8s. Estimated remaining: 4h 24m 48s. Rate: 8.3 symbols/min [2025-10-20T17:20:57.369525] [INFO ] [MAIN ] AU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:57.418899] [DEBUG] [LOCAL ] AU: Loaded 53857 existing records from primary storage. [2025-10-20T17:20:57.419213] [INFO ] [MAIN ] AU: Loaded 53857 existing records. [2025-10-20T17:20:57.584504] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:20:57.620126] [WARN ] [MAIN ] AU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:57.620308] [INFO ] [MAIN ] AU: Wiping existing primary data before full refresh. [2025-10-20T17:20:57.620354] [INFO ] [LOCAL ] AU: Starting deletion of primary data. [2025-10-20T17:20:57.620754] [DEBUG] [LOCAL ] AU: Deleted primary data file. [2025-10-20T17:20:57.620810] [INFO ] [MAIN ] AU: Starting full primary refresh. [2025-10-20T17:20:57.859158] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-05. [2025-10-20T17:20:57.979037] [INFO ] [REMOTE ] AU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:58.651366] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-07. [2025-10-20T17:20:58.777491] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T17:20:58.886461] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T17:20:59.451374] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-06. [2025-10-20T17:21:00.029135] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:21:00.161180] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T17:21:00.251373] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-05. [2025-10-20T17:21:01.051410] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-07. [2025-10-20T17:21:01.761621] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T17:21:01.851360] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-08. [2025-10-20T17:21:01.958109] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:21:02.651394] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-01. [2025-10-20T17:21:02.824445] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T17:21:03.451395] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-08. [2025-10-20T17:21:04.251395] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-06. [2025-10-20T17:21:04.399713] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T17:21:04.427773] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T17:21:05.051389] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-09. [2025-10-20T17:21:05.851356] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-09. [2025-10-20T17:21:05.966571] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T17:21:05.999745] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T17:21:06.651436] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-02. [2025-10-20T17:21:07.308005] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:21:07.451440] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-10. [2025-10-20T17:21:08.251405] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-07. [2025-10-20T17:21:08.529954] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T17:21:09.051451] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-03. [2025-10-20T17:21:09.851354] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-10. [2025-10-20T17:21:10.096536] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T17:21:10.651400] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-08. [2025-10-20T17:21:10.739356] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-10 (3.29 seconds). [2025-10-20T17:21:10.741825] [INFO ] [REMOTE ] ATS: Full fetch complete (25476 records). [2025-10-20T17:21:10.741913] [DEBUG] [VALIDATOR ] ATS: Validating primary data. [2025-10-20T17:21:11.151490] [DEBUG] [VALIDATOR ] ATS: All expected trading days are present. [2025-10-20T17:21:11.151738] [DEBUG] [VALIDATOR ] ATS: Primary data validation passed. [2025-10-20T17:21:11.152242] [INFO ] [MAIN ] ATS: Fetched primary data is valid. [2025-10-20T17:21:11.152324] [DEBUG] [LOCAL ] ATS: Saving primary data. [2025-10-20T17:21:11.153311] [DEBUG] [LOCAL ] ATS: Trimmed 25476 records to 17325 (keeping 200 days of data) [2025-10-20T17:21:11.273592] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-03 (2.22 seconds). [2025-10-20T17:21:11.294679] [DEBUG] [LOCAL ] ATS: Primary data saved (17325 records). [2025-10-20T17:21:11.295019] [INFO ] [MAIN ] ATS: Full history saved. [2025-10-20T17:21:11.295127] [INFO ] [MAIN ] Update Progress: 174/2367 symbols completed. ATS took 27.2s. Estimated remaining: 4h 26m 7s. Rate: 8.2 symbols/min [2025-10-20T17:21:11.342547] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:21:11.396798] [INFO ] [REMOTE ] ATR: Full fetch complete (43989 records). [2025-10-20T17:21:11.417976] [DEBUG] [VALIDATOR ] ATR: Validating primary data. [2025-10-20T17:21:11.456545] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-04. [2025-10-20T17:21:11.552974] [INFO ] [MAIN ] AUB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:11.596279] [DEBUG] [LOCAL ] AUB: No existing data file found in primary storage. [2025-10-20T17:21:11.599787] [WARN ] [MAIN ] AUB: No existing data found, performing full refresh instead. [2025-10-20T17:21:11.600058] [INFO ] [MAIN ] AUB: Wiping existing primary data before full refresh. [2025-10-20T17:21:11.605732] [INFO ] [LOCAL ] AUB: Starting deletion of primary data. [2025-10-20T17:21:11.610976] [DEBUG] [LOCAL ] AUB: No primary data file found to delete. [2025-10-20T17:21:11.616170] [INFO ] [MAIN ] AUB: Starting full primary refresh. [2025-10-20T17:21:12.179389] [DEBUG] [VALIDATOR ] ATR: All expected trading days are present. [2025-10-20T17:21:12.179530] [DEBUG] [VALIDATOR ] ATR: Primary data validation passed. [2025-10-20T17:21:12.180319] [INFO ] [MAIN ] ATR: Fetched primary data is valid. [2025-10-20T17:21:12.196041] [DEBUG] [LOCAL ] ATR: Saving primary data. [2025-10-20T17:21:12.260587] [INFO ] [REMOTE ] AUB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:12.261421] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-01. [2025-10-20T17:21:12.261609] [DEBUG] [LOCAL ] ATR: Trimmed 43989 records to 30554 (keeping 200 days of data) [2025-10-20T17:21:12.394057] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:21:12.508009] [DEBUG] [LOCAL ] ATR: Primary data saved (30554 records). [2025-10-20T17:21:12.508159] [INFO ] [MAIN ] ATR: Full history saved. [2025-10-20T17:21:12.508267] [INFO ] [MAIN ] Update Progress: 175/2367 symbols completed. ATR took 30.8s. Estimated remaining: 4h 24m 44s. Rate: 8.3 symbols/min [2025-10-20T17:21:12.694492] [INFO ] [MAIN ] AUNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:12.708609] [DEBUG] [LOCAL ] AUNA: Loaded 5186 existing records from primary storage. [2025-10-20T17:21:12.709078] [INFO ] [MAIN ] AUNA: Loaded 5186 existing records. [2025-10-20T17:21:12.912693] [WARN ] [MAIN ] AUNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:12.912929] [INFO ] [MAIN ] AUNA: Wiping existing primary data before full refresh. [2025-10-20T17:21:12.912982] [INFO ] [LOCAL ] AUNA: Starting deletion of primary data. [2025-10-20T17:21:12.913123] [DEBUG] [LOCAL ] AUNA: Deleted primary data file. [2025-10-20T17:21:12.913162] [INFO ] [MAIN ] AUNA: Starting full primary refresh. [2025-10-20T17:21:13.062872] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-09. [2025-10-20T17:21:13.303383] [INFO ] [REMOTE ] AUNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:13.812469] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-04 (2.36 seconds). [2025-10-20T17:21:13.851489] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-05. [2025-10-20T17:21:13.861217] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:21:14.651453] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-02. [2025-10-20T17:21:14.747124] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:21:15.451341] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-01. [2025-10-20T17:21:15.968065] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T17:21:16.033349] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T17:21:16.251378] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-10. [2025-10-20T17:21:16.258623] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T17:21:17.051360] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-02. [2025-10-20T17:21:17.500503] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T17:21:17.851372] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-03. [2025-10-20T17:21:18.090332] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T17:21:18.097908] [INFO ] [REMOTE ] ATUS: Full fetch complete (71392 records). [2025-10-20T17:21:18.098078] [DEBUG] [VALIDATOR ] ATUS: Validating primary data. [2025-10-20T17:21:18.521337] [DEBUG] [VALIDATOR ] ATUS: All expected trading days are present. [2025-10-20T17:21:18.521424] [DEBUG] [VALIDATOR ] ATUS: Primary data validation passed. [2025-10-20T17:21:18.522190] [INFO ] [MAIN ] ATUS: Fetched primary data is valid. [2025-10-20T17:21:18.522253] [DEBUG] [LOCAL ] ATUS: Saving primary data. [2025-10-20T17:21:18.523616] [DEBUG] [LOCAL ] ATUS: Trimmed 71392 records to 49947 (keeping 200 days of data) [2025-10-20T17:21:18.611345] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T17:21:18.695518] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-04. [2025-10-20T17:21:18.823674] [DEBUG] [LOCAL ] ATUS: Primary data saved (49947 records). [2025-10-20T17:21:18.823816] [INFO ] [MAIN ] ATUS: Full history saved. [2025-10-20T17:21:18.823887] [INFO ] [MAIN ] Update Progress: 176/2367 symbols completed. ATUS took 34.4s. Estimated remaining: 4h 24m 25s. Rate: 8.3 symbols/min [2025-10-20T17:21:19.007611] [INFO ] [MAIN ] AVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:19.057849] [DEBUG] [LOCAL ] AVA: Loaded 39279 existing records from primary storage. [2025-10-20T17:21:19.058254] [INFO ] [MAIN ] AVA: Loaded 39279 existing records. [2025-10-20T17:21:19.285161] [WARN ] [MAIN ] AVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:19.288846] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T17:21:19.288969] [INFO ] [MAIN ] AVA: Wiping existing primary data before full refresh. [2025-10-20T17:21:19.289103] [INFO ] [LOCAL ] AVA: Starting deletion of primary data. [2025-10-20T17:21:19.290794] [DEBUG] [LOCAL ] AVA: Deleted primary data file. [2025-10-20T17:21:19.291044] [INFO ] [MAIN ] AVA: Starting full primary refresh. [2025-10-20T17:21:19.452784] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-06. [2025-10-20T17:21:19.712785] [INFO ] [REMOTE ] AVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:20.251384] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-01. [2025-10-20T17:21:21.051422] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-03. [2025-10-20T17:21:21.456957] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:21:21.851432] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-05. [2025-10-20T17:21:22.063881] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:21:22.469243] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T17:21:22.651371] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-02. [2025-10-20T17:21:22.705758] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T17:21:23.451379] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-06. [2025-10-20T17:21:23.976612] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:21:24.245173] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:21:24.251338] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-07. [2025-10-20T17:21:24.606473] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-10-20T17:21:25.051403] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-08. [2025-10-20T17:21:25.588170] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T17:21:25.851388] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-09. [2025-10-20T17:21:26.651414] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-03. [2025-10-20T17:21:26.742454] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T17:21:27.451469] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-04. [2025-10-20T17:21:28.088031] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T17:21:28.251440] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-10. [2025-10-20T17:21:28.980221] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T17:21:28.982169] [INFO ] [REMOTE ] AUNA: Full fetch complete (7779 records). [2025-10-20T17:21:28.982247] [DEBUG] [VALIDATOR ] AUNA: Validating primary data. [2025-10-20T17:21:29.058053] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-07. [2025-10-20T17:21:29.411771] [DEBUG] [VALIDATOR ] AUNA: All expected trading days are present. [2025-10-20T17:21:29.411888] [DEBUG] [VALIDATOR ] AUNA: Primary data validation passed. [2025-10-20T17:21:29.412396] [INFO ] [MAIN ] AUNA: Fetched primary data is valid. [2025-10-20T17:21:29.412503] [DEBUG] [LOCAL ] AUNA: Saving primary data. [2025-10-20T17:21:29.413382] [DEBUG] [LOCAL ] AUNA: Trimmed 7779 records to 5212 (keeping 200 days of data) [2025-10-20T17:21:29.474922] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T17:21:29.494028] [DEBUG] [LOCAL ] AUNA: Primary data saved (5212 records). [2025-10-20T17:21:29.494415] [INFO ] [MAIN ] AUNA: Full history saved. [2025-10-20T17:21:29.494571] [INFO ] [MAIN ] Update Progress: 177/2367 symbols completed. AUNA took 17.0s. Estimated remaining: 4h 25m 0s. Rate: 8.3 symbols/min [2025-10-20T17:21:29.682370] [INFO ] [MAIN ] AVAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:29.693272] [DEBUG] [LOCAL ] AVAL: Loaded 8705 existing records from primary storage. [2025-10-20T17:21:29.693570] [INFO ] [MAIN ] AVAL: Loaded 8705 existing records. [2025-10-20T17:21:29.856514] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-05. [2025-10-20T17:21:29.892565] [WARN ] [MAIN ] AVAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:29.892902] [INFO ] [MAIN ] AVAL: Wiping existing primary data before full refresh. [2025-10-20T17:21:29.892974] [INFO ] [LOCAL ] AVAL: Starting deletion of primary data. [2025-10-20T17:21:29.893331] [DEBUG] [LOCAL ] AVAL: Deleted primary data file. [2025-10-20T17:21:29.893371] [INFO ] [MAIN ] AVAL: Starting full primary refresh. [2025-10-20T17:21:30.295921] [INFO ] [REMOTE ] AVAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:30.651391] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-04. [2025-10-20T17:21:30.861705] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T17:21:31.451391] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-01. [2025-10-20T17:21:31.610886] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T17:21:32.155153] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:21:32.251407] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-08. [2025-10-20T17:21:32.297585] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T17:21:33.051391] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-02. [2025-10-20T17:21:33.851406] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-05. [2025-10-20T17:21:34.039591] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T17:21:34.222326] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T17:21:34.651409] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-06. [2025-10-20T17:21:35.090836] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T17:21:35.451416] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-03. [2025-10-20T17:21:35.965078] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:21:36.251385] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-07. [2025-10-20T17:21:36.442812] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:21:37.051404] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-06. [2025-10-20T17:21:37.816133] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T17:21:37.851433] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-04. [2025-10-20T17:21:38.574904] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:21:38.651457] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-08. [2025-10-20T17:21:38.711978] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:21:39.451460] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-09. [2025-10-20T17:21:40.251349] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-07. [2025-10-20T17:21:40.304710] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:21:41.051389] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-05. [2025-10-20T17:21:41.675209] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T17:21:41.753289] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:21:41.790969] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:21:41.851338] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-10. [2025-10-20T17:21:42.651470] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-08. [2025-10-20T17:21:43.451709] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-09. [2025-10-20T17:21:44.057870] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T17:21:44.065807] [INFO ] [REMOTE ] AU: Full fetch complete (76604 records). [2025-10-20T17:21:44.065971] [DEBUG] [VALIDATOR ] AU: Validating primary data. [2025-10-20T17:21:44.252115] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-06. [2025-10-20T17:21:44.381330] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T17:21:44.523682] [DEBUG] [VALIDATOR ] AU: All expected trading days are present. [2025-10-20T17:21:44.523925] [DEBUG] [VALIDATOR ] AU: Primary data validation passed. [2025-10-20T17:21:44.524769] [INFO ] [MAIN ] AU: Fetched primary data is valid. [2025-10-20T17:21:44.524847] [DEBUG] [LOCAL ] AU: Saving primary data. [2025-10-20T17:21:44.526531] [DEBUG] [LOCAL ] AU: Trimmed 76604 records to 53089 (keeping 200 days of data) [2025-10-20T17:21:44.773884] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T17:21:44.889018] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:21:44.904110] [DEBUG] [LOCAL ] AU: Primary data saved (53089 records). [2025-10-20T17:21:44.904263] [INFO ] [MAIN ] AU: Full history saved. [2025-10-20T17:21:44.904709] [INFO ] [MAIN ] Update Progress: 178/2367 symbols completed. AU took 47.7s. Estimated remaining: 4h 26m 33s. Rate: 8.2 symbols/min [2025-10-20T17:21:45.056513] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-09. [2025-10-20T17:21:45.082428] [INFO ] [MAIN ] AVB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:45.133366] [DEBUG] [LOCAL ] AVB: Loaded 42866 existing records from primary storage. [2025-10-20T17:21:45.133739] [INFO ] [MAIN ] AVB: Loaded 42866 existing records. [2025-10-20T17:21:45.359163] [WARN ] [MAIN ] AVB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:45.359408] [INFO ] [MAIN ] AVB: Wiping existing primary data before full refresh. [2025-10-20T17:21:45.359456] [INFO ] [LOCAL ] AVB: Starting deletion of primary data. [2025-10-20T17:21:45.359868] [DEBUG] [LOCAL ] AVB: Deleted primary data file. [2025-10-20T17:21:45.359922] [INFO ] [MAIN ] AVB: Starting full primary refresh. [2025-10-20T17:21:45.739575] [INFO ] [REMOTE ] AVB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:45.851392] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-07. [2025-10-20T17:21:46.213239] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T17:21:46.623371] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T17:21:46.651336] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-08. [2025-10-20T17:21:47.379250] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T17:21:47.451401] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-10. [2025-10-20T17:21:48.251377] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-10. [2025-10-20T17:21:48.793312] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T17:21:48.799021] [INFO ] [REMOTE ] AVA: Full fetch complete (55483 records). [2025-10-20T17:21:48.799163] [DEBUG] [VALIDATOR ] AVA: Validating primary data. [2025-10-20T17:21:49.056615] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-09. [2025-10-20T17:21:49.182872] [DEBUG] [VALIDATOR ] AVA: All expected trading days are present. [2025-10-20T17:21:49.182961] [DEBUG] [VALIDATOR ] AVA: Primary data validation passed. [2025-10-20T17:21:49.183911] [INFO ] [MAIN ] AVA: Fetched primary data is valid. [2025-10-20T17:21:49.183974] [DEBUG] [LOCAL ] AVA: Saving primary data. [2025-10-20T17:21:49.185144] [DEBUG] [LOCAL ] AVA: Trimmed 55483 records to 38503 (keeping 200 days of data) [2025-10-20T17:21:49.418302] [DEBUG] [LOCAL ] AVA: Primary data saved (38503 records). [2025-10-20T17:21:49.418473] [INFO ] [MAIN ] AVA: Full history saved. [2025-10-20T17:21:49.418561] [INFO ] [MAIN ] Update Progress: 179/2367 symbols completed. AVA took 30.6s. Estimated remaining: 4h 25m 52s. Rate: 8.2 symbols/min [2025-10-20T17:21:49.597314] [INFO ] [MAIN ] AVBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:49.598291] [DEBUG] [LOCAL ] AVBC: No existing data file found in primary storage. [2025-10-20T17:21:49.598418] [WARN ] [MAIN ] AVBC: No existing data found, performing full refresh instead. [2025-10-20T17:21:49.598989] [INFO ] [MAIN ] AVBC: Wiping existing primary data before full refresh. [2025-10-20T17:21:49.607126] [INFO ] [LOCAL ] AVBC: Starting deletion of primary data. [2025-10-20T17:21:49.607226] [DEBUG] [LOCAL ] AVBC: No primary data file found to delete. [2025-10-20T17:21:49.630332] [INFO ] [MAIN ] AVBC: Starting full primary refresh. [2025-10-20T17:21:49.639311] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T17:21:49.661238] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:21:49.771106] [INFO ] [REMOTE ] AUB: Full fetch complete (66264 records). [2025-10-20T17:21:49.771374] [DEBUG] [VALIDATOR ] AUB: Validating primary data. [2025-10-20T17:21:49.857212] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-01. [2025-10-20T17:21:50.342450] [INFO ] [REMOTE ] AVBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:50.500369] [DEBUG] [VALIDATOR ] AUB: All expected trading days are present. [2025-10-20T17:21:50.500466] [DEBUG] [VALIDATOR ] AUB: Primary data validation passed. [2025-10-20T17:21:50.501145] [INFO ] [MAIN ] AUB: Fetched primary data is valid. [2025-10-20T17:21:50.501210] [DEBUG] [LOCAL ] AUB: Saving primary data. [2025-10-20T17:21:50.503075] [DEBUG] [LOCAL ] AUB: Trimmed 66264 records to 45950 (keeping 200 days of data) [2025-10-20T17:21:50.685021] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-01. [2025-10-20T17:21:50.790411] [DEBUG] [LOCAL ] AUB: Primary data saved (45950 records). [2025-10-20T17:21:50.790559] [INFO ] [MAIN ] AUB: Full history saved. [2025-10-20T17:21:50.790633] [INFO ] [MAIN ] Update Progress: 180/2367 symbols completed. AUB took 39.5s. Estimated remaining: 4h 24m 33s. Rate: 8.3 symbols/min [2025-10-20T17:21:50.897343] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:21:50.898247] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-01 [2025-10-20T17:21:50.914233] [FATAL] [REMOTE ] AVBC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:21:50.940049] [FATAL] [MAIN ] AVBC: Failed to fetch primary data. [2025-10-20T17:21:50.954234] [WARN ] [MAIN ] AVBC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:21:50.972865] [INFO ] [MAIN ] AVD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:50.973019] [INFO ] [LOCAL ] AVBC: Starting deletion of primary data. [2025-10-20T17:21:50.973749] [DEBUG] [LOCAL ] AVBC: No primary data file found to delete. [2025-10-20T17:21:50.973832] [INFO ] [LOCAL ] AVBC: Starting deletion of derivative data. [2025-10-20T17:21:50.973892] [DEBUG] [LOCAL ] AVBC: No derivative data file found to delete. [2025-10-20T17:21:50.973976] [INFO ] [MAIN ] Update Progress: 181/2367 symbols completed. AVBC took 1.6s. Estimated remaining: 4h 22m 60s. Rate: 8.3 symbols/min [2025-10-20T17:21:51.195479] [INFO ] [MAIN ] AVK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:51.196828] [DEBUG] [LOCAL ] AVD: Loaded 22021 existing records from primary storage. [2025-10-20T17:21:51.197879] [INFO ] [MAIN ] AVD: Loaded 22021 existing records. [2025-10-20T17:21:51.429601] [WARN ] [MAIN ] AVD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:51.430877] [DEBUG] [LOCAL ] AVK: Loaded 18858 existing records from primary storage. [2025-10-20T17:21:51.431554] [INFO ] [MAIN ] AVD: Wiping existing primary data before full refresh. [2025-10-20T17:21:51.432044] [INFO ] [LOCAL ] AVD: Starting deletion of primary data. [2025-10-20T17:21:51.431926] [INFO ] [MAIN ] AVK: Loaded 18858 existing records. [2025-10-20T17:21:51.437868] [DEBUG] [LOCAL ] AVD: Deleted primary data file. [2025-10-20T17:21:51.456994] [INFO ] [MAIN ] AVD: Starting full primary refresh. [2025-10-20T17:21:51.456598] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-10. [2025-10-20T17:21:51.884867] [WARN ] [MAIN ] AVK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:51.926311] [INFO ] [MAIN ] AVK: Wiping existing primary data before full refresh. [2025-10-20T17:21:51.942959] [INFO ] [LOCAL ] AVK: Starting deletion of primary data. [2025-10-20T17:21:51.997125] [DEBUG] [LOCAL ] AVK: Deleted primary data file. [2025-10-20T17:21:52.005158] [INFO ] [MAIN ] AVK: Starting full primary refresh. [2025-10-20T17:21:52.007088] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-01 (2.15 seconds). [2025-10-20T17:21:52.259036] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-02. [2025-10-20T17:21:52.301523] [INFO ] [REMOTE ] AVD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:52.435792] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T17:21:52.493799] [INFO ] [REMOTE ] AVAL: Full fetch complete (14958 records). [2025-10-20T17:21:52.520820] [DEBUG] [VALIDATOR ] AVAL: Validating primary data. [2025-10-20T17:21:52.758325] [INFO ] [REMOTE ] AVK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:53.065812] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-01. [2025-10-20T17:21:53.079627] [DEBUG] [VALIDATOR ] AVAL: All expected trading days are present. [2025-10-20T17:21:53.079764] [DEBUG] [VALIDATOR ] AVAL: Primary data validation passed. [2025-10-20T17:21:53.080168] [INFO ] [MAIN ] AVAL: Fetched primary data is valid. [2025-10-20T17:21:53.080389] [DEBUG] [LOCAL ] AVAL: Saving primary data. [2025-10-20T17:21:53.081049] [DEBUG] [LOCAL ] AVAL: Trimmed 14958 records to 8528 (keeping 200 days of data) [2025-10-20T17:21:53.135895] [DEBUG] [LOCAL ] AVAL: Primary data saved (8528 records). [2025-10-20T17:21:53.136091] [INFO ] [MAIN ] AVAL: Full history saved. [2025-10-20T17:21:53.136194] [INFO ] [MAIN ] Update Progress: 182/2367 symbols completed. AVAL took 23.6s. Estimated remaining: 4h 21m 52s. Rate: 8.3 symbols/min [2025-10-20T17:21:53.337358] [INFO ] [MAIN ] AVNS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:53.375765] [DEBUG] [LOCAL ] AVNS: Loaded 34111 existing records from primary storage. [2025-10-20T17:21:53.376172] [INFO ] [MAIN ] AVNS: Loaded 34111 existing records. [2025-10-20T17:21:53.576917] [WARN ] [MAIN ] AVNS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:53.577299] [INFO ] [MAIN ] AVNS: Wiping existing primary data before full refresh. [2025-10-20T17:21:53.577357] [INFO ] [LOCAL ] AVNS: Starting deletion of primary data. [2025-10-20T17:21:53.577778] [DEBUG] [LOCAL ] AVNS: Deleted primary data file. [2025-10-20T17:21:53.577839] [INFO ] [MAIN ] AVNS: Starting full primary refresh. [2025-10-20T17:21:53.855416] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-01. [2025-10-20T17:21:53.989901] [INFO ] [REMOTE ] AVNS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:54.014337] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T17:21:54.406372] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:21:54.651385] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-01. [2025-10-20T17:21:54.888058] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T17:21:55.451364] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-02. [2025-10-20T17:21:55.842807] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:21:56.251384] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-02. [2025-10-20T17:21:56.464059] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T17:21:57.051409] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-03. [2025-10-20T17:21:57.261545] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T17:21:57.851518] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-02. [2025-10-20T17:21:57.940677] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T17:21:58.651377] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-04. [2025-10-20T17:21:59.065161] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:21:59.451496] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-03. [2025-10-20T17:21:59.866127] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:22:00.251366] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-03. [2025-10-20T17:22:00.697217] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T17:22:01.051451] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-04. [2025-10-20T17:22:01.782572] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T17:22:01.851432] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-05. [2025-10-20T17:22:02.057682] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T17:22:02.651381] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-05. [2025-10-20T17:22:02.927085] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T17:22:03.451351] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-06. [2025-10-20T17:22:03.757679] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T17:22:04.251365] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-04. [2025-10-20T17:22:04.494019] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:22:05.051445] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-06. [2025-10-20T17:22:05.669248] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T17:22:05.851474] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-03. [2025-10-20T17:22:06.205326] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T17:22:06.651479] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-05. [2025-10-20T17:22:07.423573] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T17:22:07.451367] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-04. [2025-10-20T17:22:08.132120] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T17:22:08.251414] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-07. [2025-10-20T17:22:09.052228] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-06. [2025-10-20T17:22:09.059819] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:22:09.435383] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T17:22:09.851369] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-08. [2025-10-20T17:22:09.974816] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T17:22:10.651443] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-07. [2025-10-20T17:22:11.140129] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T17:22:11.451438] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-05. [2025-10-20T17:22:12.019856] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T17:22:12.251399] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-08. [2025-10-20T17:22:12.845260] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T17:22:13.051403] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-07. [2025-10-20T17:22:13.676071] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T17:22:13.851449] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-06. [2025-10-20T17:22:14.332418] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T17:22:14.651575] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-09. [2025-10-20T17:22:15.044630] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:22:15.451425] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-07. [2025-10-20T17:22:15.732972] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T17:22:16.251440] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-10. [2025-10-20T17:22:16.725608] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T17:22:17.051409] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-08. [2025-10-20T17:22:17.563530] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:22:17.566842] [INFO ] [REMOTE ] AVD: Full fetch complete (32733 records). [2025-10-20T17:22:17.566964] [DEBUG] [VALIDATOR ] AVD: Validating primary data. [2025-10-20T17:22:17.852406] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-09. [2025-10-20T17:22:17.942688] [DEBUG] [VALIDATOR ] AVD: All expected trading days are present. [2025-10-20T17:22:17.942790] [DEBUG] [VALIDATOR ] AVD: Primary data validation passed. [2025-10-20T17:22:17.943247] [INFO ] [MAIN ] AVD: Fetched primary data is valid. [2025-10-20T17:22:17.943288] [DEBUG] [LOCAL ] AVD: Saving primary data. [2025-10-20T17:22:17.944206] [DEBUG] [LOCAL ] AVD: Trimmed 32733 records to 21480 (keeping 200 days of data) [2025-10-20T17:22:18.066162] [DEBUG] [LOCAL ] AVD: Primary data saved (21480 records). [2025-10-20T17:22:18.066382] [INFO ] [MAIN ] AVD: Full history saved. [2025-10-20T17:22:18.066509] [INFO ] [MAIN ] Update Progress: 183/2367 symbols completed. AVD took 27.3s. Estimated remaining: 4h 25m 16s. Rate: 8.2 symbols/min [2025-10-20T17:22:18.217113] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-10-20T17:22:18.276135] [INFO ] [MAIN ] AVNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:18.321925] [DEBUG] [LOCAL ] AVNT: Loaded 39562 existing records from primary storage. [2025-10-20T17:22:18.322499] [INFO ] [MAIN ] AVNT: Loaded 39562 existing records. [2025-10-20T17:22:18.527163] [WARN ] [MAIN ] AVNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:18.527737] [INFO ] [MAIN ] AVNT: Wiping existing primary data before full refresh. [2025-10-20T17:22:18.527796] [INFO ] [LOCAL ] AVNT: Starting deletion of primary data. [2025-10-20T17:22:18.528065] [DEBUG] [LOCAL ] AVNT: Deleted primary data file. [2025-10-20T17:22:18.528122] [INFO ] [MAIN ] AVNT: Starting full primary refresh. [2025-10-20T17:22:18.656634] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-08. [2025-10-20T17:22:18.982963] [INFO ] [REMOTE ] AVNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:19.352132] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T17:22:19.451382] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-10. [2025-10-20T17:22:20.251374] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-09. [2025-10-20T17:22:20.514787] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T17:22:20.601584] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:22:20.607578] [INFO ] [REMOTE ] AVNS: Full fetch complete (48430 records). [2025-10-20T17:22:20.607781] [DEBUG] [VALIDATOR ] AVNS: Validating primary data. [2025-10-20T17:22:21.048340] [DEBUG] [VALIDATOR ] AVNS: All expected trading days are present. [2025-10-20T17:22:21.048579] [DEBUG] [VALIDATOR ] AVNS: Primary data validation passed. [2025-10-20T17:22:21.054755] [INFO ] [MAIN ] AVNS: Fetched primary data is valid. [2025-10-20T17:22:21.054875] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-09. [2025-10-20T17:22:21.062888] [DEBUG] [LOCAL ] AVNS: Saving primary data. [2025-10-20T17:22:21.065569] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T17:22:21.072601] [DEBUG] [LOCAL ] AVNS: Trimmed 48430 records to 33631 (keeping 200 days of data) [2025-10-20T17:22:21.291065] [DEBUG] [LOCAL ] AVNS: Primary data saved (33631 records). [2025-10-20T17:22:21.291301] [INFO ] [MAIN ] AVNS: Full history saved. [2025-10-20T17:22:21.291455] [INFO ] [MAIN ] Update Progress: 184/2367 symbols completed. AVNS took 28.2s. Estimated remaining: 4h 24m 21s. Rate: 8.3 symbols/min [2025-10-20T17:22:21.472708] [INFO ] [MAIN ] AVTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:21.537445] [DEBUG] [LOCAL ] AVTR: Loaded 54352 existing records from primary storage. [2025-10-20T17:22:21.537933] [INFO ] [MAIN ] AVTR: Loaded 54352 existing records. [2025-10-20T17:22:21.758701] [WARN ] [MAIN ] AVTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:21.759055] [INFO ] [MAIN ] AVTR: Wiping existing primary data before full refresh. [2025-10-20T17:22:21.759113] [INFO ] [LOCAL ] AVTR: Starting deletion of primary data. [2025-10-20T17:22:21.759711] [DEBUG] [LOCAL ] AVTR: Deleted primary data file. [2025-10-20T17:22:21.759771] [INFO ] [MAIN ] AVTR: Starting full primary refresh. [2025-10-20T17:22:21.861416] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-10. [2025-10-20T17:22:22.149892] [INFO ] [REMOTE ] AVTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:22.651402] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-01. [2025-10-20T17:22:22.838315] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:22:23.041885] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T17:22:23.045689] [INFO ] [REMOTE ] AVK: Full fetch complete (26698 records). [2025-10-20T17:22:23.045788] [DEBUG] [VALIDATOR ] AVK: Validating primary data. [2025-10-20T17:22:23.431194] [DEBUG] [VALIDATOR ] AVK: All expected trading days are present. [2025-10-20T17:22:23.431295] [DEBUG] [VALIDATOR ] AVK: Primary data validation passed. [2025-10-20T17:22:23.431753] [INFO ] [MAIN ] AVK: Fetched primary data is valid. [2025-10-20T17:22:23.431817] [DEBUG] [LOCAL ] AVK: Saving primary data. [2025-10-20T17:22:23.432630] [DEBUG] [LOCAL ] AVK: Trimmed 26698 records to 18702 (keeping 200 days of data) [2025-10-20T17:22:23.483849] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-01. [2025-10-20T17:22:23.562171] [DEBUG] [LOCAL ] AVK: Primary data saved (18702 records). [2025-10-20T17:22:23.563171] [INFO ] [MAIN ] AVK: Full history saved. [2025-10-20T17:22:23.563283] [INFO ] [MAIN ] Update Progress: 185/2367 symbols completed. AVK took 32.6s. Estimated remaining: 4h 23m 15s. Rate: 8.3 symbols/min [2025-10-20T17:22:23.746351] [INFO ] [MAIN ] AVY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:23.788518] [DEBUG] [LOCAL ] AVY: Loaded 38653 existing records from primary storage. [2025-10-20T17:22:23.788952] [INFO ] [MAIN ] AVY: Loaded 38653 existing records. [2025-10-20T17:22:23.994710] [WARN ] [MAIN ] AVY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:23.994942] [INFO ] [MAIN ] AVY: Wiping existing primary data before full refresh. [2025-10-20T17:22:23.994992] [INFO ] [LOCAL ] AVY: Starting deletion of primary data. [2025-10-20T17:22:23.995284] [DEBUG] [LOCAL ] AVY: Deleted primary data file. [2025-10-20T17:22:23.995323] [INFO ] [MAIN ] AVY: Starting full primary refresh. [2025-10-20T17:22:24.252249] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-10. [2025-10-20T17:22:24.479568] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:22:24.532033] [INFO ] [REMOTE ] AVY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:24.593406] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T17:22:25.051378] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-02. [2025-10-20T17:22:25.851383] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-02. [2025-10-20T17:22:25.953002] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T17:22:25.960225] [INFO ] [REMOTE ] AVB: Full fetch complete (59219 records). [2025-10-20T17:22:25.960737] [DEBUG] [VALIDATOR ] AVB: Validating primary data. [2025-10-20T17:22:26.375583] [DEBUG] [VALIDATOR ] AVB: All expected trading days are present. [2025-10-20T17:22:26.375714] [DEBUG] [VALIDATOR ] AVB: Primary data validation passed. [2025-10-20T17:22:26.376383] [INFO ] [MAIN ] AVB: Fetched primary data is valid. [2025-10-20T17:22:26.376427] [DEBUG] [LOCAL ] AVB: Saving primary data. [2025-10-20T17:22:26.377866] [DEBUG] [LOCAL ] AVB: Trimmed 59219 records to 42300 (keeping 200 days of data) [2025-10-20T17:22:26.667507] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-01. [2025-10-20T17:22:26.678064] [DEBUG] [LOCAL ] AVB: Primary data saved (42300 records). [2025-10-20T17:22:26.678353] [INFO ] [MAIN ] AVB: Full history saved. [2025-10-20T17:22:26.679020] [INFO ] [MAIN ] Update Progress: 186/2367 symbols completed. AVB took 41.8s. Estimated remaining: 4h 22m 19s. Rate: 8.3 symbols/min [2025-10-20T17:22:26.906033] [INFO ] [MAIN ] AWF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:26.912879] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T17:22:26.951334] [DEBUG] [LOCAL ] AWF: Loaded 24387 existing records from primary storage. [2025-10-20T17:22:26.951671] [INFO ] [MAIN ] AWF: Loaded 24387 existing records. [2025-10-20T17:22:27.150039] [WARN ] [MAIN ] AWF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:27.150375] [INFO ] [MAIN ] AWF: Wiping existing primary data before full refresh. [2025-10-20T17:22:27.150774] [INFO ] [LOCAL ] AWF: Starting deletion of primary data. [2025-10-20T17:22:27.151021] [DEBUG] [LOCAL ] AWF: Deleted primary data file. [2025-10-20T17:22:27.151101] [INFO ] [MAIN ] AWF: Starting full primary refresh. [2025-10-20T17:22:27.431270] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:22:27.456502] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-03. [2025-10-20T17:22:27.559807] [INFO ] [REMOTE ] AWF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:28.238732] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:22:28.251356] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-02. [2025-10-20T17:22:29.051434] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-03. [2025-10-20T17:22:29.343385] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T17:22:29.381989] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T17:22:29.851372] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-01. [2025-10-20T17:22:30.492949] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T17:22:30.651392] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-04. [2025-10-20T17:22:31.068685] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T17:22:31.451475] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-02. [2025-10-20T17:22:31.798590] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T17:22:32.251429] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-04. [2025-10-20T17:22:32.543467] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T17:22:33.051377] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-03. [2025-10-20T17:22:33.734488] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:22:33.851448] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-05. [2025-10-20T17:22:34.265352] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T17:22:34.651438] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-03. [2025-10-20T17:22:35.164135] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T17:22:35.451410] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-05. [2025-10-20T17:22:35.837074] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T17:22:36.251353] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-06. [2025-10-20T17:22:37.051352] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-04. [2025-10-20T17:22:37.231574] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T17:22:37.319626] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T17:22:37.851431] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-07. [2025-10-20T17:22:38.660296] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-04. [2025-10-20T17:22:38.663539] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:22:39.139214] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T17:22:39.451470] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-06. [2025-10-20T17:22:39.530819] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-10-20T17:22:40.251448] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-05. [2025-10-20T17:22:41.051387] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-08. [2025-10-20T17:22:41.129112] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T17:22:41.851400] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-07. [2025-10-20T17:22:41.939232] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T17:22:42.517978] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T17:22:42.651379] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-06. [2025-10-20T17:22:43.255274] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T17:22:43.451375] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-09. [2025-10-20T17:22:44.016792] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:22:44.251397] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-05. [2025-10-20T17:22:44.826506] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:22:45.051380] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-10. [2025-10-20T17:22:45.458331] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T17:22:45.851446] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-08. [2025-10-20T17:22:46.356438] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:22:46.363312] [INFO ] [REMOTE ] AVNT: Full fetch complete (56309 records). [2025-10-20T17:22:46.363450] [DEBUG] [VALIDATOR ] AVNT: Validating primary data. [2025-10-20T17:22:46.656530] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-07. [2025-10-20T17:22:46.780444] [DEBUG] [VALIDATOR ] AVNT: All expected trading days are present. [2025-10-20T17:22:46.780513] [DEBUG] [VALIDATOR ] AVNT: Primary data validation passed. [2025-10-20T17:22:46.781245] [INFO ] [MAIN ] AVNT: Fetched primary data is valid. [2025-10-20T17:22:46.781311] [DEBUG] [LOCAL ] AVNT: Saving primary data. [2025-10-20T17:22:46.782570] [DEBUG] [LOCAL ] AVNT: Trimmed 56309 records to 38938 (keeping 200 days of data) [2025-10-20T17:22:47.017901] [DEBUG] [LOCAL ] AVNT: Primary data saved (38938 records). [2025-10-20T17:22:47.018055] [INFO ] [MAIN ] AVNT: Full history saved. [2025-10-20T17:22:47.018136] [INFO ] [MAIN ] Update Progress: 187/2367 symbols completed. AVNT took 29.0s. Estimated remaining: 4h 24m 45s. Rate: 8.2 symbols/min [2025-10-20T17:22:47.213767] [INFO ] [MAIN ] AWI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:47.242135] [DEBUG] [LOCAL ] AWI: Loaded 25722 existing records from primary storage. [2025-10-20T17:22:47.242413] [INFO ] [MAIN ] AWI: Loaded 25722 existing records. [2025-10-20T17:22:47.427777] [WARN ] [MAIN ] AWI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:47.428224] [INFO ] [MAIN ] AWI: Wiping existing primary data before full refresh. [2025-10-20T17:22:47.428385] [INFO ] [LOCAL ] AWI: Starting deletion of primary data. [2025-10-20T17:22:47.428666] [DEBUG] [LOCAL ] AWI: Deleted primary data file. [2025-10-20T17:22:47.428724] [INFO ] [MAIN ] AWI: Starting full primary refresh. [2025-10-20T17:22:47.460362] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-06. [2025-10-20T17:22:47.844608] [INFO ] [REMOTE ] AWI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:47.879758] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-08 (2.03 seconds). [2025-10-20T17:22:48.127400] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T17:22:48.251411] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-08. [2025-10-20T17:22:48.582524] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T17:22:49.051484] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-09. [2025-10-20T17:22:49.498093] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T17:22:49.851399] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-09. [2025-10-20T17:22:50.651469] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-07. [2025-10-20T17:22:50.902439] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:22:51.303141] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T17:22:51.367686] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T17:22:51.451387] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-10. [2025-10-20T17:22:52.251407] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-08. [2025-10-20T17:22:53.033198] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:22:53.042259] [INFO ] [REMOTE ] AVTR: Full fetch complete (77650 records). [2025-10-20T17:22:53.042433] [DEBUG] [VALIDATOR ] AVTR: Validating primary data. [2025-10-20T17:22:53.057245] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-10. [2025-10-20T17:22:53.522607] [DEBUG] [VALIDATOR ] AVTR: All expected trading days are present. [2025-10-20T17:22:53.527978] [DEBUG] [VALIDATOR ] AVTR: Primary data validation passed. [2025-10-20T17:22:53.538199] [INFO ] [MAIN ] AVTR: Fetched primary data is valid. [2025-10-20T17:22:53.538990] [DEBUG] [LOCAL ] AVTR: Saving primary data. [2025-10-20T17:22:53.539675] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T17:22:53.546842] [DEBUG] [LOCAL ] AVTR: Trimmed 77650 records to 53572 (keeping 200 days of data) [2025-10-20T17:22:53.853833] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-01. [2025-10-20T17:22:53.867991] [DEBUG] [LOCAL ] AVTR: Primary data saved (53572 records). [2025-10-20T17:22:53.868204] [INFO ] [MAIN ] AVTR: Full history saved. [2025-10-20T17:22:53.868308] [INFO ] [MAIN ] Update Progress: 188/2367 symbols completed. AVTR took 32.6s. Estimated remaining: 4h 24m 33s. Rate: 8.2 symbols/min [2025-10-20T17:22:54.084715] [INFO ] [MAIN ] AWK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:54.137001] [DEBUG] [LOCAL ] AWK: Loaded 49233 existing records from primary storage. [2025-10-20T17:22:54.137381] [INFO ] [MAIN ] AWK: Loaded 49233 existing records. [2025-10-20T17:22:54.323151] [WARN ] [MAIN ] AWK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:54.323409] [INFO ] [MAIN ] AWK: Wiping existing primary data before full refresh. [2025-10-20T17:22:54.323459] [INFO ] [LOCAL ] AWK: Starting deletion of primary data. [2025-10-20T17:22:54.323729] [DEBUG] [LOCAL ] AWK: Deleted primary data file. [2025-10-20T17:22:54.323786] [INFO ] [MAIN ] AWK: Starting full primary refresh. [2025-10-20T17:22:54.660680] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-09. [2025-10-20T17:22:54.690145] [INFO ] [REMOTE ] AWK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:54.788239] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T17:22:54.901598] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T17:22:54.907981] [INFO ] [REMOTE ] AVY: Full fetch complete (56634 records). [2025-10-20T17:22:54.908105] [DEBUG] [VALIDATOR ] AVY: Validating primary data. [2025-10-20T17:22:55.297431] [DEBUG] [VALIDATOR ] AVY: All expected trading days are present. [2025-10-20T17:22:55.297565] [DEBUG] [VALIDATOR ] AVY: Primary data validation passed. [2025-10-20T17:22:55.298211] [INFO ] [MAIN ] AVY: Fetched primary data is valid. [2025-10-20T17:22:55.298275] [DEBUG] [LOCAL ] AVY: Saving primary data. [2025-10-20T17:22:55.299581] [DEBUG] [LOCAL ] AVY: Trimmed 56634 records to 38172 (keeping 200 days of data) [2025-10-20T17:22:55.479482] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-02. [2025-10-20T17:22:55.576521] [DEBUG] [LOCAL ] AVY: Primary data saved (38172 records). [2025-10-20T17:22:55.578037] [INFO ] [MAIN ] AVY: Full history saved. [2025-10-20T17:22:55.577920] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T17:22:55.578226] [INFO ] [MAIN ] Update Progress: 189/2367 symbols completed. AVY took 32.0s. Estimated remaining: 4h 23m 21s. Rate: 8.3 symbols/min [2025-10-20T17:22:55.831290] [INFO ] [MAIN ] AWP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:55.863086] [DEBUG] [LOCAL ] AWP: Loaded 21887 existing records from primary storage. [2025-10-20T17:22:55.863480] [INFO ] [MAIN ] AWP: Loaded 21887 existing records. [2025-10-20T17:22:56.144371] [WARN ] [MAIN ] AWP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:56.144681] [INFO ] [MAIN ] AWP: Wiping existing primary data before full refresh. [2025-10-20T17:22:56.144745] [INFO ] [LOCAL ] AWP: Starting deletion of primary data. [2025-10-20T17:22:56.144985] [DEBUG] [LOCAL ] AWP: Deleted primary data file. [2025-10-20T17:22:56.145042] [INFO ] [MAIN ] AWP: Starting full primary refresh. [2025-10-20T17:22:56.256838] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-10. [2025-10-20T17:22:56.621795] [INFO ] [REMOTE ] AWP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:56.700975] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:22:57.051444] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-01. [2025-10-20T17:22:57.173364] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T17:22:57.177426] [INFO ] [REMOTE ] AWF: Full fetch complete (35941 records). [2025-10-20T17:22:57.177525] [DEBUG] [VALIDATOR ] AWF: Validating primary data. [2025-10-20T17:22:57.576659] [DEBUG] [VALIDATOR ] AWF: All expected trading days are present. [2025-10-20T17:22:57.576754] [DEBUG] [VALIDATOR ] AWF: Primary data validation passed. [2025-10-20T17:22:57.577406] [INFO ] [MAIN ] AWF: Fetched primary data is valid. [2025-10-20T17:22:57.577502] [DEBUG] [LOCAL ] AWF: Saving primary data. [2025-10-20T17:22:57.578710] [DEBUG] [LOCAL ] AWF: Trimmed 35941 records to 24036 (keeping 200 days of data) [2025-10-20T17:22:57.728191] [DEBUG] [LOCAL ] AWF: Primary data saved (24036 records). [2025-10-20T17:22:57.728334] [INFO ] [MAIN ] AWF: Full history saved. [2025-10-20T17:22:57.728443] [INFO ] [MAIN ] Update Progress: 190/2367 symbols completed. AWF took 31.0s. Estimated remaining: 4h 22m 15s. Rate: 8.3 symbols/min [2025-10-20T17:22:57.856485] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-01. [2025-10-20T17:22:57.913631] [INFO ] [MAIN ] AWR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:57.934858] [DEBUG] [LOCAL ] AWR: Loaded 23114 existing records from primary storage. [2025-10-20T17:22:57.935158] [INFO ] [MAIN ] AWR: Loaded 23114 existing records. [2025-10-20T17:22:58.127286] [WARN ] [MAIN ] AWR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:58.127534] [INFO ] [MAIN ] AWR: Wiping existing primary data before full refresh. [2025-10-20T17:22:58.127611] [INFO ] [LOCAL ] AWR: Starting deletion of primary data. [2025-10-20T17:22:58.127847] [DEBUG] [LOCAL ] AWR: Deleted primary data file. [2025-10-20T17:22:58.127939] [INFO ] [MAIN ] AWR: Starting full primary refresh. [2025-10-20T17:22:58.393059] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:22:58.556331] [INFO ] [REMOTE ] AWR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:58.651425] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-02. [2025-10-20T17:22:59.451371] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-03. [2025-10-20T17:22:59.668713] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T17:22:59.754234] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T17:23:00.251429] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-01. [2025-10-20T17:23:00.707564] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:23:01.051400] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-02. [2025-10-20T17:23:01.574120] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:23:01.851434] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-03. [2025-10-20T17:23:02.647988] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:23:02.656902] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-04. [2025-10-20T17:23:02.696298] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T17:23:03.451405] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-04. [2025-10-20T17:23:03.987621] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:23:04.251425] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-02. [2025-10-20T17:23:04.482288] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T17:23:05.051432] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-03. [2025-10-20T17:23:05.362556] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T17:23:05.851421] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-03. [2025-10-20T17:23:06.651365] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-05. [2025-10-20T17:23:06.839395] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T17:23:07.081156] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T17:23:07.451596] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-04. [2025-10-20T17:23:07.552793] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T17:23:08.251398] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-04. [2025-10-20T17:23:09.051409] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-05. [2025-10-20T17:23:09.319343] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T17:23:09.369153] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T17:23:09.851601] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-05. [2025-10-20T17:23:10.354543] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T17:23:10.651469] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-06. [2025-10-20T17:23:10.996621] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T17:23:11.451389] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-05. [2025-10-20T17:23:11.803730] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T17:23:12.251413] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-06. [2025-10-20T17:23:13.051378] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-06. [2025-10-20T17:23:13.353448] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T17:23:13.395241] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:23:13.851393] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-07. [2025-10-20T17:23:13.973122] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T17:23:14.651405] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-07. [2025-10-20T17:23:15.164332] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T17:23:15.451369] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-08. [2025-10-20T17:23:15.604425] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:23:16.251368] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-07. [2025-10-20T17:23:16.342136] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T17:23:17.051385] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-06. [2025-10-20T17:23:17.384929] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T17:23:17.851384] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-08. [2025-10-20T17:23:18.347145] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T17:23:18.651399] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-07. [2025-10-20T17:23:18.985293] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:23:19.451425] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-09. [2025-10-20T17:23:20.049218] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T17:23:20.251425] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-08. [2025-10-20T17:23:20.475497] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:23:21.051377] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-09. [2025-10-20T17:23:21.263952] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:23:21.851416] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-09. [2025-10-20T17:23:22.186639] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T17:23:22.651356] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-10. [2025-10-20T17:23:22.981393] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T17:23:23.451397] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-10. [2025-10-20T17:23:23.782730] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T17:23:23.786833] [INFO ] [REMOTE ] AWP: Full fetch complete (31702 records). [2025-10-20T17:23:23.786938] [DEBUG] [VALIDATOR ] AWP: Validating primary data. [2025-10-20T17:23:23.787568] [FATAL] [VALIDATOR ] AWP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:23:23.787688] [FATAL] [MAIN ] AWP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:23:23.787759] [WARN ] [MAIN ] AWP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:23:23.787838] [INFO ] [LOCAL ] AWP: Starting deletion of primary data. [2025-10-20T17:23:23.787935] [DEBUG] [LOCAL ] AWP: No primary data file found to delete. [2025-10-20T17:23:23.787992] [INFO ] [LOCAL ] AWP: Starting deletion of derivative data. [2025-10-20T17:23:23.788243] [DEBUG] [LOCAL ] AWP: Deleted derivative data file. [2025-10-20T17:23:23.788354] [INFO ] [MAIN ] Update Progress: 191/2367 symbols completed. AWP took 28.2s. Estimated remaining: 4h 25m 43s. Rate: 8.2 symbols/min [2025-10-20T17:23:24.001257] [INFO ] [MAIN ] AX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:24.001453] [DEBUG] [LOCAL ] AX: No existing data file found in primary storage. [2025-10-20T17:23:24.001490] [WARN ] [MAIN ] AX: No existing data found, performing full refresh instead. [2025-10-20T17:23:24.001629] [INFO ] [MAIN ] AX: Wiping existing primary data before full refresh. [2025-10-20T17:23:24.001706] [INFO ] [LOCAL ] AX: Starting deletion of primary data. [2025-10-20T17:23:24.001758] [DEBUG] [LOCAL ] AX: No primary data file found to delete. [2025-10-20T17:23:24.001790] [INFO ] [MAIN ] AX: Starting full primary refresh. [2025-10-20T17:23:24.256545] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-10. [2025-10-20T17:23:24.360122] [INFO ] [REMOTE ] AX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:24.601988] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:23:24.607348] [INFO ] [REMOTE ] AWR: Full fetch complete (32381 records). [2025-10-20T17:23:24.607480] [DEBUG] [VALIDATOR ] AWR: Validating primary data. [2025-10-20T17:23:24.988467] [DEBUG] [VALIDATOR ] AWR: All expected trading days are present. [2025-10-20T17:23:24.988589] [DEBUG] [VALIDATOR ] AWR: Primary data validation passed. [2025-10-20T17:23:24.989143] [INFO ] [MAIN ] AWR: Fetched primary data is valid. [2025-10-20T17:23:24.989210] [DEBUG] [LOCAL ] AWR: Saving primary data. [2025-10-20T17:23:24.990467] [DEBUG] [LOCAL ] AWR: Trimmed 32381 records to 22778 (keeping 200 days of data) [2025-10-20T17:23:25.052818] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-01. [2025-10-20T17:23:25.157314] [DEBUG] [LOCAL ] AWR: Primary data saved (22778 records). [2025-10-20T17:23:25.157543] [INFO ] [MAIN ] AWR: Full history saved. [2025-10-20T17:23:25.157673] [INFO ] [MAIN ] Update Progress: 192/2367 symbols completed. AWR took 27.4s. Estimated remaining: 4h 24m 28s. Rate: 8.2 symbols/min [2025-10-20T17:23:25.345770] [INFO ] [MAIN ] AXL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:25.404781] [DEBUG] [LOCAL ] AXL: Loaded 51268 existing records from primary storage. [2025-10-20T17:23:25.405286] [INFO ] [MAIN ] AXL: Loaded 51268 existing records. [2025-10-20T17:23:25.550242] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T17:23:25.576858] [INFO ] [REMOTE ] AWI: Full fetch complete (36037 records). [2025-10-20T17:23:25.592499] [DEBUG] [VALIDATOR ] AWI: Validating primary data. [2025-10-20T17:23:25.752145] [WARN ] [MAIN ] AXL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:25.754722] [INFO ] [MAIN ] AXL: Wiping existing primary data before full refresh. [2025-10-20T17:23:25.754797] [INFO ] [LOCAL ] AXL: Starting deletion of primary data. [2025-10-20T17:23:25.760437] [DEBUG] [LOCAL ] AXL: Deleted primary data file. [2025-10-20T17:23:25.776013] [INFO ] [MAIN ] AXL: Starting full primary refresh. [2025-10-20T17:23:25.877789] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-08. [2025-10-20T17:23:26.404267] [DEBUG] [VALIDATOR ] AWI: All expected trading days are present. [2025-10-20T17:23:26.404004] [INFO ] [REMOTE ] AXL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:26.404785] [DEBUG] [VALIDATOR ] AWI: Primary data validation passed. [2025-10-20T17:23:26.405364] [INFO ] [MAIN ] AWI: Fetched primary data is valid. [2025-10-20T17:23:26.405448] [DEBUG] [LOCAL ] AWI: Saving primary data. [2025-10-20T17:23:26.406391] [DEBUG] [LOCAL ] AWI: Trimmed 36037 records to 25285 (keeping 200 days of data) [2025-10-20T17:23:26.566627] [DEBUG] [LOCAL ] AWI: Primary data saved (25285 records). [2025-10-20T17:23:26.566894] [INFO ] [MAIN ] AWI: Full history saved. [2025-10-20T17:23:26.567044] [INFO ] [MAIN ] Update Progress: 193/2367 symbols completed. AWI took 39.5s. Estimated remaining: 4h 23m 14s. Rate: 8.3 symbols/min [2025-10-20T17:23:26.658983] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-01. [2025-10-20T17:23:26.754873] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T17:23:26.792668] [INFO ] [MAIN ] AXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:26.848149] [DEBUG] [LOCAL ] AXP: Loaded 53945 existing records from primary storage. [2025-10-20T17:23:26.848620] [INFO ] [MAIN ] AXP: Loaded 53945 existing records. [2025-10-20T17:23:27.026046] [WARN ] [MAIN ] AXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:27.026248] [INFO ] [MAIN ] AXP: Wiping existing primary data before full refresh. [2025-10-20T17:23:27.026292] [INFO ] [LOCAL ] AXP: Starting deletion of primary data. [2025-10-20T17:23:27.026511] [DEBUG] [LOCAL ] AXP: Deleted primary data file. [2025-10-20T17:23:27.026566] [INFO ] [MAIN ] AXP: Starting full primary refresh. [2025-10-20T17:23:27.371010] [INFO ] [REMOTE ] AXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:27.451390] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-02. [2025-10-20T17:23:27.781785] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T17:23:28.251413] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-01. [2025-10-20T17:23:28.463194] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:23:28.682734] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:23:29.051401] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-02. [2025-10-20T17:23:29.851419] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-03. [2025-10-20T17:23:30.651401] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-09. [2025-10-20T17:23:30.701685] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-01 (2.45 seconds). [2025-10-20T17:23:30.750621] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T17:23:31.171411] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T17:23:31.451454] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-03. [2025-10-20T17:23:32.014205] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T17:23:32.251380] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-02. [2025-10-20T17:23:33.051441] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-10. [2025-10-20T17:23:33.121363] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:23:33.851434] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-04. [2025-10-20T17:23:34.227881] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T17:23:34.651388] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-04. [2025-10-20T17:23:34.789578] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:23:34.797510] [INFO ] [REMOTE ] AWK: Full fetch complete (70910 records). [2025-10-20T17:23:34.797678] [DEBUG] [VALIDATOR ] AWK: Validating primary data. [2025-10-20T17:23:35.279032] [DEBUG] [VALIDATOR ] AWK: All expected trading days are present. [2025-10-20T17:23:35.279823] [DEBUG] [VALIDATOR ] AWK: Primary data validation passed. [2025-10-20T17:23:35.280581] [INFO ] [MAIN ] AWK: Fetched primary data is valid. [2025-10-20T17:23:35.281933] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T17:23:35.282044] [DEBUG] [LOCAL ] AWK: Saving primary data. [2025-10-20T17:23:35.291964] [DEBUG] [LOCAL ] AWK: Trimmed 70910 records to 48510 (keeping 200 days of data) [2025-10-20T17:23:35.474820] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-05. [2025-10-20T17:23:35.599627] [DEBUG] [LOCAL ] AWK: Primary data saved (48510 records). [2025-10-20T17:23:35.599803] [INFO ] [MAIN ] AWK: Full history saved. [2025-10-20T17:23:35.600493] [INFO ] [MAIN ] Update Progress: 194/2367 symbols completed. AWK took 41.7s. Estimated remaining: 4h 23m 27s. Rate: 8.2 symbols/min [2025-10-20T17:23:35.768277] [INFO ] [MAIN ] AXR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:35.773116] [DEBUG] [LOCAL ] AXR: Loaded 2975 existing records from primary storage. [2025-10-20T17:23:35.773474] [INFO ] [MAIN ] AXR: Loaded 2975 existing records. [2025-10-20T17:23:35.955791] [WARN ] [MAIN ] AXR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:35.956099] [INFO ] [MAIN ] AXR: Wiping existing primary data before full refresh. [2025-10-20T17:23:35.956156] [INFO ] [LOCAL ] AXR: Starting deletion of primary data. [2025-10-20T17:23:35.956283] [DEBUG] [LOCAL ] AXR: Deleted primary data file. [2025-10-20T17:23:35.956321] [INFO ] [MAIN ] AXR: Starting full primary refresh. [2025-10-20T17:23:36.274498] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-03. [2025-10-20T17:23:36.356484] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:23:36.428844] [INFO ] [REMOTE ] AXR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:36.460265] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T17:23:37.051465] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-06. [2025-10-20T17:23:37.851430] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-01. [2025-10-20T17:23:37.986346] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T17:23:38.449866] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-10-20T17:23:38.565229] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T17:23:38.651397] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-02. [2025-10-20T17:23:39.117826] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T17:23:39.451452] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-07. [2025-10-20T17:23:40.251455] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-04. [2025-10-20T17:23:40.687936] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T17:23:41.051455] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-03. [2025-10-20T17:23:41.622159] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T17:23:41.851425] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-08. [2025-10-20T17:23:42.618152] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-04 (2.37 seconds). [2025-10-20T17:23:42.656076] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-05. [2025-10-20T17:23:42.664108] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T17:23:43.451444] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-09. [2025-10-20T17:23:44.106993] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T17:23:44.251418] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-05. [2025-10-20T17:23:44.772809] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T17:23:45.051423] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-04. [2025-10-20T17:23:45.777137] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-10-20T17:23:45.851389] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-05. [2025-10-20T17:23:46.382237] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T17:23:46.541140] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T17:23:46.651440] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-06. [2025-10-20T17:23:47.451500] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-10. [2025-10-20T17:23:48.239868] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T17:23:48.251325] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-07. [2025-10-20T17:23:48.862860] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:23:48.868030] [INFO ] [REMOTE ] AX: Full fetch complete (42245 records). [2025-10-20T17:23:48.868204] [DEBUG] [VALIDATOR ] AX: Validating primary data. [2025-10-20T17:23:49.054432] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-06. [2025-10-20T17:23:49.267917] [DEBUG] [VALIDATOR ] AX: All expected trading days are present. [2025-10-20T17:23:49.268012] [DEBUG] [VALIDATOR ] AX: Primary data validation passed. [2025-10-20T17:23:49.268634] [INFO ] [MAIN ] AX: Fetched primary data is valid. [2025-10-20T17:23:49.268730] [DEBUG] [LOCAL ] AX: Saving primary data. [2025-10-20T17:23:49.269845] [DEBUG] [LOCAL ] AX: Trimmed 42245 records to 28103 (keeping 200 days of data) [2025-10-20T17:23:49.431054] [DEBUG] [LOCAL ] AX: Primary data saved (28103 records). [2025-10-20T17:23:49.431196] [INFO ] [MAIN ] AX: Full history saved. [2025-10-20T17:23:49.431270] [INFO ] [MAIN ] Update Progress: 195/2367 symbols completed. AX took 25.6s. Estimated remaining: 4h 24m 33s. Rate: 8.2 symbols/min [2025-10-20T17:23:49.600392] [INFO ] [MAIN ] AXS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:49.635254] [DEBUG] [LOCAL ] AXS: Loaded 39196 existing records from primary storage. [2025-10-20T17:23:49.635564] [INFO ] [MAIN ] AXS: Loaded 39196 existing records. [2025-10-20T17:23:49.819682] [WARN ] [MAIN ] AXS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:49.820142] [INFO ] [MAIN ] AXS: Wiping existing primary data before full refresh. [2025-10-20T17:23:49.820252] [INFO ] [LOCAL ] AXS: Starting deletion of primary data. [2025-10-20T17:23:49.820461] [DEBUG] [LOCAL ] AXS: Deleted primary data file. [2025-10-20T17:23:49.820528] [INFO ] [MAIN ] AXS: Starting full primary refresh. [2025-10-20T17:23:49.852529] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-06. [2025-10-20T17:23:49.931720] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T17:23:50.046152] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:23:50.248900] [INFO ] [REMOTE ] AXS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:50.651437] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-01. [2025-10-20T17:23:51.451390] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-07. [2025-10-20T17:23:51.497914] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T17:23:52.072758] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-10-20T17:23:52.251398] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-07. [2025-10-20T17:23:52.342788] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:23:53.051425] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-08. [2025-10-20T17:23:53.475460] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T17:23:53.851347] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-09. [2025-10-20T17:23:54.257197] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T17:23:54.535741] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-07 (2.28 seconds). [2025-10-20T17:23:54.651434] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-02. [2025-10-20T17:23:55.451376] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-08. [2025-10-20T17:23:55.749144] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:23:56.251394] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-10. [2025-10-20T17:23:56.761831] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T17:23:56.762988] [INFO ] [REMOTE ] AXR: Full fetch complete (4413 records). [2025-10-20T17:23:56.763066] [DEBUG] [VALIDATOR ] AXR: Validating primary data. [2025-10-20T17:23:57.056855] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-08. [2025-10-20T17:23:57.152333] [DEBUG] [VALIDATOR ] AXR: All expected trading days are present. [2025-10-20T17:23:57.152472] [DEBUG] [VALIDATOR ] AXR: Primary data validation passed. [2025-10-20T17:23:57.152733] [INFO ] [MAIN ] AXR: Fetched primary data is valid. [2025-10-20T17:23:57.152806] [DEBUG] [LOCAL ] AXR: Saving primary data. [2025-10-20T17:23:57.153183] [DEBUG] [LOCAL ] AXR: Trimmed 4413 records to 2925 (keeping 200 days of data) [2025-10-20T17:23:57.170471] [DEBUG] [LOCAL ] AXR: Primary data saved (2925 records). [2025-10-20T17:23:57.170684] [INFO ] [MAIN ] AXR: Full history saved. [2025-10-20T17:23:57.170788] [INFO ] [MAIN ] Update Progress: 196/2367 symbols completed. AXR took 21.6s. Estimated remaining: 4h 24m 30s. Rate: 8.2 symbols/min [2025-10-20T17:23:57.352863] [INFO ] [MAIN ] AXTA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:57.413562] [DEBUG] [LOCAL ] AXTA: Loaded 53031 existing records from primary storage. [2025-10-20T17:23:57.413959] [INFO ] [MAIN ] AXTA: Loaded 53031 existing records. [2025-10-20T17:23:57.596708] [WARN ] [MAIN ] AXTA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:57.597100] [INFO ] [MAIN ] AXTA: Wiping existing primary data before full refresh. [2025-10-20T17:23:57.597292] [INFO ] [LOCAL ] AXTA: Starting deletion of primary data. [2025-10-20T17:23:57.597610] [DEBUG] [LOCAL ] AXTA: Deleted primary data file. [2025-10-20T17:23:57.597711] [INFO ] [MAIN ] AXTA: Starting full primary refresh. [2025-10-20T17:23:57.858455] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-03. [2025-10-20T17:23:57.951574] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-08 (2.50 seconds). [2025-10-20T17:23:58.059160] [INFO ] [REMOTE ] AXTA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:58.567634] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:23:58.651440] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-01. [2025-10-20T17:23:59.451456] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-09. [2025-10-20T17:23:59.531994] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:24:00.106463] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T17:24:00.251389] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-02. [2025-10-20T17:24:01.051523] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-04. [2025-10-20T17:24:01.170677] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T17:24:01.851346] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-09. [2025-10-20T17:24:02.082676] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T17:24:02.419967] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T17:24:02.651439] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-05. [2025-10-20T17:24:03.451401] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-03. [2025-10-20T17:24:03.938956] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-10-20T17:24:04.151130] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T17:24:04.251352] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-06. [2025-10-20T17:24:05.056500] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-10. [2025-10-20T17:24:05.069304] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:24:05.767053] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:24:05.851387] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-07. [2025-10-20T17:24:06.651387] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-04. [2025-10-20T17:24:06.928093] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T17:24:07.456631] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-08. [2025-10-20T17:24:07.490001] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-10 (2.43 seconds). [2025-10-20T17:24:07.500981] [INFO ] [REMOTE ] AXP: Full fetch complete (77124 records). [2025-10-20T17:24:07.501211] [DEBUG] [VALIDATOR ] AXP: Validating primary data. [2025-10-20T17:24:08.011830] [DEBUG] [VALIDATOR ] AXP: All expected trading days are present. [2025-10-20T17:24:08.012047] [DEBUG] [VALIDATOR ] AXP: Primary data validation passed. [2025-10-20T17:24:08.013188] [INFO ] [MAIN ] AXP: Fetched primary data is valid. [2025-10-20T17:24:08.013410] [DEBUG] [LOCAL ] AXP: Saving primary data. [2025-10-20T17:24:08.015828] [DEBUG] [LOCAL ] AXP: Trimmed 77124 records to 53169 (keeping 200 days of data) [2025-10-20T17:24:08.259616] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-10. [2025-10-20T17:24:08.432860] [DEBUG] [LOCAL ] AXP: Primary data saved (53169 records). [2025-10-20T17:24:08.437176] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:24:08.437393] [INFO ] [MAIN ] AXP: Full history saved. [2025-10-20T17:24:08.448787] [INFO ] [MAIN ] Update Progress: 197/2367 symbols completed. AXP took 41.9s. Estimated remaining: 4h 25m 6s. Rate: 8.2 symbols/min [2025-10-20T17:24:08.617259] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T17:24:09.051459] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-05. [2025-10-20T17:24:09.851414] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-09. [2025-10-20T17:24:10.180843] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:24:10.191383] [INFO ] [REMOTE ] AXL: Full fetch complete (73085 records). [2025-10-20T17:24:10.191685] [DEBUG] [VALIDATOR ] AXL: Validating primary data. [2025-10-20T17:24:10.618141] [DEBUG] [VALIDATOR ] AXL: All expected trading days are present. [2025-10-20T17:24:10.618291] [DEBUG] [VALIDATOR ] AXL: Primary data validation passed. [2025-10-20T17:24:10.619354] [INFO ] [MAIN ] AXL: Fetched primary data is valid. [2025-10-20T17:24:10.619467] [DEBUG] [LOCAL ] AXL: Saving primary data. [2025-10-20T17:24:10.621692] [DEBUG] [LOCAL ] AXL: Trimmed 73085 records to 50539 (keeping 200 days of data) [2025-10-20T17:24:10.766034] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:24:10.787209] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-06. [2025-10-20T17:24:10.982057] [DEBUG] [LOCAL ] AXL: Primary data saved (50539 records). [2025-10-20T17:24:10.982419] [INFO ] [MAIN ] AXL: Full history saved. [2025-10-20T17:24:10.982546] [INFO ] [MAIN ] Update Progress: 198/2367 symbols completed. AXL took 45.8s. Estimated remaining: 4h 24m 7s. Rate: 8.2 symbols/min [2025-10-20T17:24:11.027923] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T17:24:11.451397] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-10. [2025-10-20T17:24:12.569678] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:24:12.581241] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-07. [2025-10-20T17:24:13.085732] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:24:13.096083] [INFO ] [REMOTE ] AXS: Full fetch complete (56020 records). [2025-10-20T17:24:13.096346] [DEBUG] [VALIDATOR ] AXS: Validating primary data. [2025-10-20T17:24:13.521440] [DEBUG] [VALIDATOR ] AXS: All expected trading days are present. [2025-10-20T17:24:13.521565] [DEBUG] [VALIDATOR ] AXS: Primary data validation passed. [2025-10-20T17:24:13.522448] [INFO ] [MAIN ] AXS: Fetched primary data is valid. [2025-10-20T17:24:13.522519] [DEBUG] [LOCAL ] AXS: Saving primary data. [2025-10-20T17:24:13.524385] [DEBUG] [LOCAL ] AXS: Trimmed 56020 records to 38702 (keeping 200 days of data) [2025-10-20T17:24:13.776292] [DEBUG] [LOCAL ] AXS: Primary data saved (38702 records). [2025-10-20T17:24:13.776463] [INFO ] [MAIN ] AXS: Full history saved. [2025-10-20T17:24:13.776554] [INFO ] [MAIN ] Update Progress: 199/2367 symbols completed. AXS took 24.3s. Estimated remaining: 4h 23m 10s. Rate: 8.2 symbols/min [2025-10-20T17:24:14.206416] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T17:24:14.218450] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-08. [2025-10-20T17:24:15.838324] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T17:24:15.851003] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-09. [2025-10-20T17:24:17.632983] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:24:17.645174] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-10. [2025-10-20T17:24:19.557910] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T17:24:19.566786] [INFO ] [REMOTE ] AXTA: Full fetch complete (75415 records). [2025-10-20T17:24:19.567002] [DEBUG] [VALIDATOR ] AXTA: Validating primary data. [2025-10-20T17:24:19.992164] [DEBUG] [VALIDATOR ] AXTA: All expected trading days are present. [2025-10-20T17:24:19.992327] [DEBUG] [VALIDATOR ] AXTA: Primary data validation passed. [2025-10-20T17:24:19.993209] [INFO ] [MAIN ] AXTA: Fetched primary data is valid. [2025-10-20T17:24:19.993278] [DEBUG] [LOCAL ] AXTA: Saving primary data. [2025-10-20T17:24:19.995996] [DEBUG] [LOCAL ] AXTA: Trimmed 75415 records to 52286 (keeping 200 days of data) [2025-10-20T17:24:20.317385] [DEBUG] [LOCAL ] AXTA: Primary data saved (52286 records). [2025-10-20T17:24:20.317540] [INFO ] [MAIN ] AXTA: Full history saved. [2025-10-20T17:24:20.317621] [INFO ] [MAIN ] Update Progress: 200/2367 symbols completed. AXTA took 23.1s. Estimated remaining: 4h 22m 55s. Rate: 8.2 symbols/min [2025-10-20T17:24:21.104042] [INFO ] [MAIN ] AZO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:21.175934] [INFO ] [MAIN ] B: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:21.170600] [INFO ] [MAIN ] AZZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:21.178701] [INFO ] [MAIN ] AYI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:21.176120] [DEBUG] [LOCAL ] B: No existing data file found in primary storage. [2025-10-20T17:24:21.178869] [WARN ] [MAIN ] B: No existing data found, performing full refresh instead. [2025-10-20T17:24:21.179450] [INFO ] [MAIN ] B: Wiping existing primary data before full refresh. [2025-10-20T17:24:21.179688] [INFO ] [LOCAL ] B: Starting deletion of primary data. [2025-10-20T17:24:21.179785] [DEBUG] [LOCAL ] B: No primary data file found to delete. [2025-10-20T17:24:21.179911] [INFO ] [MAIN ] B: Starting full primary refresh. [2025-10-20T17:24:21.261346] [DEBUG] [LOCAL ] AZO: Loaded 11148 existing records from primary storage. [2025-10-20T17:24:21.279422] [INFO ] [MAIN ] AZO: Loaded 11148 existing records. [2025-10-20T17:24:21.485873] [DEBUG] [LOCAL ] AYI: Loaded 25175 existing records from primary storage. [2025-10-20T17:24:21.550966] [DEBUG] [LOCAL ] AZZ: Loaded 23075 existing records from primary storage. [2025-10-20T17:24:21.565828] [INFO ] [MAIN ] AYI: Loaded 25175 existing records. [2025-10-20T17:24:21.701082] [INFO ] [MAIN ] AZZ: Loaded 23075 existing records. [2025-10-20T17:24:21.787695] [WARN ] [MAIN ] AZO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:21.922569] [INFO ] [MAIN ] AZO: Wiping existing primary data before full refresh. [2025-10-20T17:24:21.998201] [INFO ] [LOCAL ] AZO: Starting deletion of primary data. [2025-10-20T17:24:22.083593] [DEBUG] [LOCAL ] AZO: Deleted primary data file. [2025-10-20T17:24:22.130941] [INFO ] [MAIN ] AZO: Starting full primary refresh. [2025-10-20T17:24:22.213431] [WARN ] [MAIN ] AYI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:22.270768] [INFO ] [REMOTE ] B: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:22.281098] [INFO ] [MAIN ] AYI: Wiping existing primary data before full refresh. [2025-10-20T17:24:22.312961] [INFO ] [REMOTE ] B: Fetching primary data for 2025-01. [2025-10-20T17:24:22.323350] [INFO ] [LOCAL ] AYI: Starting deletion of primary data. [2025-10-20T17:24:22.346207] [DEBUG] [LOCAL ] AYI: Deleted primary data file. [2025-10-20T17:24:22.355860] [INFO ] [MAIN ] AYI: Starting full primary refresh. [2025-10-20T17:24:22.524521] [WARN ] [MAIN ] AZZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:22.540482] [INFO ] [MAIN ] AZZ: Wiping existing primary data before full refresh. [2025-10-20T17:24:22.540674] [INFO ] [LOCAL ] AZZ: Starting deletion of primary data. [2025-10-20T17:24:22.604692] [DEBUG] [LOCAL ] AZZ: Deleted primary data file. [2025-10-20T17:24:22.610126] [INFO ] [MAIN ] AZZ: Starting full primary refresh. [2025-10-20T17:24:23.432506] [INFO ] [REMOTE ] AZO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:23.465317] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-01. [2025-10-20T17:24:23.694114] [INFO ] [REMOTE ] AYI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:23.690472] [INFO ] [REMOTE ] AZZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:23.694602] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-01. [2025-10-20T17:24:23.694831] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-01. [2025-10-20T17:24:23.728299] [INFO ] [REMOTE ] B: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:24:23.735206] [INFO ] [REMOTE ] B: Fetching primary data for 2025-02. [2025-10-20T17:24:23.943137] [DEBUG] [REMOTE ] B: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:24:23.943561] [WARN ] [REMOTE ] B: No data returned for month 2025-02 [2025-10-20T17:24:23.943680] [FATAL] [REMOTE ] B: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:24:23.943777] [FATAL] [MAIN ] B: Failed to fetch primary data. [2025-10-20T17:24:23.943825] [WARN ] [MAIN ] B: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:24:23.943868] [INFO ] [LOCAL ] B: Starting deletion of primary data. [2025-10-20T17:24:23.943934] [DEBUG] [LOCAL ] B: No primary data file found to delete. [2025-10-20T17:24:23.943971] [INFO ] [LOCAL ] B: Starting deletion of derivative data. [2025-10-20T17:24:23.944011] [DEBUG] [LOCAL ] B: No derivative data file found to delete. [2025-10-20T17:24:23.944084] [INFO ] [MAIN ] Update Progress: 201/2367 symbols completed. B took 3.6s. Estimated remaining: 4h 22m 8s. Rate: 8.3 symbols/min [2025-10-20T17:24:24.130199] [INFO ] [MAIN ] BA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:24.190725] [DEBUG] [LOCAL ] BA: Loaded 54570 existing records from primary storage. [2025-10-20T17:24:24.191137] [INFO ] [MAIN ] BA: Loaded 54570 existing records. [2025-10-20T17:24:24.401091] [WARN ] [MAIN ] BA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:24.402382] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T17:24:24.407178] [INFO ] [MAIN ] BA: Wiping existing primary data before full refresh. [2025-10-20T17:24:24.407082] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-02. [2025-10-20T17:24:24.408535] [INFO ] [LOCAL ] BA: Starting deletion of primary data. [2025-10-20T17:24:24.408848] [DEBUG] [LOCAL ] BA: Deleted primary data file. [2025-10-20T17:24:24.408910] [INFO ] [MAIN ] BA: Starting full primary refresh. [2025-10-20T17:24:24.817599] [INFO ] [REMOTE ] BA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:24.818253] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-01. [2025-10-20T17:24:24.833990] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T17:24:24.838893] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-02. [2025-10-20T17:24:25.276271] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T17:24:25.279057] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-03. [2025-10-20T17:24:25.358735] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:24:25.366042] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-02. [2025-10-20T17:24:25.915835] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:24:25.923218] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-03. [2025-10-20T17:24:26.371807] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:24:26.376058] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-04. [2025-10-20T17:24:26.707613] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T17:24:26.714032] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-04. [2025-10-20T17:24:26.824790] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T17:24:26.835954] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-02. [2025-10-20T17:24:26.912807] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:24:26.918018] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-03. [2025-10-20T17:24:27.390842] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T17:24:27.451461] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-05. [2025-10-20T17:24:28.011500] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T17:24:28.146477] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T17:24:28.251555] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-05. [2025-10-20T17:24:28.401615] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T17:24:28.657473] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T17:24:29.051467] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-04. [2025-10-20T17:24:29.312103] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T17:24:29.851401] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-06. [2025-10-20T17:24:30.510198] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T17:24:30.539556] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T17:24:30.651595] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-05. [2025-10-20T17:24:31.451386] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-07. [2025-10-20T17:24:31.509047] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T17:24:32.251612] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-06. [2025-10-20T17:24:32.857293] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T17:24:33.051443] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-06. [2025-10-20T17:24:33.534269] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T17:24:33.851450] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-03. [2025-10-20T17:24:33.996884] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T17:24:34.651466] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-08. [2025-10-20T17:24:35.451503] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-07. [2025-10-20T17:24:35.809181] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T17:24:36.258698] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-09. [2025-10-20T17:24:36.269001] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-03 (2.42 seconds). [2025-10-20T17:24:36.416189] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T17:24:37.051440] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-04. [2025-10-20T17:24:37.398882] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T17:24:37.851396] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-10. [2025-10-20T17:24:38.651453] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-07. [2025-10-20T17:24:39.241169] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:24:39.246682] [INFO ] [REMOTE ] AZZ: Full fetch complete (32093 records). [2025-10-20T17:24:39.246855] [DEBUG] [VALIDATOR ] AZZ: Validating primary data. [2025-10-20T17:24:39.461046] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-08. [2025-10-20T17:24:39.706626] [DEBUG] [VALIDATOR ] AZZ: All expected trading days are present. [2025-10-20T17:24:39.706879] [DEBUG] [VALIDATOR ] AZZ: Primary data validation passed. [2025-10-20T17:24:39.707515] [INFO ] [MAIN ] AZZ: Fetched primary data is valid. [2025-10-20T17:24:39.707602] [DEBUG] [LOCAL ] AZZ: Saving primary data. [2025-10-20T17:24:39.709392] [DEBUG] [LOCAL ] AZZ: Trimmed 32093 records to 22882 (keeping 200 days of data) [2025-10-20T17:24:39.865020] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-04 (2.81 seconds). [2025-10-20T17:24:39.930467] [DEBUG] [LOCAL ] AZZ: Primary data saved (22882 records). [2025-10-20T17:24:39.930893] [INFO ] [MAIN ] AZZ: Full history saved. [2025-10-20T17:24:39.931118] [INFO ] [MAIN ] Update Progress: 202/2367 symbols completed. AZZ took 19.6s. Estimated remaining: 4h 23m 34s. Rate: 8.2 symbols/min [2025-10-20T17:24:40.028979] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:24:40.176216] [INFO ] [MAIN ] BABA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:40.189965] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T17:24:40.261102] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-08. [2025-10-20T17:24:40.267018] [DEBUG] [LOCAL ] BABA: Loaded 54600 existing records from primary storage. [2025-10-20T17:24:40.267591] [INFO ] [MAIN ] BABA: Loaded 54600 existing records. [2025-10-20T17:24:40.474000] [WARN ] [MAIN ] BABA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:40.474365] [INFO ] [MAIN ] BABA: Wiping existing primary data before full refresh. [2025-10-20T17:24:40.474418] [INFO ] [LOCAL ] BABA: Starting deletion of primary data. [2025-10-20T17:24:40.474893] [DEBUG] [LOCAL ] BABA: Deleted primary data file. [2025-10-20T17:24:40.474952] [INFO ] [MAIN ] BABA: Starting full primary refresh. [2025-10-20T17:24:40.898237] [INFO ] [REMOTE ] BABA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:41.051466] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-09. [2025-10-20T17:24:41.395509] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:24:41.851368] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-05. [2025-10-20T17:24:41.916999] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T17:24:42.651462] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-01. [2025-10-20T17:24:43.451449] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-09. [2025-10-20T17:24:43.989750] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-10-20T17:24:44.251427] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-10. [2025-10-20T17:24:44.526583] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T17:24:45.022076] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T17:24:45.024544] [INFO ] [REMOTE ] AZO: Full fetch complete (15936 records). [2025-10-20T17:24:45.024634] [DEBUG] [VALIDATOR ] AZO: Validating primary data. [2025-10-20T17:24:45.059259] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-06. [2025-10-20T17:24:45.466893] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-01 (2.82 seconds). [2025-10-20T17:24:45.513045] [DEBUG] [VALIDATOR ] AZO: All expected trading days are present. [2025-10-20T17:24:45.513127] [DEBUG] [VALIDATOR ] AZO: Primary data validation passed. [2025-10-20T17:24:45.513469] [INFO ] [MAIN ] AZO: Fetched primary data is valid. [2025-10-20T17:24:45.513510] [DEBUG] [LOCAL ] AZO: Saving primary data. [2025-10-20T17:24:45.514352] [DEBUG] [LOCAL ] AZO: Trimmed 15936 records to 10945 (keeping 200 days of data) [2025-10-20T17:24:45.592464] [DEBUG] [LOCAL ] AZO: Primary data saved (10945 records). [2025-10-20T17:24:45.592626] [INFO ] [MAIN ] AZO: Full history saved. [2025-10-20T17:24:45.592954] [INFO ] [MAIN ] Update Progress: 203/2367 symbols completed. AZO took 25.3s. Estimated remaining: 4h 23m 10s. Rate: 8.2 symbols/min [2025-10-20T17:24:45.796526] [INFO ] [MAIN ] BAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:45.849481] [DEBUG] [LOCAL ] BAC: Loaded 54588 existing records from primary storage. [2025-10-20T17:24:45.849933] [INFO ] [MAIN ] BAC: Loaded 54588 existing records. [2025-10-20T17:24:45.861634] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-10. [2025-10-20T17:24:46.066386] [WARN ] [MAIN ] BAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:46.066680] [INFO ] [MAIN ] BAC: Wiping existing primary data before full refresh. [2025-10-20T17:24:46.066929] [INFO ] [LOCAL ] BAC: Starting deletion of primary data. [2025-10-20T17:24:46.067363] [DEBUG] [LOCAL ] BAC: Deleted primary data file. [2025-10-20T17:24:46.067407] [INFO ] [MAIN ] BAC: Starting full primary refresh. [2025-10-20T17:24:46.456836] [INFO ] [REMOTE ] BAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:46.651479] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-01. [2025-10-20T17:24:47.113215] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T17:24:47.118920] [INFO ] [REMOTE ] AYI: Full fetch complete (36997 records). [2025-10-20T17:24:47.119147] [DEBUG] [VALIDATOR ] AYI: Validating primary data. [2025-10-20T17:24:47.456704] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-02. [2025-10-20T17:24:47.635928] [DEBUG] [VALIDATOR ] AYI: All expected trading days are present. [2025-10-20T17:24:47.639830] [DEBUG] [VALIDATOR ] AYI: Primary data validation passed. [2025-10-20T17:24:47.640741] [INFO ] [MAIN ] AYI: Fetched primary data is valid. [2025-10-20T17:24:47.641075] [DEBUG] [LOCAL ] AYI: Saving primary data. [2025-10-20T17:24:47.645170] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-06 (2.59 seconds). [2025-10-20T17:24:47.657782] [DEBUG] [LOCAL ] AYI: Trimmed 36997 records to 24589 (keeping 200 days of data) [2025-10-20T17:24:47.832719] [DEBUG] [LOCAL ] AYI: Primary data saved (24589 records). [2025-10-20T17:24:47.832902] [INFO ] [MAIN ] AYI: Full history saved. [2025-10-20T17:24:47.832987] [INFO ] [MAIN ] Update Progress: 204/2367 symbols completed. AYI took 27.5s. Estimated remaining: 4h 22m 9s. Rate: 8.3 symbols/min [2025-10-20T17:24:48.027788] [INFO ] [MAIN ] BAH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:48.096025] [DEBUG] [LOCAL ] BAH: Loaded 52467 existing records from primary storage. [2025-10-20T17:24:48.096399] [INFO ] [MAIN ] BAH: Loaded 52467 existing records. [2025-10-20T17:24:48.280735] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-07. [2025-10-20T17:24:48.287801] [WARN ] [MAIN ] BAH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:48.289695] [INFO ] [MAIN ] BAH: Wiping existing primary data before full refresh. [2025-10-20T17:24:48.289760] [INFO ] [LOCAL ] BAH: Starting deletion of primary data. [2025-10-20T17:24:48.290008] [DEBUG] [LOCAL ] BAH: Deleted primary data file. [2025-10-20T17:24:48.290065] [INFO ] [MAIN ] BAH: Starting full primary refresh. [2025-10-20T17:24:48.734159] [INFO ] [REMOTE ] BAH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:48.783709] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-01 (2.13 seconds). [2025-10-20T17:24:49.051425] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-02. [2025-10-20T17:24:49.851472] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-01. [2025-10-20T17:24:50.102124] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-02 (2.65 seconds). [2025-10-20T17:24:50.544431] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T17:24:50.651372] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-03. [2025-10-20T17:24:50.890486] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T17:24:51.120666] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:24:51.451427] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-03. [2025-10-20T17:24:52.251515] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-02. [2025-10-20T17:24:53.051368] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-08. [2025-10-20T17:24:53.465971] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-03 (2.81 seconds). [2025-10-20T17:24:53.856233] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-04. [2025-10-20T17:24:53.863475] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-03 (2.41 seconds). [2025-10-20T17:24:54.027088] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T17:24:54.651462] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-04. [2025-10-20T17:24:55.458014] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-03. [2025-10-20T17:24:55.484592] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-08 (2.43 seconds). [2025-10-20T17:24:56.251425] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-09. [2025-10-20T17:24:56.752587] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-04 (2.90 seconds). [2025-10-20T17:24:57.051385] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-05. [2025-10-20T17:24:57.238816] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T17:24:57.290470] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-04 (2.64 seconds). [2025-10-20T17:24:57.851404] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-04. [2025-10-20T17:24:58.467184] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T17:24:58.651406] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-10. [2025-10-20T17:24:59.190509] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T17:24:59.199792] [INFO ] [REMOTE ] BA: Full fetch complete (77969 records). [2025-10-20T17:24:59.200016] [DEBUG] [VALIDATOR ] BA: Validating primary data. [2025-10-20T17:24:59.466405] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-05. [2025-10-20T17:24:59.679418] [DEBUG] [VALIDATOR ] BA: All expected trading days are present. [2025-10-20T17:24:59.679764] [DEBUG] [VALIDATOR ] BA: Primary data validation passed. [2025-10-20T17:24:59.680960] [INFO ] [MAIN ] BA: Fetched primary data is valid. [2025-10-20T17:24:59.681268] [DEBUG] [LOCAL ] BA: Saving primary data. [2025-10-20T17:24:59.683888] [DEBUG] [LOCAL ] BA: Trimmed 77969 records to 53790 (keeping 200 days of data) [2025-10-20T17:24:59.820087] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-05 (2.77 seconds). [2025-10-20T17:24:59.997191] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T17:25:00.149998] [DEBUG] [LOCAL ] BA: Primary data saved (53790 records). [2025-10-20T17:25:00.150414] [INFO ] [MAIN ] BA: Full history saved. [2025-10-20T17:25:00.150512] [INFO ] [MAIN ] Update Progress: 205/2367 symbols completed. BA took 36.2s. Estimated remaining: 4h 22m 55s. Rate: 8.2 symbols/min [2025-10-20T17:25:00.256773] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-06. [2025-10-20T17:25:00.345408] [INFO ] [MAIN ] BAK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:00.393276] [DEBUG] [LOCAL ] BAK: Loaded 39871 existing records from primary storage. [2025-10-20T17:25:00.393706] [INFO ] [MAIN ] BAK: Loaded 39871 existing records. [2025-10-20T17:25:00.599772] [WARN ] [MAIN ] BAK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:00.600107] [INFO ] [MAIN ] BAK: Wiping existing primary data before full refresh. [2025-10-20T17:25:00.600158] [INFO ] [LOCAL ] BAK: Starting deletion of primary data. [2025-10-20T17:25:00.600497] [DEBUG] [LOCAL ] BAK: Deleted primary data file. [2025-10-20T17:25:00.600561] [INFO ] [MAIN ] BAK: Starting full primary refresh. [2025-10-20T17:25:01.052958] [INFO ] [REMOTE ] BAK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:01.053041] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-05. [2025-10-20T17:25:01.851492] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-01. [2025-10-20T17:25:02.167718] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-05 (2.70 seconds). [2025-10-20T17:25:02.651488] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-06. [2025-10-20T17:25:02.992065] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T17:25:03.169089] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-06 (2.91 seconds). [2025-10-20T17:25:03.447079] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:25:03.456767] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-07. [2025-10-20T17:25:04.251425] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-02. [2025-10-20T17:25:05.051407] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-06. [2025-10-20T17:25:05.145958] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-06 (2.49 seconds). [2025-10-20T17:25:05.774552] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:25:05.851394] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-07. [2025-10-20T17:25:05.965367] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-07 (2.51 seconds). [2025-10-20T17:25:06.507166] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T17:25:06.651408] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-03. [2025-10-20T17:25:07.451397] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-08. [2025-10-20T17:25:08.070779] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:25:08.251387] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-04. [2025-10-20T17:25:08.476135] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-07 (2.62 seconds). [2025-10-20T17:25:09.051388] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-08. [2025-10-20T17:25:09.369071] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T17:25:09.851584] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-05. [2025-10-20T17:25:10.248588] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-08 (2.80 seconds). [2025-10-20T17:25:10.651463] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-07. [2025-10-20T17:25:10.980820] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:25:11.451380] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-09. [2025-10-20T17:25:11.499903] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-10-20T17:25:12.251403] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-09. [2025-10-20T17:25:12.617133] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T17:25:13.051395] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-08. [2025-10-20T17:25:13.851474] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-06. [2025-10-20T17:25:13.957243] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-09 (2.51 seconds). [2025-10-20T17:25:14.589334] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-09 (2.34 seconds). [2025-10-20T17:25:14.651461] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-10. [2025-10-20T17:25:14.710319] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T17:25:15.207409] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T17:25:15.451363] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-07. [2025-10-20T17:25:16.251410] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-09. [2025-10-20T17:25:16.947004] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:25:17.051412] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-10. [2025-10-20T17:25:17.455494] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-10 (2.80 seconds). [2025-10-20T17:25:17.464417] [INFO ] [REMOTE ] BABA: Full fetch complete (78000 records). [2025-10-20T17:25:17.464668] [DEBUG] [VALIDATOR ] BABA: Validating primary data. [2025-10-20T17:25:17.852759] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-08. [2025-10-20T17:25:17.931342] [DEBUG] [VALIDATOR ] BABA: All expected trading days are present. [2025-10-20T17:25:17.931518] [DEBUG] [VALIDATOR ] BABA: Primary data validation passed. [2025-10-20T17:25:17.932621] [INFO ] [MAIN ] BABA: Fetched primary data is valid. [2025-10-20T17:25:17.932731] [DEBUG] [LOCAL ] BABA: Saving primary data. [2025-10-20T17:25:17.965387] [DEBUG] [LOCAL ] BABA: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-10-20T17:25:17.968006] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T17:25:18.325924] [DEBUG] [LOCAL ] BABA: Primary data saved (53820 records). [2025-10-20T17:25:18.326083] [INFO ] [MAIN ] BABA: Full history saved. [2025-10-20T17:25:18.326212] [INFO ] [MAIN ] Update Progress: 206/2367 symbols completed. BABA took 38.4s. Estimated remaining: 4h 24m 41s. Rate: 8.2 symbols/min [2025-10-20T17:25:18.496988] [INFO ] [MAIN ] BALL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:18.549357] [DEBUG] [LOCAL ] BALL: Loaded 52656 existing records from primary storage. [2025-10-20T17:25:18.549729] [INFO ] [MAIN ] BALL: Loaded 52656 existing records. [2025-10-20T17:25:18.652106] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-10. [2025-10-20T17:25:18.732605] [WARN ] [MAIN ] BALL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:18.732890] [INFO ] [MAIN ] BALL: Wiping existing primary data before full refresh. [2025-10-20T17:25:18.732939] [INFO ] [LOCAL ] BALL: Starting deletion of primary data. [2025-10-20T17:25:18.733206] [DEBUG] [LOCAL ] BALL: Deleted primary data file. [2025-10-20T17:25:18.733264] [INFO ] [MAIN ] BALL: Starting full primary refresh. [2025-10-20T17:25:19.081634] [INFO ] [REMOTE ] BALL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:19.371241] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:25:19.451529] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-09. [2025-10-20T17:25:19.706482] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-10 (2.66 seconds). [2025-10-20T17:25:19.714779] [INFO ] [REMOTE ] BAC: Full fetch complete (77598 records). [2025-10-20T17:25:19.714967] [DEBUG] [VALIDATOR ] BAC: Validating primary data. [2025-10-20T17:25:19.716028] [FATAL] [VALIDATOR ] BAC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:25:19.716126] [FATAL] [MAIN ] BAC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:25:19.716172] [WARN ] [MAIN ] BAC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:25:19.716219] [INFO ] [LOCAL ] BAC: Starting deletion of primary data. [2025-10-20T17:25:19.716287] [DEBUG] [LOCAL ] BAC: No primary data file found to delete. [2025-10-20T17:25:19.716326] [INFO ] [LOCAL ] BAC: Starting deletion of derivative data. [2025-10-20T17:25:19.716514] [DEBUG] [LOCAL ] BAC: Deleted derivative data file. [2025-10-20T17:25:19.716907] [INFO ] [MAIN ] Update Progress: 207/2367 symbols completed. BAC took 34.1s. Estimated remaining: 4h 23m 32s. Rate: 8.2 symbols/min [2025-10-20T17:25:19.928002] [INFO ] [MAIN ] BALY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:19.928231] [DEBUG] [LOCAL ] BALY: No existing data file found in primary storage. [2025-10-20T17:25:19.928274] [WARN ] [MAIN ] BALY: No existing data found, performing full refresh instead. [2025-10-20T17:25:19.928477] [INFO ] [MAIN ] BALY: Wiping existing primary data before full refresh. [2025-10-20T17:25:19.928527] [INFO ] [LOCAL ] BALY: Starting deletion of primary data. [2025-10-20T17:25:19.928572] [DEBUG] [LOCAL ] BALY: No primary data file found to delete. [2025-10-20T17:25:19.928606] [INFO ] [MAIN ] BALY: Starting full primary refresh. [2025-10-20T17:25:20.252559] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-01. [2025-10-20T17:25:20.336459] [INFO ] [REMOTE ] BALY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:20.654252] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T17:25:20.663500] [INFO ] [REMOTE ] BAH: Full fetch complete (75242 records). [2025-10-20T17:25:20.663690] [DEBUG] [VALIDATOR ] BAH: Validating primary data. [2025-10-20T17:25:21.057459] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-01. [2025-10-20T17:25:21.099431] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:25:21.127909] [DEBUG] [VALIDATOR ] BAH: All expected trading days are present. [2025-10-20T17:25:21.128035] [DEBUG] [VALIDATOR ] BAH: Primary data validation passed. [2025-10-20T17:25:21.129235] [INFO ] [MAIN ] BAH: Fetched primary data is valid. [2025-10-20T17:25:21.129975] [DEBUG] [LOCAL ] BAH: Saving primary data. [2025-10-20T17:25:21.131747] [DEBUG] [LOCAL ] BAH: Trimmed 75242 records to 51683 (keeping 200 days of data) [2025-10-20T17:25:21.459307] [DEBUG] [LOCAL ] BAH: Primary data saved (51683 records). [2025-10-20T17:25:21.459454] [INFO ] [MAIN ] BAH: Full history saved. [2025-10-20T17:25:21.459541] [INFO ] [MAIN ] Update Progress: 208/2367 symbols completed. BAH took 33.6s. Estimated remaining: 4h 22m 27s. Rate: 8.2 symbols/min [2025-10-20T17:25:21.630530] [INFO ] [MAIN ] BAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:21.679163] [DEBUG] [LOCAL ] BAM: Loaded 52877 existing records from primary storage. [2025-10-20T17:25:21.679489] [INFO ] [MAIN ] BAM: Loaded 52877 existing records. [2025-10-20T17:25:21.859821] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-10. [2025-10-20T17:25:21.865717] [WARN ] [MAIN ] BAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:21.867174] [INFO ] [MAIN ] BAM: Wiping existing primary data before full refresh. [2025-10-20T17:25:21.867369] [INFO ] [LOCAL ] BAM: Starting deletion of primary data. [2025-10-20T17:25:21.868063] [DEBUG] [LOCAL ] BAM: Deleted primary data file. [2025-10-20T17:25:21.868198] [INFO ] [MAIN ] BAM: Starting full primary refresh. [2025-10-20T17:25:22.059925] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:25:22.284500] [INFO ] [REMOTE ] BAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:22.558038] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:25:22.651370] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-02. [2025-10-20T17:25:23.454996] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-01. [2025-10-20T17:25:23.459070] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T17:25:23.467155] [INFO ] [REMOTE ] BAK: Full fetch complete (56869 records). [2025-10-20T17:25:23.467305] [DEBUG] [VALIDATOR ] BAK: Validating primary data. [2025-10-20T17:25:23.928768] [DEBUG] [VALIDATOR ] BAK: All expected trading days are present. [2025-10-20T17:25:23.928894] [DEBUG] [VALIDATOR ] BAK: Primary data validation passed. [2025-10-20T17:25:23.929552] [INFO ] [MAIN ] BAK: Fetched primary data is valid. [2025-10-20T17:25:23.929623] [DEBUG] [LOCAL ] BAK: Saving primary data. [2025-10-20T17:25:23.931008] [DEBUG] [LOCAL ] BAK: Trimmed 56869 records to 39438 (keeping 200 days of data) [2025-10-20T17:25:24.163130] [DEBUG] [LOCAL ] BAK: Primary data saved (39438 records). [2025-10-20T17:25:24.163399] [INFO ] [MAIN ] BAK: Full history saved. [2025-10-20T17:25:24.163571] [INFO ] [MAIN ] Update Progress: 209/2367 symbols completed. BAK took 24.0s. Estimated remaining: 4h 21m 32s. Rate: 8.3 symbols/min [2025-10-20T17:25:24.262122] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-02. [2025-10-20T17:25:24.305887] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:25:24.391306] [INFO ] [MAIN ] BANC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:24.391507] [DEBUG] [LOCAL ] BANC: No existing data file found in primary storage. [2025-10-20T17:25:24.391545] [WARN ] [MAIN ] BANC: No existing data found, performing full refresh instead. [2025-10-20T17:25:24.391682] [INFO ] [MAIN ] BANC: Wiping existing primary data before full refresh. [2025-10-20T17:25:24.391735] [INFO ] [LOCAL ] BANC: Starting deletion of primary data. [2025-10-20T17:25:24.391777] [DEBUG] [LOCAL ] BANC: No primary data file found to delete. [2025-10-20T17:25:24.391809] [INFO ] [MAIN ] BANC: Starting full primary refresh. [2025-10-20T17:25:24.760103] [INFO ] [REMOTE ] BANC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:25.051356] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-03. [2025-10-20T17:25:25.109951] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T17:25:25.142263] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:25:25.851382] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-02. [2025-10-20T17:25:26.651362] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-03. [2025-10-20T17:25:26.803970] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:25:27.386303] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T17:25:27.455419] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-01. [2025-10-20T17:25:27.461552] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T17:25:28.251371] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-04. [2025-10-20T17:25:28.861315] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T17:25:29.051396] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-02. [2025-10-20T17:25:29.851360] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-03. [2025-10-20T17:25:30.014944] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T17:25:30.605106] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:25:30.651380] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-03. [2025-10-20T17:25:31.115709] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:25:31.451423] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-04. [2025-10-20T17:25:32.251399] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-04. [2025-10-20T17:25:32.454935] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T17:25:32.851280] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:25:32.988219] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T17:25:33.051367] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-04. [2025-10-20T17:25:33.851390] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-05. [2025-10-20T17:25:34.656485] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-05. [2025-10-20T17:25:34.665878] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:25:35.346399] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T17:25:35.451460] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-06. [2025-10-20T17:25:35.633881] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T17:25:36.251468] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-06. [2025-10-20T17:25:37.051438] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-05. [2025-10-20T17:25:37.096682] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:25:37.368114] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:25:37.851526] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-05. [2025-10-20T17:25:38.651468] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-07. [2025-10-20T17:25:38.817856] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:25:39.431837] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T17:25:39.451340] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-08. [2025-10-20T17:25:39.646147] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T17:25:40.106990] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T17:25:40.251400] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-06. [2025-10-20T17:25:41.051449] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-06. [2025-10-20T17:25:41.829855] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T17:25:41.851393] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-07. [2025-10-20T17:25:42.651411] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-09. [2025-10-20T17:25:42.739635] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:25:43.341137] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T17:25:43.451370] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-10. [2025-10-20T17:25:43.622351] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T17:25:44.083287] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T17:25:44.085768] [INFO ] [REMOTE ] BALY: Full fetch complete (16198 records). [2025-10-20T17:25:44.085843] [DEBUG] [VALIDATOR ] BALY: Validating primary data. [2025-10-20T17:25:44.258915] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-08. [2025-10-20T17:25:44.302553] [WARN ] [VALIDATOR ] BALY: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T17:25:44.302975] [FATAL] [MAIN ] BALY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:25:44.304003] [WARN ] [MAIN ] BALY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:25:44.304083] [INFO ] [LOCAL ] BALY: Starting deletion of primary data. [2025-10-20T17:25:44.304147] [DEBUG] [LOCAL ] BALY: No primary data file found to delete. [2025-10-20T17:25:44.304185] [INFO ] [LOCAL ] BALY: Starting deletion of derivative data. [2025-10-20T17:25:44.304226] [DEBUG] [LOCAL ] BALY: No derivative data file found to delete. [2025-10-20T17:25:44.304290] [INFO ] [MAIN ] Update Progress: 210/2367 symbols completed. BALY took 24.6s. Estimated remaining: 4h 23m 37s. Rate: 8.2 symbols/min [2025-10-20T17:25:44.515270] [INFO ] [MAIN ] BAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:44.545381] [DEBUG] [LOCAL ] BAP: Loaded 26192 existing records from primary storage. [2025-10-20T17:25:44.545756] [INFO ] [MAIN ] BAP: Loaded 26192 existing records. [2025-10-20T17:25:44.771901] [WARN ] [MAIN ] BAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:44.772172] [INFO ] [MAIN ] BAP: Wiping existing primary data before full refresh. [2025-10-20T17:25:44.772222] [INFO ] [LOCAL ] BAP: Starting deletion of primary data. [2025-10-20T17:25:44.772626] [DEBUG] [LOCAL ] BAP: Deleted primary data file. [2025-10-20T17:25:44.772726] [INFO ] [MAIN ] BAP: Starting full primary refresh. [2025-10-20T17:25:45.061373] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-07. [2025-10-20T17:25:45.220980] [INFO ] [REMOTE ] BAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:45.851417] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-07. [2025-10-20T17:25:45.970570] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:25:46.651444] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-01. [2025-10-20T17:25:47.121749] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T17:25:47.451392] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-08. [2025-10-20T17:25:47.539083] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T17:25:47.705629] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T17:25:48.251438] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-02. [2025-10-20T17:25:48.925928] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T17:25:49.051515] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-08. [2025-10-20T17:25:49.233195] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:25:49.851413] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-09. [2025-10-20T17:25:50.651389] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-03. [2025-10-20T17:25:50.787134] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:25:51.451430] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-09. [2025-10-20T17:25:51.588737] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:25:51.727966] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T17:25:52.251461] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-09. [2025-10-20T17:25:53.051354] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-10. [2025-10-20T17:25:53.151169] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:25:53.851385] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-04. [2025-10-20T17:25:53.940318] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T17:25:54.651630] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-10. [2025-10-20T17:25:54.774638] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T17:25:54.877671] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T17:25:54.885975] [INFO ] [REMOTE ] BANC: Full fetch complete (74580 records). [2025-10-20T17:25:54.886134] [DEBUG] [VALIDATOR ] BANC: Validating primary data. [2025-10-20T17:25:55.321690] [DEBUG] [VALIDATOR ] BANC: All expected trading days are present. [2025-10-20T17:25:55.321833] [DEBUG] [VALIDATOR ] BANC: Primary data validation passed. [2025-10-20T17:25:55.322637] [INFO ] [MAIN ] BANC: Fetched primary data is valid. [2025-10-20T17:25:55.322727] [DEBUG] [LOCAL ] BANC: Saving primary data. [2025-10-20T17:25:55.324306] [DEBUG] [LOCAL ] BANC: Trimmed 74580 records to 51387 (keeping 200 days of data) [2025-10-20T17:25:55.504926] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-10. [2025-10-20T17:25:55.683710] [DEBUG] [LOCAL ] BANC: Primary data saved (51387 records). [2025-10-20T17:25:55.683886] [INFO ] [MAIN ] BANC: Full history saved. [2025-10-20T17:25:55.683967] [INFO ] [MAIN ] Update Progress: 211/2367 symbols completed. BANC took 31.5s. Estimated remaining: 4h 24m 11s. Rate: 8.2 symbols/min [2025-10-20T17:25:55.913947] [INFO ] [MAIN ] BARK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:55.961525] [DEBUG] [LOCAL ] BARK: Loaded 38766 existing records from primary storage. [2025-10-20T17:25:55.961990] [INFO ] [MAIN ] BARK: Loaded 38766 existing records. [2025-10-20T17:25:56.234141] [WARN ] [MAIN ] BARK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:56.234551] [INFO ] [MAIN ] BARK: Wiping existing primary data before full refresh. [2025-10-20T17:25:56.234694] [INFO ] [LOCAL ] BARK: Starting deletion of primary data. [2025-10-20T17:25:56.235057] [DEBUG] [LOCAL ] BARK: Deleted primary data file. [2025-10-20T17:25:56.235125] [INFO ] [MAIN ] BARK: Starting full primary refresh. [2025-10-20T17:25:56.260353] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-05. [2025-10-20T17:25:56.777761] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-10 (2.13 seconds). [2025-10-20T17:25:56.788882] [INFO ] [REMOTE ] BARK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:56.797140] [INFO ] [REMOTE ] BAM: Full fetch complete (75138 records). [2025-10-20T17:25:56.797330] [DEBUG] [VALIDATOR ] BAM: Validating primary data. [2025-10-20T17:25:57.067021] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-01. [2025-10-20T17:25:57.234960] [DEBUG] [VALIDATOR ] BAM: All expected trading days are present. [2025-10-20T17:25:57.235082] [DEBUG] [VALIDATOR ] BAM: Primary data validation passed. [2025-10-20T17:25:57.235966] [INFO ] [MAIN ] BAM: Fetched primary data is valid. [2025-10-20T17:25:57.236172] [DEBUG] [LOCAL ] BAM: Saving primary data. [2025-10-20T17:25:57.237850] [DEBUG] [LOCAL ] BAM: Trimmed 75138 records to 52103 (keeping 200 days of data) [2025-10-20T17:25:57.443029] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T17:25:57.507032] [INFO ] [REMOTE ] BALL: Full fetch complete (75632 records). [2025-10-20T17:25:57.515256] [DEBUG] [VALIDATOR ] BALL: Validating primary data. [2025-10-20T17:25:57.630695] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:25:57.797511] [DEBUG] [LOCAL ] BAM: Primary data saved (52103 records). [2025-10-20T17:25:57.812408] [INFO ] [MAIN ] BAM: Full history saved. [2025-10-20T17:25:57.812862] [INFO ] [MAIN ] Update Progress: 212/2367 symbols completed. BAM took 36.4s. Estimated remaining: 4h 23m 11s. Rate: 8.2 symbols/min [2025-10-20T17:25:57.863823] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-06. [2025-10-20T17:25:58.152745] [INFO ] [MAIN ] BAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:58.237810] [DEBUG] [VALIDATOR ] BALL: All expected trading days are present. [2025-10-20T17:25:58.237969] [DEBUG] [VALIDATOR ] BALL: Primary data validation passed. [2025-10-20T17:25:58.238999] [INFO ] [MAIN ] BALL: Fetched primary data is valid. [2025-10-20T17:25:58.239358] [DEBUG] [LOCAL ] BALL: Saving primary data. [2025-10-20T17:25:58.241720] [DEBUG] [LOCAL ] BALL: Trimmed 75632 records to 51927 (keeping 200 days of data) [2025-10-20T17:25:58.388980] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:25:58.591635] [DEBUG] [LOCAL ] BAX: Loaded 54309 existing records from primary storage. [2025-10-20T17:25:58.628271] [INFO ] [MAIN ] BAX: Loaded 54309 existing records. [2025-10-20T17:25:58.665811] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-02. [2025-10-20T17:25:58.726339] [DEBUG] [LOCAL ] BALL: Primary data saved (51927 records). [2025-10-20T17:25:58.760235] [INFO ] [MAIN ] BALL: Full history saved. [2025-10-20T17:25:58.760375] [INFO ] [MAIN ] Update Progress: 213/2367 symbols completed. BALL took 40.4s. Estimated remaining: 4h 21m 59s. Rate: 8.2 symbols/min [2025-10-20T17:25:59.115480] [INFO ] [MAIN ] BB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:59.094753] [WARN ] [MAIN ] BAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:59.116026] [INFO ] [MAIN ] BAX: Wiping existing primary data before full refresh. [2025-10-20T17:25:59.116236] [INFO ] [LOCAL ] BAX: Starting deletion of primary data. [2025-10-20T17:25:59.117340] [DEBUG] [LOCAL ] BAX: Deleted primary data file. [2025-10-20T17:25:59.117412] [INFO ] [MAIN ] BAX: Starting full primary refresh. [2025-10-20T17:25:59.400338] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T17:25:59.458257] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-07. [2025-10-20T17:25:59.484010] [DEBUG] [LOCAL ] BB: Loaded 53673 existing records from primary storage. [2025-10-20T17:25:59.486350] [INFO ] [MAIN ] BB: Loaded 53673 existing records. [2025-10-20T17:25:59.670877] [INFO ] [REMOTE ] BAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:59.764161] [WARN ] [MAIN ] BB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:59.764677] [INFO ] [MAIN ] BB: Wiping existing primary data before full refresh. [2025-10-20T17:25:59.764746] [INFO ] [LOCAL ] BB: Starting deletion of primary data. [2025-10-20T17:25:59.765020] [DEBUG] [LOCAL ] BB: Deleted primary data file. [2025-10-20T17:25:59.765078] [INFO ] [MAIN ] BB: Starting full primary refresh. [2025-10-20T17:26:00.248814] [INFO ] [REMOTE ] BB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:00.251166] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T17:26:00.258414] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-01. [2025-10-20T17:26:00.741431] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T17:26:01.051396] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-01. [2025-10-20T17:26:01.851397] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-08. [2025-10-20T17:26:01.998846] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T17:26:02.656509] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-03. [2025-10-20T17:26:02.675924] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T17:26:02.982696] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:26:03.451376] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-02. [2025-10-20T17:26:03.745503] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:26:04.251382] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-04. [2025-10-20T17:26:05.051481] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-02. [2025-10-20T17:26:05.580264] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-10-20T17:26:05.718324] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:26:05.851403] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-03. [2025-10-20T17:26:06.651356] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-09. [2025-10-20T17:26:06.754942] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T17:26:07.451365] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-05. [2025-10-20T17:26:07.861986] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T17:26:07.878831] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T17:26:08.251377] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-03. [2025-10-20T17:26:08.854477] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:26:09.051434] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-10. [2025-10-20T17:26:09.851363] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-04. [2025-10-20T17:26:10.127577] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T17:26:10.313027] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T17:26:10.317266] [INFO ] [REMOTE ] BAP: Full fetch complete (37382 records). [2025-10-20T17:26:10.317375] [DEBUG] [VALIDATOR ] BAP: Validating primary data. [2025-10-20T17:26:10.656487] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-06. [2025-10-20T17:26:10.733281] [DEBUG] [VALIDATOR ] BAP: All expected trading days are present. [2025-10-20T17:26:10.733362] [DEBUG] [VALIDATOR ] BAP: Primary data validation passed. [2025-10-20T17:26:10.733868] [INFO ] [MAIN ] BAP: Fetched primary data is valid. [2025-10-20T17:26:10.733929] [DEBUG] [LOCAL ] BAP: Saving primary data. [2025-10-20T17:26:10.734825] [DEBUG] [LOCAL ] BAP: Trimmed 37382 records to 25879 (keeping 200 days of data) [2025-10-20T17:26:10.904924] [DEBUG] [LOCAL ] BAP: Primary data saved (25879 records). [2025-10-20T17:26:10.905066] [INFO ] [MAIN ] BAP: Full history saved. [2025-10-20T17:26:10.905144] [INFO ] [MAIN ] Update Progress: 214/2367 symbols completed. BAP took 26.6s. Estimated remaining: 4h 22m 40s. Rate: 8.2 symbols/min [2025-10-20T17:26:11.081136] [INFO ] [MAIN ] BBAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:11.136323] [DEBUG] [LOCAL ] BBAI: Loaded 54600 existing records from primary storage. [2025-10-20T17:26:11.136895] [INFO ] [MAIN ] BBAI: Loaded 54600 existing records. [2025-10-20T17:26:11.325903] [WARN ] [MAIN ] BBAI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:11.326123] [INFO ] [MAIN ] BBAI: Wiping existing primary data before full refresh. [2025-10-20T17:26:11.326181] [INFO ] [LOCAL ] BBAI: Starting deletion of primary data. [2025-10-20T17:26:11.326424] [DEBUG] [LOCAL ] BBAI: Deleted primary data file. [2025-10-20T17:26:11.326480] [INFO ] [MAIN ] BBAI: Starting full primary refresh. [2025-10-20T17:26:11.466456] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-04. [2025-10-20T17:26:11.690065] [INFO ] [REMOTE ] BBAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:11.720037] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T17:26:12.026621] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:26:12.251404] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-01. [2025-10-20T17:26:13.051433] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-05. [2025-10-20T17:26:13.416489] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T17:26:13.851478] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-07. [2025-10-20T17:26:14.651439] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-05. [2025-10-20T17:26:14.892216] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-01 (2.64 seconds). [2025-10-20T17:26:14.949795] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T17:26:15.292918] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:26:15.451384] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-08. [2025-10-20T17:26:16.217425] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:26:16.251368] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-06. [2025-10-20T17:26:16.516971] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T17:26:17.051420] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-06. [2025-10-20T17:26:17.851398] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-09. [2025-10-20T17:26:18.280009] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T17:26:18.651350] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-07. [2025-10-20T17:26:18.719092] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:26:19.319807] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T17:26:19.451387] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-02. [2025-10-20T17:26:20.251386] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-07. [2025-10-20T17:26:20.506134] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T17:26:21.051366] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-08. [2025-10-20T17:26:21.851357] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-10. [2025-10-20T17:26:22.050850] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-02 (2.60 seconds). [2025-10-20T17:26:22.143497] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T17:26:22.629037] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:26:22.651350] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-09. [2025-10-20T17:26:22.752391] [DEBUG] [REMOTE ] BB: Error fetching primary data for 2025-09: 'API call returned an empty response.' [2025-10-20T17:26:22.752718] [WARN ] [REMOTE ] BB: No data returned for month 2025-09 [2025-10-20T17:26:22.752804] [FATAL] [REMOTE ] BB: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:26:22.752882] [FATAL] [MAIN ] BB: Failed to fetch primary data. [2025-10-20T17:26:22.752919] [WARN ] [MAIN ] BB: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:26:22.752959] [INFO ] [LOCAL ] BB: Starting deletion of primary data. [2025-10-20T17:26:22.753014] [DEBUG] [LOCAL ] BB: No primary data file found to delete. [2025-10-20T17:26:22.753048] [INFO ] [LOCAL ] BB: Starting deletion of derivative data. [2025-10-20T17:26:22.753176] [DEBUG] [LOCAL ] BB: Deleted derivative data file. [2025-10-20T17:26:22.753239] [INFO ] [MAIN ] Update Progress: 215/2367 symbols completed. BB took 24.0s. Estimated remaining: 4h 23m 18s. Rate: 8.2 symbols/min [2025-10-20T17:26:22.969419] [INFO ] [MAIN ] BBAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:23.013070] [DEBUG] [LOCAL ] BBAR: Loaded 42985 existing records from primary storage. [2025-10-20T17:26:23.013460] [INFO ] [MAIN ] BBAR: Loaded 42985 existing records. [2025-10-20T17:26:23.233140] [WARN ] [MAIN ] BBAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:23.233563] [INFO ] [MAIN ] BBAR: Wiping existing primary data before full refresh. [2025-10-20T17:26:23.233636] [INFO ] [LOCAL ] BBAR: Starting deletion of primary data. [2025-10-20T17:26:23.234001] [DEBUG] [LOCAL ] BBAR: Deleted primary data file. [2025-10-20T17:26:23.234065] [INFO ] [MAIN ] BBAR: Starting full primary refresh. [2025-10-20T17:26:23.466768] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-03. [2025-10-20T17:26:23.514948] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T17:26:23.589400] [INFO ] [REMOTE ] BARK: Full fetch complete (56339 records). [2025-10-20T17:26:23.603520] [DEBUG] [VALIDATOR ] BARK: Validating primary data. [2025-10-20T17:26:23.783893] [INFO ] [REMOTE ] BBAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:24.169203] [DEBUG] [VALIDATOR ] BARK: All expected trading days are present. [2025-10-20T17:26:24.169302] [DEBUG] [VALIDATOR ] BARK: Primary data validation passed. [2025-10-20T17:26:24.170030] [INFO ] [MAIN ] BARK: Fetched primary data is valid. [2025-10-20T17:26:24.170098] [DEBUG] [LOCAL ] BARK: Saving primary data. [2025-10-20T17:26:24.171405] [DEBUG] [LOCAL ] BARK: Trimmed 56339 records to 38200 (keeping 200 days of data) [2025-10-20T17:26:24.252728] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-08. [2025-10-20T17:26:24.395612] [DEBUG] [LOCAL ] BARK: Primary data saved (38200 records). [2025-10-20T17:26:24.395809] [INFO ] [MAIN ] BARK: Full history saved. [2025-10-20T17:26:24.395917] [INFO ] [MAIN ] Update Progress: 216/2367 symbols completed. BARK took 28.7s. Estimated remaining: 4h 22m 14s. Rate: 8.2 symbols/min [2025-10-20T17:26:24.574895] [INFO ] [MAIN ] BBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:24.632126] [DEBUG] [LOCAL ] BBD: Loaded 54039 existing records from primary storage. [2025-10-20T17:26:24.632484] [INFO ] [MAIN ] BBD: Loaded 54039 existing records. [2025-10-20T17:26:24.815901] [WARN ] [MAIN ] BBD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:24.816154] [INFO ] [MAIN ] BBD: Wiping existing primary data before full refresh. [2025-10-20T17:26:24.816203] [INFO ] [LOCAL ] BBD: Starting deletion of primary data. [2025-10-20T17:26:24.816400] [DEBUG] [LOCAL ] BBD: Deleted primary data file. [2025-10-20T17:26:24.816437] [INFO ] [MAIN ] BBD: Starting full primary refresh. [2025-10-20T17:26:25.074346] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-01. [2025-10-20T17:26:25.199934] [INFO ] [REMOTE ] BBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:25.851442] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-01. [2025-10-20T17:26:26.166179] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:26:26.394882] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-03 (2.93 seconds). [2025-10-20T17:26:26.651390] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-04. [2025-10-20T17:26:26.929776] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:26:27.451419] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-02. [2025-10-20T17:26:27.638739] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:26:28.251489] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-02. [2025-10-20T17:26:28.731961] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T17:26:29.051413] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-03. [2025-10-20T17:26:29.292955] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-04 (2.64 seconds). [2025-10-20T17:26:29.851473] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-05. [2025-10-20T17:26:29.930219] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T17:26:30.120067] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T17:26:30.651576] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-04. [2025-10-20T17:26:31.451435] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-09. [2025-10-20T17:26:32.114360] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:26:32.251470] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-05. [2025-10-20T17:26:32.492553] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-05 (2.64 seconds). [2025-10-20T17:26:32.920816] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T17:26:33.051597] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-06. [2025-10-20T17:26:33.729997] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T17:26:33.851469] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-06. [2025-10-20T17:26:34.651518] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-03. [2025-10-20T17:26:35.418805] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T17:26:35.451338] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-07. [2025-10-20T17:26:35.632674] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-06 (2.58 seconds). [2025-10-20T17:26:35.977875] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T17:26:36.251398] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-10. [2025-10-20T17:26:36.785251] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T17:26:37.051386] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-04. [2025-10-20T17:26:37.851387] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-08. [2025-10-20T17:26:37.985573] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:26:37.994215] [INFO ] [REMOTE ] BAX: Full fetch complete (77616 records). [2025-10-20T17:26:37.994579] [DEBUG] [VALIDATOR ] BAX: Validating primary data. [2025-10-20T17:26:38.416365] [DEBUG] [VALIDATOR ] BAX: All expected trading days are present. [2025-10-20T17:26:38.416448] [DEBUG] [VALIDATOR ] BAX: Primary data validation passed. [2025-10-20T17:26:38.417236] [INFO ] [MAIN ] BAX: Fetched primary data is valid. [2025-10-20T17:26:38.417297] [DEBUG] [LOCAL ] BAX: Saving primary data. [2025-10-20T17:26:38.418832] [DEBUG] [LOCAL ] BAX: Trimmed 77616 records to 53537 (keeping 200 days of data) [2025-10-20T17:26:38.651451] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-07. [2025-10-20T17:26:38.779380] [DEBUG] [LOCAL ] BAX: Primary data saved (53537 records). [2025-10-20T17:26:38.779703] [INFO ] [MAIN ] BAX: Full history saved. [2025-10-20T17:26:38.779858] [INFO ] [MAIN ] Update Progress: 217/2367 symbols completed. BAX took 41.0s. Estimated remaining: 4h 23m 17s. Rate: 8.2 symbols/min [2025-10-20T17:26:38.990248] [INFO ] [MAIN ] BBDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:39.023915] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T17:26:39.060183] [DEBUG] [LOCAL ] BBDC: Loaded 33616 existing records from primary storage. [2025-10-20T17:26:39.060453] [INFO ] [MAIN ] BBDC: Loaded 33616 existing records. [2025-10-20T17:26:39.248604] [WARN ] [MAIN ] BBDC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:39.248887] [INFO ] [MAIN ] BBDC: Wiping existing primary data before full refresh. [2025-10-20T17:26:39.249110] [INFO ] [LOCAL ] BBDC: Starting deletion of primary data. [2025-10-20T17:26:39.249356] [DEBUG] [LOCAL ] BBDC: Deleted primary data file. [2025-10-20T17:26:39.249423] [INFO ] [MAIN ] BBDC: Starting full primary refresh. [2025-10-20T17:26:39.463959] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-05. [2025-10-20T17:26:39.527846] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:26:39.691004] [INFO ] [REMOTE ] BBDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:40.251634] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-01. [2025-10-20T17:26:41.051407] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-09. [2025-10-20T17:26:41.267401] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T17:26:41.312468] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T17:26:41.773351] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-07 (3.12 seconds). [2025-10-20T17:26:41.851382] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-08. [2025-10-20T17:26:42.651377] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-02. [2025-10-20T17:26:42.739550] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T17:26:43.451421] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-10. [2025-10-20T17:26:43.600133] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T17:26:44.251378] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-06. [2025-10-20T17:26:44.528515] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-08 (2.68 seconds). [2025-10-20T17:26:45.051365] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-09. [2025-10-20T17:26:45.367541] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:26:45.374369] [INFO ] [REMOTE ] BBAR: Full fetch complete (61404 records). [2025-10-20T17:26:45.374466] [DEBUG] [VALIDATOR ] BBAR: Validating primary data. [2025-10-20T17:26:45.762463] [DEBUG] [VALIDATOR ] BBAR: All expected trading days are present. [2025-10-20T17:26:45.762545] [DEBUG] [VALIDATOR ] BBAR: Primary data validation passed. [2025-10-20T17:26:45.763229] [INFO ] [MAIN ] BBAR: Fetched primary data is valid. [2025-10-20T17:26:45.763290] [DEBUG] [LOCAL ] BBAR: Saving primary data. [2025-10-20T17:26:45.764462] [DEBUG] [LOCAL ] BBAR: Trimmed 61404 records to 42660 (keeping 200 days of data) [2025-10-20T17:26:45.854353] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-03. [2025-10-20T17:26:46.033365] [DEBUG] [LOCAL ] BBAR: Primary data saved (42660 records). [2025-10-20T17:26:46.045124] [INFO ] [MAIN ] BBAR: Full history saved. [2025-10-20T17:26:46.045225] [INFO ] [MAIN ] Update Progress: 218/2367 symbols completed. BBAR took 23.3s. Estimated remaining: 4h 23m 9s. Rate: 8.2 symbols/min [2025-10-20T17:26:46.172442] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:26:46.255053] [INFO ] [MAIN ] BBDO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:46.264165] [DEBUG] [LOCAL ] BBDO: Loaded 5104 existing records from primary storage. [2025-10-20T17:26:46.264317] [INFO ] [MAIN ] BBDO: Loaded 5104 existing records. [2025-10-20T17:26:46.449490] [WARN ] [MAIN ] BBDO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:46.449738] [INFO ] [MAIN ] BBDO: Wiping existing primary data before full refresh. [2025-10-20T17:26:46.449796] [INFO ] [LOCAL ] BBDO: Starting deletion of primary data. [2025-10-20T17:26:46.449933] [DEBUG] [LOCAL ] BBDO: Deleted primary data file. [2025-10-20T17:26:46.449972] [INFO ] [MAIN ] BBDO: Starting full primary refresh. [2025-10-20T17:26:46.661719] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-07. [2025-10-20T17:26:46.817510] [INFO ] [REMOTE ] BBDO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:47.327821] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T17:26:47.451356] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-04. [2025-10-20T17:26:47.747255] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-09 (2.70 seconds). [2025-10-20T17:26:48.197689] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:26:48.251339] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-08. [2025-10-20T17:26:48.926824] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:26:49.051413] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-05. [2025-10-20T17:26:49.851446] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-01. [2025-10-20T17:26:50.053676] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T17:26:50.419927] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:26:50.586528] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T17:26:50.651435] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-06. [2025-10-20T17:26:51.451426] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-09. [2025-10-20T17:26:52.156353] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T17:26:52.251364] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-02. [2025-10-20T17:26:52.848195] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T17:26:53.051372] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-10. [2025-10-20T17:26:53.207091] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T17:26:53.851373] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-07. [2025-10-20T17:26:54.651346] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-03. [2025-10-20T17:26:55.118026] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T17:26:55.349087] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T17:26:55.451367] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-08. [2025-10-20T17:26:55.754090] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-10 (2.70 seconds). [2025-10-20T17:26:55.761886] [INFO ] [REMOTE ] BBAI: Full fetch complete (78000 records). [2025-10-20T17:26:55.762021] [DEBUG] [VALIDATOR ] BBAI: Validating primary data. [2025-10-20T17:26:56.155658] [DEBUG] [VALIDATOR ] BBAI: All expected trading days are present. [2025-10-20T17:26:56.155752] [DEBUG] [VALIDATOR ] BBAI: Primary data validation passed. [2025-10-20T17:26:56.156751] [INFO ] [MAIN ] BBAI: Fetched primary data is valid. [2025-10-20T17:26:56.156814] [DEBUG] [LOCAL ] BBAI: Saving primary data. [2025-10-20T17:26:56.158624] [DEBUG] [LOCAL ] BBAI: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-10-20T17:26:56.251824] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-04. [2025-10-20T17:26:56.520692] [DEBUG] [LOCAL ] BBAI: Primary data saved (53820 records). [2025-10-20T17:26:56.522307] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T17:26:56.522620] [INFO ] [MAIN ] BBAI: Full history saved. [2025-10-20T17:26:56.529461] [INFO ] [MAIN ] Update Progress: 219/2367 symbols completed. BBAI took 45.6s. Estimated remaining: 4h 23m 32s. Rate: 8.2 symbols/min [2025-10-20T17:26:56.701299] [INFO ] [MAIN ] BBN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:56.727074] [DEBUG] [LOCAL ] BBN: Loaded 18233 existing records from primary storage. [2025-10-20T17:26:56.727319] [INFO ] [MAIN ] BBN: Loaded 18233 existing records. [2025-10-20T17:26:56.850811] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T17:26:56.934576] [WARN ] [MAIN ] BBN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:56.934832] [INFO ] [MAIN ] BBN: Wiping existing primary data before full refresh. [2025-10-20T17:26:56.934879] [INFO ] [LOCAL ] BBN: Starting deletion of primary data. [2025-10-20T17:26:56.935044] [DEBUG] [LOCAL ] BBN: Deleted primary data file. [2025-10-20T17:26:56.935083] [INFO ] [MAIN ] BBN: Starting full primary refresh. [2025-10-20T17:26:57.056445] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-09. [2025-10-20T17:26:57.318744] [INFO ] [REMOTE ] BBN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:57.851367] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-05. [2025-10-20T17:26:58.292923] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T17:26:58.293624] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T17:26:58.651378] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-10. [2025-10-20T17:26:59.451435] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-01. [2025-10-20T17:27:00.251356] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-10. [2025-10-20T17:27:00.627452] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:27:00.750445] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:27:00.758819] [INFO ] [REMOTE ] BBD: Full fetch complete (77276 records). [2025-10-20T17:27:00.758953] [DEBUG] [VALIDATOR ] BBD: Validating primary data. [2025-10-20T17:27:01.057763] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-02. [2025-10-20T17:27:01.179904] [DEBUG] [VALIDATOR ] BBD: All expected trading days are present. [2025-10-20T17:27:01.180020] [DEBUG] [VALIDATOR ] BBD: Primary data validation passed. [2025-10-20T17:27:01.181398] [INFO ] [MAIN ] BBD: Fetched primary data is valid. [2025-10-20T17:27:01.181474] [DEBUG] [LOCAL ] BBD: Saving primary data. [2025-10-20T17:27:01.183697] [DEBUG] [LOCAL ] BBD: Trimmed 77276 records to 53276 (keeping 200 days of data) [2025-10-20T17:27:01.534750] [DEBUG] [LOCAL ] BBD: Primary data saved (53276 records). [2025-10-20T17:27:01.534901] [INFO ] [MAIN ] BBD: Full history saved. [2025-10-20T17:27:01.534985] [INFO ] [MAIN ] Update Progress: 220/2367 symbols completed. BBD took 37.1s. Estimated remaining: 4h 23m 2s. Rate: 8.2 symbols/min [2025-10-20T17:27:01.725092] [INFO ] [MAIN ] BBT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:01.725307] [DEBUG] [LOCAL ] BBT: No existing data file found in primary storage. [2025-10-20T17:27:01.725349] [WARN ] [MAIN ] BBT: No existing data found, performing full refresh instead. [2025-10-20T17:27:01.725482] [INFO ] [MAIN ] BBT: Wiping existing primary data before full refresh. [2025-10-20T17:27:01.725527] [INFO ] [LOCAL ] BBT: Starting deletion of primary data. [2025-10-20T17:27:01.725571] [DEBUG] [LOCAL ] BBT: No primary data file found to delete. [2025-10-20T17:27:01.725608] [INFO ] [MAIN ] BBT: Starting full primary refresh. [2025-10-20T17:27:01.853201] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-06. [2025-10-20T17:27:01.919123] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T17:27:02.005145] [INFO ] [REMOTE ] BBDC: Full fetch complete (48865 records). [2025-10-20T17:27:02.037835] [DEBUG] [VALIDATOR ] BBDC: Validating primary data. [2025-10-20T17:27:02.130443] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T17:27:02.263819] [INFO ] [REMOTE ] BBT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:02.567046] [DEBUG] [VALIDATOR ] BBDC: All expected trading days are present. [2025-10-20T17:27:02.567136] [DEBUG] [VALIDATOR ] BBDC: Primary data validation passed. [2025-10-20T17:27:02.567703] [INFO ] [MAIN ] BBDC: Fetched primary data is valid. [2025-10-20T17:27:02.567765] [DEBUG] [LOCAL ] BBDC: Saving primary data. [2025-10-20T17:27:02.568992] [DEBUG] [LOCAL ] BBDC: Trimmed 48865 records to 33048 (keeping 200 days of data) [2025-10-20T17:27:02.655132] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-01. [2025-10-20T17:27:02.667243] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T17:27:02.779528] [DEBUG] [LOCAL ] BBDC: Primary data saved (33048 records). [2025-10-20T17:27:02.779700] [INFO ] [MAIN ] BBDC: Full history saved. [2025-10-20T17:27:02.779803] [INFO ] [MAIN ] Update Progress: 221/2367 symbols completed. BBDC took 24.0s. Estimated remaining: 4h 21m 55s. Rate: 8.2 symbols/min [2025-10-20T17:27:02.879305] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:27:02.882385] [WARN ] [REMOTE ] BBT: No data returned for month 2025-01 [2025-10-20T17:27:02.887876] [FATAL] [REMOTE ] BBT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:27:02.898182] [FATAL] [MAIN ] BBT: Failed to fetch primary data. [2025-10-20T17:27:02.908506] [WARN ] [MAIN ] BBT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:27:02.934905] [INFO ] [LOCAL ] BBT: Starting deletion of primary data. [2025-10-20T17:27:02.952838] [INFO ] [MAIN ] BBU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:02.950163] [DEBUG] [LOCAL ] BBT: No primary data file found to delete. [2025-10-20T17:27:02.953018] [INFO ] [LOCAL ] BBT: Starting deletion of derivative data. [2025-10-20T17:27:02.953399] [DEBUG] [LOCAL ] BBT: No derivative data file found to delete. [2025-10-20T17:27:02.953497] [INFO ] [MAIN ] Update Progress: 222/2367 symbols completed. BBT took 1.4s. Estimated remaining: 4h 20m 39s. Rate: 8.2 symbols/min [2025-10-20T17:27:03.061834] [DEBUG] [LOCAL ] BBU: Loaded 5708 existing records from primary storage. [2025-10-20T17:27:03.065071] [INFO ] [MAIN ] BBU: Loaded 5708 existing records. [2025-10-20T17:27:03.265690] [INFO ] [MAIN ] BBUC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:03.360402] [WARN ] [MAIN ] BBU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:03.365973] [DEBUG] [LOCAL ] BBUC: Loaded 14077 existing records from primary storage. [2025-10-20T17:27:03.366252] [INFO ] [MAIN ] BBU: Wiping existing primary data before full refresh. [2025-10-20T17:27:03.366517] [INFO ] [MAIN ] BBUC: Loaded 14077 existing records. [2025-10-20T17:27:03.399399] [INFO ] [LOCAL ] BBU: Starting deletion of primary data. [2025-10-20T17:27:03.456498] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-03. [2025-10-20T17:27:03.444546] [DEBUG] [LOCAL ] BBU: Deleted primary data file. [2025-10-20T17:27:03.474210] [INFO ] [MAIN ] BBU: Starting full primary refresh. [2025-10-20T17:27:03.661849] [WARN ] [MAIN ] BBUC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:03.664396] [INFO ] [MAIN ] BBUC: Wiping existing primary data before full refresh. [2025-10-20T17:27:03.664472] [INFO ] [LOCAL ] BBUC: Starting deletion of primary data. [2025-10-20T17:27:03.680431] [DEBUG] [LOCAL ] BBUC: Deleted primary data file. [2025-10-20T17:27:03.690722] [INFO ] [MAIN ] BBUC: Starting full primary refresh. [2025-10-20T17:27:04.194032] [INFO ] [REMOTE ] BBU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:04.257118] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-01. [2025-10-20T17:27:04.300781] [INFO ] [REMOTE ] BBUC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:04.404230] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:27:04.993996] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T17:27:05.051389] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-02. [2025-10-20T17:27:05.622251] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T17:27:05.851388] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-04. [2025-10-20T17:27:06.651334] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-01. [2025-10-20T17:27:06.672048] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T17:27:07.451397] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-03. [2025-10-20T17:27:07.703937] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T17:27:08.003913] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T17:27:08.251402] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-04. [2025-10-20T17:27:08.659812] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T17:27:09.051365] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-05. [2025-10-20T17:27:09.529045] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T17:27:09.851416] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-06. [2025-10-20T17:27:10.323001] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T17:27:10.651374] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-05. [2025-10-20T17:27:11.451387] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-07. [2025-10-20T17:27:11.791237] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:27:11.888277] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-10-20T17:27:12.251385] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-08. [2025-10-20T17:27:12.767179] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T17:27:13.051408] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-09. [2025-10-20T17:27:13.798295] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T17:27:13.851405] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-02. [2025-10-20T17:27:14.398979] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T17:27:14.651398] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-03. [2025-10-20T17:27:15.451361] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-10. [2025-10-20T17:27:15.643183] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:27:16.251376] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-06. [2025-10-20T17:27:16.355339] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:27:16.357869] [INFO ] [REMOTE ] BBU: Full fetch complete (7467 records). [2025-10-20T17:27:16.357938] [DEBUG] [VALIDATOR ] BBU: Validating primary data. [2025-10-20T17:27:16.779364] [DEBUG] [VALIDATOR ] BBU: All expected trading days are present. [2025-10-20T17:27:16.779446] [DEBUG] [VALIDATOR ] BBU: Primary data validation passed. [2025-10-20T17:27:16.779697] [INFO ] [MAIN ] BBU: Fetched primary data is valid. [2025-10-20T17:27:16.779758] [DEBUG] [LOCAL ] BBU: Saving primary data. [2025-10-20T17:27:16.780275] [DEBUG] [LOCAL ] BBU: Trimmed 7467 records to 5727 (keeping 200 days of data) [2025-10-20T17:27:16.815879] [DEBUG] [LOCAL ] BBU: Primary data saved (5727 records). [2025-10-20T17:27:16.816032] [INFO ] [MAIN ] BBU: Full history saved. [2025-10-20T17:27:16.816126] [INFO ] [MAIN ] Update Progress: 223/2367 symbols completed. BBU took 14.0s. Estimated remaining: 4h 21m 35s. Rate: 8.2 symbols/min [2025-10-20T17:27:16.998387] [INFO ] [MAIN ] BBVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:17.054182] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-07. [2025-10-20T17:27:17.060608] [DEBUG] [LOCAL ] BBVA: Loaded 49862 existing records from primary storage. [2025-10-20T17:27:17.061336] [INFO ] [MAIN ] BBVA: Loaded 49862 existing records. [2025-10-20T17:27:17.303387] [WARN ] [MAIN ] BBVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:17.303808] [INFO ] [MAIN ] BBVA: Wiping existing primary data before full refresh. [2025-10-20T17:27:17.303941] [INFO ] [LOCAL ] BBVA: Starting deletion of primary data. [2025-10-20T17:27:17.315799] [DEBUG] [LOCAL ] BBVA: Deleted primary data file. [2025-10-20T17:27:17.315871] [INFO ] [MAIN ] BBVA: Starting full primary refresh. [2025-10-20T17:27:17.316610] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T17:27:17.737493] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T17:27:17.756783] [INFO ] [REMOTE ] BBVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:17.851414] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-01. [2025-10-20T17:27:18.651448] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-08. [2025-10-20T17:27:19.004766] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-10-20T17:27:19.451405] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-09. [2025-10-20T17:27:19.629487] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T17:27:20.121021] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T17:27:20.251420] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-04. [2025-10-20T17:27:21.051450] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-10. [2025-10-20T17:27:21.079549] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T17:27:21.494085] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T17:27:21.495549] [INFO ] [REMOTE ] BBDO: Full fetch complete (7511 records). [2025-10-20T17:27:21.495621] [DEBUG] [VALIDATOR ] BBDO: Validating primary data. [2025-10-20T17:27:21.861709] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-07. [2025-10-20T17:27:21.862742] [DEBUG] [VALIDATOR ] BBDO: All expected trading days are present. [2025-10-20T17:27:21.862793] [DEBUG] [VALIDATOR ] BBDO: Primary data validation passed. [2025-10-20T17:27:21.862987] [INFO ] [MAIN ] BBDO: Fetched primary data is valid. [2025-10-20T17:27:21.863025] [DEBUG] [LOCAL ] BBDO: Saving primary data. [2025-10-20T17:27:21.863446] [DEBUG] [LOCAL ] BBDO: Trimmed 7511 records to 5012 (keeping 200 days of data) [2025-10-20T17:27:21.893322] [DEBUG] [LOCAL ] BBDO: Primary data saved (5012 records). [2025-10-20T17:27:21.893442] [INFO ] [MAIN ] BBDO: Full history saved. [2025-10-20T17:27:21.893518] [INFO ] [MAIN ] Update Progress: 224/2367 symbols completed. BBDO took 35.8s. Estimated remaining: 4h 21m 6s. Rate: 8.2 symbols/min [2025-10-20T17:27:22.069774] [INFO ] [MAIN ] BBW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:22.098104] [DEBUG] [LOCAL ] BBW: Loaded 27830 existing records from primary storage. [2025-10-20T17:27:22.098412] [INFO ] [MAIN ] BBW: Loaded 27830 existing records. [2025-10-20T17:27:22.285447] [WARN ] [MAIN ] BBW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:22.285614] [INFO ] [MAIN ] BBW: Wiping existing primary data before full refresh. [2025-10-20T17:27:22.285690] [INFO ] [LOCAL ] BBW: Starting deletion of primary data. [2025-10-20T17:27:22.285879] [DEBUG] [LOCAL ] BBW: Deleted primary data file. [2025-10-20T17:27:22.285919] [INFO ] [MAIN ] BBW: Starting full primary refresh. [2025-10-20T17:27:22.638076] [INFO ] [REMOTE ] BBW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:22.651340] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-01. [2025-10-20T17:27:23.015541] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T17:27:23.451390] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-02. [2025-10-20T17:27:23.965938] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T17:27:24.251389] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-02. [2025-10-20T17:27:24.703170] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:27:25.051378] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-03. [2025-10-20T17:27:25.132542] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T17:27:25.851368] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-08. [2025-10-20T17:27:26.651364] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-03. [2025-10-20T17:27:26.795089] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:27:26.928191] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T17:27:27.451385] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-04. [2025-10-20T17:27:27.599730] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:27:28.251380] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-04. [2025-10-20T17:27:29.051364] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-05. [2025-10-20T17:27:29.257825] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T17:27:29.586145] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T17:27:29.722477] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:27:29.851357] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-06. [2025-10-20T17:27:30.641075] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:27:30.651324] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-07. [2025-10-20T17:27:31.451381] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-09. [2025-10-20T17:27:31.520517] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T17:27:32.172957] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T17:27:32.251396] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-08. [2025-10-20T17:27:33.051406] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-05. [2025-10-20T17:27:33.077676] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T17:27:33.851381] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-10. [2025-10-20T17:27:34.651400] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-09. [2025-10-20T17:27:34.823880] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:27:34.961639] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T17:27:34.965177] [INFO ] [REMOTE ] BBN: Full fetch complete (25135 records). [2025-10-20T17:27:34.965271] [DEBUG] [VALIDATOR ] BBN: Validating primary data. [2025-10-20T17:27:35.358813] [DEBUG] [VALIDATOR ] BBN: All expected trading days are present. [2025-10-20T17:27:35.367447] [DEBUG] [VALIDATOR ] BBN: Primary data validation passed. [2025-10-20T17:27:35.368305] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T17:27:35.368776] [INFO ] [MAIN ] BBN: Fetched primary data is valid. [2025-10-20T17:27:35.372420] [DEBUG] [LOCAL ] BBN: Saving primary data. [2025-10-20T17:27:35.373398] [DEBUG] [LOCAL ] BBN: Trimmed 25135 records to 17924 (keeping 200 days of data) [2025-10-20T17:27:35.451872] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-10. [2025-10-20T17:27:35.483829] [DEBUG] [LOCAL ] BBN: Primary data saved (17924 records). [2025-10-20T17:27:35.483969] [INFO ] [MAIN ] BBN: Full history saved. [2025-10-20T17:27:35.484047] [INFO ] [MAIN ] Update Progress: 225/2367 symbols completed. BBN took 39.0s. Estimated remaining: 4h 21m 58s. Rate: 8.2 symbols/min [2025-10-20T17:27:35.653730] [INFO ] [MAIN ] BBWI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:35.653896] [DEBUG] [LOCAL ] BBWI: No existing data file found in primary storage. [2025-10-20T17:27:35.653934] [WARN ] [MAIN ] BBWI: No existing data found, performing full refresh instead. [2025-10-20T17:27:35.654048] [INFO ] [MAIN ] BBWI: Wiping existing primary data before full refresh. [2025-10-20T17:27:35.654101] [INFO ] [LOCAL ] BBWI: Starting deletion of primary data. [2025-10-20T17:27:35.654145] [DEBUG] [LOCAL ] BBWI: No primary data file found to delete. [2025-10-20T17:27:35.654194] [INFO ] [MAIN ] BBWI: Starting full primary refresh. [2025-10-20T17:27:36.009804] [INFO ] [REMOTE ] BBWI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:36.251348] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-05. [2025-10-20T17:27:36.556474] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T17:27:36.560018] [INFO ] [REMOTE ] BBUC: Full fetch complete (19160 records). [2025-10-20T17:27:36.560129] [DEBUG] [VALIDATOR ] BBUC: Validating primary data. [2025-10-20T17:27:36.938756] [DEBUG] [VALIDATOR ] BBUC: All expected trading days are present. [2025-10-20T17:27:36.938834] [DEBUG] [VALIDATOR ] BBUC: Primary data validation passed. [2025-10-20T17:27:36.939148] [INFO ] [MAIN ] BBUC: Fetched primary data is valid. [2025-10-20T17:27:36.939185] [DEBUG] [LOCAL ] BBUC: Saving primary data. [2025-10-20T17:27:36.939728] [DEBUG] [LOCAL ] BBUC: Trimmed 19160 records to 13935 (keeping 200 days of data) [2025-10-20T17:27:37.020094] [DEBUG] [LOCAL ] BBUC: Primary data saved (13935 records). [2025-10-20T17:27:37.020222] [INFO ] [MAIN ] BBUC: Full history saved. [2025-10-20T17:27:37.020297] [INFO ] [MAIN ] Update Progress: 226/2367 symbols completed. BBUC took 34.1s. Estimated remaining: 4h 20m 56s. Rate: 8.2 symbols/min [2025-10-20T17:27:37.056967] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-06. [2025-10-20T17:27:37.198725] [INFO ] [MAIN ] BBY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:37.239443] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T17:27:37.262147] [DEBUG] [LOCAL ] BBY: Loaded 54218 existing records from primary storage. [2025-10-20T17:27:37.262451] [INFO ] [MAIN ] BBY: Loaded 54218 existing records. [2025-10-20T17:27:37.443815] [WARN ] [MAIN ] BBY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:37.444058] [INFO ] [MAIN ] BBY: Wiping existing primary data before full refresh. [2025-10-20T17:27:37.444124] [INFO ] [LOCAL ] BBY: Starting deletion of primary data. [2025-10-20T17:27:37.444745] [DEBUG] [LOCAL ] BBY: Deleted primary data file. [2025-10-20T17:27:37.444817] [INFO ] [MAIN ] BBY: Starting full primary refresh. [2025-10-20T17:27:37.824959] [INFO ] [REMOTE ] BBY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:37.851422] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-06. [2025-10-20T17:27:38.656494] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-01. [2025-10-20T17:27:38.670680] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T17:27:38.719453] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T17:27:39.451381] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-07. [2025-10-20T17:27:40.251415] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-07. [2025-10-20T17:27:40.470730] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:27:40.509965] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T17:27:41.051375] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-08. [2025-10-20T17:27:41.773757] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T17:27:41.851379] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-02. [2025-10-20T17:27:42.315407] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:27:42.651362] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-01. [2025-10-20T17:27:43.451388] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-09. [2025-10-20T17:27:43.589171] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T17:27:44.251413] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-03. [2025-10-20T17:27:44.571757] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T17:27:44.863811] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T17:27:45.051408] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-10. [2025-10-20T17:27:45.859607] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-08. [2025-10-20T17:27:45.862729] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T17:27:46.438369] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:27:46.443450] [INFO ] [REMOTE ] BBW: Full fetch complete (39732 records). [2025-10-20T17:27:46.443529] [DEBUG] [VALIDATOR ] BBW: Validating primary data. [2025-10-20T17:27:46.658545] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-04. [2025-10-20T17:27:46.827074] [DEBUG] [VALIDATOR ] BBW: All expected trading days are present. [2025-10-20T17:27:46.827156] [DEBUG] [VALIDATOR ] BBW: Primary data validation passed. [2025-10-20T17:27:46.827666] [INFO ] [MAIN ] BBW: Fetched primary data is valid. [2025-10-20T17:27:46.827730] [DEBUG] [LOCAL ] BBW: Saving primary data. [2025-10-20T17:27:46.828704] [DEBUG] [LOCAL ] BBW: Trimmed 39732 records to 27378 (keeping 200 days of data) [2025-10-20T17:27:46.992148] [DEBUG] [LOCAL ] BBW: Primary data saved (27378 records). [2025-10-20T17:27:46.992322] [INFO ] [MAIN ] BBW: Full history saved. [2025-10-20T17:27:46.992417] [INFO ] [MAIN ] Update Progress: 227/2367 symbols completed. BBW took 25.1s. Estimated remaining: 4h 21m 14s. Rate: 8.2 symbols/min [2025-10-20T17:27:47.187994] [INFO ] [MAIN ] BC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:47.230395] [DEBUG] [LOCAL ] BC: Loaded 44654 existing records from primary storage. [2025-10-20T17:27:47.230704] [INFO ] [MAIN ] BC: Loaded 44654 existing records. [2025-10-20T17:27:47.407202] [WARN ] [MAIN ] BC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:47.407472] [INFO ] [MAIN ] BC: Wiping existing primary data before full refresh. [2025-10-20T17:27:47.407708] [INFO ] [LOCAL ] BC: Starting deletion of primary data. [2025-10-20T17:27:47.407999] [DEBUG] [LOCAL ] BC: Deleted primary data file. [2025-10-20T17:27:47.408058] [INFO ] [MAIN ] BC: Starting full primary refresh. [2025-10-20T17:27:47.464270] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-02. [2025-10-20T17:27:47.733418] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T17:27:47.836866] [INFO ] [REMOTE ] BC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:48.251383] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-09. [2025-10-20T17:27:48.398326] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T17:27:49.051387] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-01. [2025-10-20T17:27:49.207992] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T17:27:49.851380] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-05. [2025-10-20T17:27:49.956181] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:27:50.651363] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-03. [2025-10-20T17:27:50.748691] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T17:27:51.451390] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-02. [2025-10-20T17:27:51.805867] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T17:27:52.251360] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-10. [2025-10-20T17:27:52.463341] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T17:27:52.573861] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:27:53.051388] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-06. [2025-10-20T17:27:53.851393] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-03. [2025-10-20T17:27:54.068319] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T17:27:54.076614] [INFO ] [REMOTE ] BBVA: Full fetch complete (71846 records). [2025-10-20T17:27:54.076868] [DEBUG] [VALIDATOR ] BBVA: Validating primary data. [2025-10-20T17:27:54.473557] [DEBUG] [VALIDATOR ] BBVA: All expected trading days are present. [2025-10-20T17:27:54.473686] [DEBUG] [VALIDATOR ] BBVA: Primary data validation passed. [2025-10-20T17:27:54.474582] [INFO ] [MAIN ] BBVA: Fetched primary data is valid. [2025-10-20T17:27:54.474631] [DEBUG] [LOCAL ] BBVA: Saving primary data. [2025-10-20T17:27:54.476215] [DEBUG] [LOCAL ] BBVA: Trimmed 71846 records to 49119 (keeping 200 days of data) [2025-10-20T17:27:54.652186] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-04. [2025-10-20T17:27:54.771717] [DEBUG] [LOCAL ] BBVA: Primary data saved (49119 records). [2025-10-20T17:27:54.771852] [INFO ] [MAIN ] BBVA: Full history saved. [2025-10-20T17:27:54.771946] [INFO ] [MAIN ] Update Progress: 228/2367 symbols completed. BBVA took 38.0s. Estimated remaining: 4h 21m 11s. Rate: 8.2 symbols/min [2025-10-20T17:27:54.945531] [INFO ] [MAIN ] BCAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:54.988064] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-10-20T17:27:55.024547] [DEBUG] [LOCAL ] BCAT: Loaded 34802 existing records from primary storage. [2025-10-20T17:27:55.024860] [INFO ] [MAIN ] BCAT: Loaded 34802 existing records. [2025-10-20T17:27:55.198175] [WARN ] [MAIN ] BCAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:55.198465] [INFO ] [MAIN ] BCAT: Wiping existing primary data before full refresh. [2025-10-20T17:27:55.198522] [INFO ] [LOCAL ] BCAT: Starting deletion of primary data. [2025-10-20T17:27:55.198881] [DEBUG] [LOCAL ] BCAT: Deleted primary data file. [2025-10-20T17:27:55.198940] [INFO ] [MAIN ] BCAT: Starting full primary refresh. [2025-10-20T17:27:55.456482] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-07. [2025-10-20T17:27:55.575456] [INFO ] [REMOTE ] BCAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:55.604430] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:27:56.251355] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-04. [2025-10-20T17:27:56.417514] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T17:27:57.051419] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-01. [2025-10-20T17:27:57.499761] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T17:27:57.851347] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-08. [2025-10-20T17:27:58.043572] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:27:58.098670] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T17:27:58.651456] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-02. [2025-10-20T17:27:59.253029] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:27:59.451465] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-05. [2025-10-20T17:27:59.988928] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T17:28:00.251382] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-05. [2025-10-20T17:28:00.983095] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T17:28:01.051387] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-06. [2025-10-20T17:28:01.851369] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-03. [2025-10-20T17:28:02.211197] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T17:28:02.651365] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-06. [2025-10-20T17:28:02.682414] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T17:28:02.952043] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T17:28:03.451413] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-09. [2025-10-20T17:28:04.251380] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-07. [2025-10-20T17:28:04.369741] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T17:28:05.051408] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-07. [2025-10-20T17:28:05.245233] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T17:28:05.672239] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:28:05.851377] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-08. [2025-10-20T17:28:06.651415] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-04. [2025-10-20T17:28:06.994209] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T17:28:07.301555] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T17:28:07.451450] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-10. [2025-10-20T17:28:08.112490] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:28:08.251377] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-05. [2025-10-20T17:28:09.011492] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T17:28:09.019872] [INFO ] [REMOTE ] BBY: Full fetch complete (77395 records). [2025-10-20T17:28:09.020048] [DEBUG] [VALIDATOR ] BBY: Validating primary data. [2025-10-20T17:28:09.056621] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-08. [2025-10-20T17:28:09.447468] [DEBUG] [VALIDATOR ] BBY: All expected trading days are present. [2025-10-20T17:28:09.447564] [DEBUG] [VALIDATOR ] BBY: Primary data validation passed. [2025-10-20T17:28:09.448372] [INFO ] [MAIN ] BBY: Fetched primary data is valid. [2025-10-20T17:28:09.448439] [DEBUG] [LOCAL ] BBY: Saving primary data. [2025-10-20T17:28:09.449923] [DEBUG] [LOCAL ] BBY: Trimmed 77395 records to 53436 (keeping 200 days of data) [2025-10-20T17:28:09.784466] [DEBUG] [LOCAL ] BBY: Primary data saved (53436 records). [2025-10-20T17:28:09.784783] [INFO ] [MAIN ] BBY: Full history saved. [2025-10-20T17:28:09.784897] [INFO ] [MAIN ] Update Progress: 229/2367 symbols completed. BBY took 32.8s. Estimated remaining: 4h 22m 15s. Rate: 8.2 symbols/min [2025-10-20T17:28:09.855721] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-09. [2025-10-20T17:28:09.965689] [INFO ] [MAIN ] BCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:09.984577] [DEBUG] [LOCAL ] BCC: No existing data file found in primary storage. [2025-10-20T17:28:09.985418] [WARN ] [MAIN ] BCC: No existing data found, performing full refresh instead. [2025-10-20T17:28:09.985622] [INFO ] [MAIN ] BCC: Wiping existing primary data before full refresh. [2025-10-20T17:28:09.986573] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:28:09.986694] [INFO ] [LOCAL ] BCC: Starting deletion of primary data. [2025-10-20T17:28:09.994792] [DEBUG] [LOCAL ] BCC: No primary data file found to delete. [2025-10-20T17:28:09.995596] [INFO ] [MAIN ] BCC: Starting full primary refresh. [2025-10-20T17:28:10.359750] [INFO ] [REMOTE ] BCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:10.651365] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-06. [2025-10-20T17:28:10.856831] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T17:28:11.451440] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-09. [2025-10-20T17:28:11.489420] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T17:28:12.051249] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T17:28:12.251391] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-07. [2025-10-20T17:28:12.977798] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:28:13.051370] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-10. [2025-10-20T17:28:13.353048] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T17:28:13.851441] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-10. [2025-10-20T17:28:14.659440] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-08. [2025-10-20T17:28:14.674566] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T17:28:14.681456] [INFO ] [REMOTE ] BC: Full fetch complete (63673 records). [2025-10-20T17:28:14.681607] [DEBUG] [VALIDATOR ] BC: Validating primary data. [2025-10-20T17:28:15.076151] [DEBUG] [VALIDATOR ] BC: All expected trading days are present. [2025-10-20T17:28:15.076260] [DEBUG] [VALIDATOR ] BC: Primary data validation passed. [2025-10-20T17:28:15.077019] [INFO ] [MAIN ] BC: Fetched primary data is valid. [2025-10-20T17:28:15.077083] [DEBUG] [LOCAL ] BC: Saving primary data. [2025-10-20T17:28:15.078396] [DEBUG] [LOCAL ] BC: Trimmed 63673 records to 43999 (keeping 200 days of data) [2025-10-20T17:28:15.344510] [DEBUG] [LOCAL ] BC: Primary data saved (43999 records). [2025-10-20T17:28:15.344669] [INFO ] [MAIN ] BC: Full history saved. [2025-10-20T17:28:15.344767] [INFO ] [MAIN ] Update Progress: 230/2367 symbols completed. BC took 28.4s. Estimated remaining: 4h 21m 51s. Rate: 8.2 symbols/min [2025-10-20T17:28:15.456498] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-01. [2025-10-20T17:28:15.522586] [INFO ] [MAIN ] BCE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:15.568877] [DEBUG] [LOCAL ] BCE: Loaded 53022 existing records from primary storage. [2025-10-20T17:28:15.569190] [INFO ] [MAIN ] BCE: Loaded 53022 existing records. [2025-10-20T17:28:15.756597] [WARN ] [MAIN ] BCE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:15.757052] [INFO ] [MAIN ] BCE: Wiping existing primary data before full refresh. [2025-10-20T17:28:15.757115] [INFO ] [LOCAL ] BCE: Starting deletion of primary data. [2025-10-20T17:28:15.775323] [DEBUG] [LOCAL ] BCE: Deleted primary data file. [2025-10-20T17:28:15.775478] [INFO ] [MAIN ] BCE: Starting full primary refresh. [2025-10-20T17:28:15.844891] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T17:28:15.892249] [INFO ] [REMOTE ] BBWI: Full fetch complete (77629 records). [2025-10-20T17:28:15.918754] [DEBUG] [VALIDATOR ] BBWI: Validating primary data. [2025-10-20T17:28:16.340279] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:28:16.382026] [INFO ] [REMOTE ] BCE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:16.402832] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-01. [2025-10-20T17:28:16.590062] [DEBUG] [VALIDATOR ] BBWI: All expected trading days are present. [2025-10-20T17:28:16.590157] [DEBUG] [VALIDATOR ] BBWI: Primary data validation passed. [2025-10-20T17:28:16.590933] [INFO ] [MAIN ] BBWI: Fetched primary data is valid. [2025-10-20T17:28:16.590994] [DEBUG] [LOCAL ] BBWI: Saving primary data. [2025-10-20T17:28:16.592382] [DEBUG] [LOCAL ] BBWI: Trimmed 77629 records to 53584 (keeping 200 days of data) [2025-10-20T17:28:16.911020] [DEBUG] [LOCAL ] BBWI: Primary data saved (53584 records). [2025-10-20T17:28:16.911180] [INFO ] [MAIN ] BBWI: Full history saved. [2025-10-20T17:28:16.911361] [INFO ] [MAIN ] Update Progress: 231/2367 symbols completed. BBWI took 41.4s. Estimated remaining: 4h 20m 50s. Rate: 8.2 symbols/min [2025-10-20T17:28:16.996524] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:28:17.056478] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-02. [2025-10-20T17:28:17.103495] [INFO ] [MAIN ] BCH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:17.135298] [DEBUG] [LOCAL ] BCH: Loaded 30818 existing records from primary storage. [2025-10-20T17:28:17.135627] [INFO ] [MAIN ] BCH: Loaded 30818 existing records. [2025-10-20T17:28:17.309122] [WARN ] [MAIN ] BCH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:17.309332] [INFO ] [MAIN ] BCH: Wiping existing primary data before full refresh. [2025-10-20T17:28:17.309387] [INFO ] [LOCAL ] BCH: Starting deletion of primary data. [2025-10-20T17:28:17.309605] [DEBUG] [LOCAL ] BCH: Deleted primary data file. [2025-10-20T17:28:17.309699] [INFO ] [MAIN ] BCH: Starting full primary refresh. [2025-10-20T17:28:17.672275] [INFO ] [REMOTE ] BCH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:17.851386] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-09. [2025-10-20T17:28:18.283826] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:28:18.454045] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T17:28:18.651434] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-01. [2025-10-20T17:28:19.235869] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:28:19.451397] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-03. [2025-10-20T17:28:19.572947] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:28:20.251454] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-02. [2025-10-20T17:28:20.556459] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T17:28:21.051403] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-04. [2025-10-20T17:28:21.851418] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-10. [2025-10-20T17:28:21.938887] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:28:22.042557] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T17:28:22.651369] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-05. [2025-10-20T17:28:23.183683] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:28:23.189165] [INFO ] [REMOTE ] BCAT: Full fetch complete (50025 records). [2025-10-20T17:28:23.189254] [DEBUG] [VALIDATOR ] BCAT: Validating primary data. [2025-10-20T17:28:23.456584] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-03. [2025-10-20T17:28:23.574812] [DEBUG] [VALIDATOR ] BCAT: All expected trading days are present. [2025-10-20T17:28:23.575004] [DEBUG] [VALIDATOR ] BCAT: Primary data validation passed. [2025-10-20T17:28:23.575739] [INFO ] [MAIN ] BCAT: Fetched primary data is valid. [2025-10-20T17:28:23.575819] [DEBUG] [LOCAL ] BCAT: Saving primary data. [2025-10-20T17:28:23.577093] [DEBUG] [LOCAL ] BCAT: Trimmed 50025 records to 34298 (keeping 200 days of data) [2025-10-20T17:28:23.802743] [DEBUG] [LOCAL ] BCAT: Primary data saved (34298 records). [2025-10-20T17:28:23.811281] [INFO ] [MAIN ] BCAT: Full history saved. [2025-10-20T17:28:23.811401] [INFO ] [MAIN ] Update Progress: 232/2367 symbols completed. BCAT took 29.0s. Estimated remaining: 4h 20m 39s. Rate: 8.2 symbols/min [2025-10-20T17:28:23.905819] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T17:28:24.033429] [INFO ] [MAIN ] BCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:24.033624] [DEBUG] [LOCAL ] BCO: No existing data file found in primary storage. [2025-10-20T17:28:24.033693] [WARN ] [MAIN ] BCO: No existing data found, performing full refresh instead. [2025-10-20T17:28:24.033811] [INFO ] [MAIN ] BCO: Wiping existing primary data before full refresh. [2025-10-20T17:28:24.033853] [INFO ] [LOCAL ] BCO: Starting deletion of primary data. [2025-10-20T17:28:24.033893] [DEBUG] [LOCAL ] BCO: No primary data file found to delete. [2025-10-20T17:28:24.033924] [INFO ] [MAIN ] BCO: Starting full primary refresh. [2025-10-20T17:28:24.260701] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-02. [2025-10-20T17:28:24.404240] [INFO ] [REMOTE ] BCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:25.058122] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-06. [2025-10-20T17:28:25.061277] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:28:25.532125] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T17:28:25.851366] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-01. [2025-10-20T17:28:26.187459] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T17:28:26.651443] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-04. [2025-10-20T17:28:27.041883] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:28:27.451470] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-02. [2025-10-20T17:28:28.251347] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-07. [2025-10-20T17:28:28.350666] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:28:28.417410] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T17:28:29.051375] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-03. [2025-10-20T17:28:29.324726] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T17:28:29.851344] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-08. [2025-10-20T17:28:30.188803] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T17:28:30.651372] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-04. [2025-10-20T17:28:31.386218] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:28:31.451372] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-09. [2025-10-20T17:28:32.251403] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-05. [2025-10-20T17:28:32.564571] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T17:28:33.051410] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-05. [2025-10-20T17:28:33.099431] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:28:33.851388] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-10. [2025-10-20T17:28:34.613778] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-05 (2.36 seconds). [2025-10-20T17:28:34.651336] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-06. [2025-10-20T17:28:34.725254] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T17:28:35.218736] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:28:35.224827] [INFO ] [REMOTE ] BCC: Full fetch complete (41522 records). [2025-10-20T17:28:35.224984] [DEBUG] [VALIDATOR ] BCC: Validating primary data. [2025-10-20T17:28:35.461765] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-06. [2025-10-20T17:28:35.603077] [DEBUG] [VALIDATOR ] BCC: All expected trading days are present. [2025-10-20T17:28:35.603167] [DEBUG] [VALIDATOR ] BCC: Primary data validation passed. [2025-10-20T17:28:35.603823] [INFO ] [MAIN ] BCC: Fetched primary data is valid. [2025-10-20T17:28:35.603886] [DEBUG] [LOCAL ] BCC: Saving primary data. [2025-10-20T17:28:35.604878] [DEBUG] [LOCAL ] BCC: Trimmed 41522 records to 28671 (keeping 200 days of data) [2025-10-20T17:28:35.796575] [DEBUG] [LOCAL ] BCC: Primary data saved (28671 records). [2025-10-20T17:28:35.796747] [INFO ] [MAIN ] BCC: Full history saved. [2025-10-20T17:28:35.796828] [INFO ] [MAIN ] Update Progress: 233/2367 symbols completed. BCC took 26.0s. Estimated remaining: 4h 21m 14s. Rate: 8.2 symbols/min [2025-10-20T17:28:35.986736] [INFO ] [MAIN ] BCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:36.039700] [DEBUG] [LOCAL ] BCS: Loaded 54222 existing records from primary storage. [2025-10-20T17:28:36.040055] [INFO ] [MAIN ] BCS: Loaded 54222 existing records. [2025-10-20T17:28:36.239370] [WARN ] [MAIN ] BCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:36.240273] [INFO ] [MAIN ] BCS: Wiping existing primary data before full refresh. [2025-10-20T17:28:36.240552] [INFO ] [LOCAL ] BCS: Starting deletion of primary data. [2025-10-20T17:28:36.240842] [DEBUG] [LOCAL ] BCS: Deleted primary data file. [2025-10-20T17:28:36.240892] [INFO ] [MAIN ] BCS: Starting full primary refresh. [2025-10-20T17:28:36.269864] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-03. [2025-10-20T17:28:36.592477] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T17:28:36.650291] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:28:36.725039] [INFO ] [REMOTE ] BCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:37.051357] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-07. [2025-10-20T17:28:37.781430] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T17:28:37.851415] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-04. [2025-10-20T17:28:38.207392] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:28:38.651466] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-07. [2025-10-20T17:28:39.238807] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:28:39.451434] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-08. [2025-10-20T17:28:40.251337] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-05. [2025-10-20T17:28:40.353124] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T17:28:40.718252] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T17:28:41.051414] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-08. [2025-10-20T17:28:41.113466] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T17:28:41.851379] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-06. [2025-10-20T17:28:42.414480] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T17:28:42.612021] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T17:28:42.651368] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-09. [2025-10-20T17:28:43.451378] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-07. [2025-10-20T17:28:44.251416] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-09. [2025-10-20T17:28:44.425638] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:28:44.763505] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T17:28:44.990539] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T17:28:45.051392] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-10. [2025-10-20T17:28:45.851380] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-10. [2025-10-20T17:28:46.651366] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-01. [2025-10-20T17:28:46.861060] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T17:28:46.868917] [INFO ] [REMOTE ] BCE: Full fetch complete (76119 records). [2025-10-20T17:28:46.869045] [DEBUG] [VALIDATOR ] BCE: Validating primary data. [2025-10-20T17:28:47.240511] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:28:47.282059] [INFO ] [REMOTE ] BCO: Full fetch complete (36223 records). [2025-10-20T17:28:47.293211] [DEBUG] [VALIDATOR ] BCO: Validating primary data. [2025-10-20T17:28:47.377230] [DEBUG] [VALIDATOR ] BCE: All expected trading days are present. [2025-10-20T17:28:47.383684] [DEBUG] [VALIDATOR ] BCE: Primary data validation passed. [2025-10-20T17:28:47.404339] [INFO ] [MAIN ] BCE: Fetched primary data is valid. [2025-10-20T17:28:47.426273] [DEBUG] [LOCAL ] BCE: Saving primary data. [2025-10-20T17:28:47.461138] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-08. [2025-10-20T17:28:47.542579] [DEBUG] [LOCAL ] BCE: Trimmed 76119 records to 52271 (keeping 200 days of data) [2025-10-20T17:28:47.861017] [DEBUG] [VALIDATOR ] BCO: All expected trading days are present. [2025-10-20T17:28:47.864681] [DEBUG] [VALIDATOR ] BCO: Primary data validation passed. [2025-10-20T17:28:47.867089] [INFO ] [MAIN ] BCO: Fetched primary data is valid. [2025-10-20T17:28:47.868893] [DEBUG] [LOCAL ] BCO: Saving primary data. [2025-10-20T17:28:47.934044] [DEBUG] [LOCAL ] BCO: Trimmed 36223 records to 25219 (keeping 200 days of data) [2025-10-20T17:28:48.228635] [DEBUG] [LOCAL ] BCE: Primary data saved (52271 records). [2025-10-20T17:28:48.231850] [INFO ] [MAIN ] BCE: Full history saved. [2025-10-20T17:28:48.231963] [INFO ] [MAIN ] Update Progress: 234/2367 symbols completed. BCE took 32.9s. Estimated remaining: 4h 21m 53s. Rate: 8.1 symbols/min [2025-10-20T17:28:48.340817] [DEBUG] [LOCAL ] BCO: Primary data saved (25219 records). [2025-10-20T17:28:48.340984] [INFO ] [MAIN ] BCO: Full history saved. [2025-10-20T17:28:48.346822] [INFO ] [MAIN ] Update Progress: 235/2367 symbols completed. BCO took 24.5s. Estimated remaining: 4h 20m 40s. Rate: 8.2 symbols/min [2025-10-20T17:28:48.517753] [INFO ] [MAIN ] BCSF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:48.590795] [INFO ] [MAIN ] BCX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:48.630557] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T17:28:48.645693] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-02. [2025-10-20T17:28:48.663216] [DEBUG] [LOCAL ] BCX: Loaded 22404 existing records from primary storage. [2025-10-20T17:28:48.662880] [DEBUG] [LOCAL ] BCSF: Loaded 28764 existing records from primary storage. [2025-10-20T17:28:48.663541] [INFO ] [MAIN ] BCX: Loaded 22404 existing records. [2025-10-20T17:28:48.684016] [INFO ] [MAIN ] BCSF: Loaded 28764 existing records. [2025-10-20T17:28:48.973731] [WARN ] [MAIN ] BCX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:49.005050] [INFO ] [MAIN ] BCX: Wiping existing primary data before full refresh. [2025-10-20T17:28:49.037303] [INFO ] [LOCAL ] BCX: Starting deletion of primary data. [2025-10-20T17:28:49.040177] [WARN ] [MAIN ] BCSF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:49.041475] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:28:49.041588] [DEBUG] [LOCAL ] BCX: Deleted primary data file. [2025-10-20T17:28:49.047412] [INFO ] [MAIN ] BCX: Starting full primary refresh. [2025-10-20T17:28:49.059821] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-09. [2025-10-20T17:28:49.047614] [INFO ] [MAIN ] BCSF: Wiping existing primary data before full refresh. [2025-10-20T17:28:49.150160] [INFO ] [LOCAL ] BCSF: Starting deletion of primary data. [2025-10-20T17:28:49.152935] [DEBUG] [LOCAL ] BCSF: Deleted primary data file. [2025-10-20T17:28:49.158317] [INFO ] [MAIN ] BCSF: Starting full primary refresh. [2025-10-20T17:28:49.718073] [INFO ] [REMOTE ] BCX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:49.800236] [INFO ] [REMOTE ] BCSF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:49.851331] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-01. [2025-10-20T17:28:50.114238] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T17:28:50.223589] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:28:50.651385] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-01. [2025-10-20T17:28:51.117782] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:28:51.451374] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-02. [2025-10-20T17:28:52.168086] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T17:28:52.251368] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-02. [2025-10-20T17:28:52.700851] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:28:53.051372] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-10. [2025-10-20T17:28:53.303183] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T17:28:53.851402] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-03. [2025-10-20T17:28:54.232721] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T17:28:54.238131] [INFO ] [REMOTE ] BCH: Full fetch complete (42007 records). [2025-10-20T17:28:54.238222] [DEBUG] [VALIDATOR ] BCH: Validating primary data. [2025-10-20T17:28:54.614457] [DEBUG] [VALIDATOR ] BCH: All expected trading days are present. [2025-10-20T17:28:54.614567] [DEBUG] [VALIDATOR ] BCH: Primary data validation passed. [2025-10-20T17:28:54.615093] [INFO ] [MAIN ] BCH: Fetched primary data is valid. [2025-10-20T17:28:54.615157] [DEBUG] [LOCAL ] BCH: Saving primary data. [2025-10-20T17:28:54.616106] [DEBUG] [LOCAL ] BCH: Trimmed 42007 records to 30410 (keeping 200 days of data) [2025-10-20T17:28:54.662593] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-03. [2025-10-20T17:28:54.791027] [DEBUG] [LOCAL ] BCH: Primary data saved (30410 records). [2025-10-20T17:28:54.791173] [INFO ] [MAIN ] BCH: Full history saved. [2025-10-20T17:28:54.791505] [INFO ] [MAIN ] Update Progress: 236/2367 symbols completed. BCH took 37.9s. Estimated remaining: 4h 20m 25s. Rate: 8.2 symbols/min [2025-10-20T17:28:54.959988] [INFO ] [MAIN ] BDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:54.975219] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T17:28:55.002667] [DEBUG] [LOCAL ] BDC: Loaded 24777 existing records from primary storage. [2025-10-20T17:28:55.002995] [INFO ] [MAIN ] BDC: Loaded 24777 existing records. [2025-10-20T17:28:55.176097] [WARN ] [MAIN ] BDC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:55.176313] [INFO ] [MAIN ] BDC: Wiping existing primary data before full refresh. [2025-10-20T17:28:55.176358] [INFO ] [LOCAL ] BDC: Starting deletion of primary data. [2025-10-20T17:28:55.176578] [DEBUG] [LOCAL ] BDC: Deleted primary data file. [2025-10-20T17:28:55.176634] [INFO ] [MAIN ] BDC: Starting full primary refresh. [2025-10-20T17:28:55.453420] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-03. [2025-10-20T17:28:55.532439] [INFO ] [REMOTE ] BDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:55.981372] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T17:28:56.251364] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-04. [2025-10-20T17:28:57.051452] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-04. [2025-10-20T17:28:57.112525] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T17:28:57.464703] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:28:57.851396] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-05. [2025-10-20T17:28:58.392260] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:28:58.651350] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-01. [2025-10-20T17:28:59.041682] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T17:28:59.451359] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-04. [2025-10-20T17:28:59.944226] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T17:29:00.251425] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-06. [2025-10-20T17:29:01.051440] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-05. [2025-10-20T17:29:01.681753] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T17:29:01.732928] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T17:29:01.851362] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-02. [2025-10-20T17:29:02.148907] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T17:29:02.651453] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-06. [2025-10-20T17:29:02.727862] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T17:29:03.451473] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-07. [2025-10-20T17:29:03.932100] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T17:29:04.251355] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-07. [2025-10-20T17:29:04.604777] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T17:29:05.051349] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-05. [2025-10-20T17:29:05.130105] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T17:29:05.851424] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-08. [2025-10-20T17:29:06.591532] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T17:29:06.651343] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-03. [2025-10-20T17:29:06.957795] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:29:07.451393] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-09. [2025-10-20T17:29:07.578165] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T17:29:08.251381] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-08. [2025-10-20T17:29:08.422233] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T17:29:09.051432] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-10. [2025-10-20T17:29:09.359845] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:29:09.851375] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-04. [2025-10-20T17:29:10.362623] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:29:10.366858] [INFO ] [REMOTE ] BCX: Full fetch complete (33678 records). [2025-10-20T17:29:10.366935] [DEBUG] [VALIDATOR ] BCX: Validating primary data. [2025-10-20T17:29:10.658121] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-06. [2025-10-20T17:29:10.752225] [DEBUG] [VALIDATOR ] BCX: All expected trading days are present. [2025-10-20T17:29:10.752304] [DEBUG] [VALIDATOR ] BCX: Primary data validation passed. [2025-10-20T17:29:10.752787] [INFO ] [MAIN ] BCX: Fetched primary data is valid. [2025-10-20T17:29:10.752861] [DEBUG] [LOCAL ] BCX: Saving primary data. [2025-10-20T17:29:10.753587] [DEBUG] [LOCAL ] BCX: Trimmed 33678 records to 22104 (keeping 200 days of data) [2025-10-20T17:29:10.881816] [DEBUG] [LOCAL ] BCX: Primary data saved (22104 records). [2025-10-20T17:29:10.881938] [INFO ] [MAIN ] BCX: Full history saved. [2025-10-20T17:29:10.882010] [INFO ] [MAIN ] Update Progress: 237/2367 symbols completed. BCX took 22.5s. Estimated remaining: 4h 21m 36s. Rate: 8.1 symbols/min [2025-10-20T17:29:11.143219] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T17:29:11.451388] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-05. [2025-10-20T17:29:12.251375] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-09. [2025-10-20T17:29:12.578858] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:29:12.640166] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T17:29:13.051424] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-07. [2025-10-20T17:29:13.758694] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T17:29:13.851398] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-10. [2025-10-20T17:29:14.651403] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-06. [2025-10-20T17:29:15.059864] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T17:29:15.157221] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:29:15.162660] [INFO ] [REMOTE ] BCSF: Full fetch complete (39744 records). [2025-10-20T17:29:15.162755] [DEBUG] [VALIDATOR ] BCSF: Validating primary data. [2025-10-20T17:29:15.451602] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-08. [2025-10-20T17:29:15.540969] [DEBUG] [VALIDATOR ] BCSF: All expected trading days are present. [2025-10-20T17:29:15.541104] [DEBUG] [VALIDATOR ] BCSF: Primary data validation passed. [2025-10-20T17:29:15.541721] [INFO ] [MAIN ] BCSF: Fetched primary data is valid. [2025-10-20T17:29:15.541781] [DEBUG] [LOCAL ] BCSF: Saving primary data. [2025-10-20T17:29:15.542714] [DEBUG] [LOCAL ] BCSF: Trimmed 39744 records to 28311 (keeping 200 days of data) [2025-10-20T17:29:15.711144] [DEBUG] [LOCAL ] BCSF: Primary data saved (28311 records). [2025-10-20T17:29:15.711277] [INFO ] [MAIN ] BCSF: Full history saved. [2025-10-20T17:29:15.711352] [INFO ] [MAIN ] Update Progress: 238/2367 symbols completed. BCSF took 27.5s. Estimated remaining: 4h 21m 6s. Rate: 8.2 symbols/min [2025-10-20T17:29:15.791134] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:29:16.251459] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-07. [2025-10-20T17:29:17.365470] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:29:17.377299] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-09. [2025-10-20T17:29:17.465587] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T17:29:17.851495] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-08. [2025-10-20T17:29:18.966396] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:29:18.972316] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-09. [2025-10-20T17:29:19.290834] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T17:29:19.451428] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-10. [2025-10-20T17:29:20.084195] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T17:29:20.251398] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-10. [2025-10-20T17:29:21.221559] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T17:29:21.229000] [INFO ] [REMOTE ] BCS: Full fetch complete (77131 records). [2025-10-20T17:29:21.229163] [DEBUG] [VALIDATOR ] BCS: Validating primary data. [2025-10-20T17:29:21.229997] [FATAL] [VALIDATOR ] BCS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:29:21.230091] [FATAL] [MAIN ] BCS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:29:21.230135] [WARN ] [MAIN ] BCS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:29:21.230199] [INFO ] [LOCAL ] BCS: Starting deletion of primary data. [2025-10-20T17:29:21.230279] [DEBUG] [LOCAL ] BCS: No primary data file found to delete. [2025-10-20T17:29:21.230330] [INFO ] [LOCAL ] BCS: Starting deletion of derivative data. [2025-10-20T17:29:21.230535] [DEBUG] [LOCAL ] BCS: Deleted derivative data file. [2025-10-20T17:29:21.230681] [INFO ] [MAIN ] Update Progress: 239/2367 symbols completed. BCS took 45.4s. Estimated remaining: 4h 20m 42s. Rate: 8.2 symbols/min [2025-10-20T17:29:21.359947] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T17:29:21.363814] [INFO ] [REMOTE ] BDC: Full fetch complete (35779 records). [2025-10-20T17:29:21.363914] [DEBUG] [VALIDATOR ] BDC: Validating primary data. [2025-10-20T17:29:21.783918] [DEBUG] [VALIDATOR ] BDC: All expected trading days are present. [2025-10-20T17:29:21.784005] [DEBUG] [VALIDATOR ] BDC: Primary data validation passed. [2025-10-20T17:29:21.784448] [INFO ] [MAIN ] BDC: Fetched primary data is valid. [2025-10-20T17:29:21.784489] [DEBUG] [LOCAL ] BDC: Saving primary data. [2025-10-20T17:29:21.785431] [DEBUG] [LOCAL ] BDC: Trimmed 35779 records to 24267 (keeping 200 days of data) [2025-10-20T17:29:21.939211] [DEBUG] [LOCAL ] BDC: Primary data saved (24267 records). [2025-10-20T17:29:21.939373] [INFO ] [MAIN ] BDC: Full history saved. [2025-10-20T17:29:21.939468] [INFO ] [MAIN ] Update Progress: 240/2367 symbols completed. BDC took 27.1s. Estimated remaining: 4h 19m 36s. Rate: 8.2 symbols/min [2025-10-20T17:29:22.578092] [INFO ] [MAIN ] BDJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:22.562524] [INFO ] [MAIN ] BDX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:22.628370] [INFO ] [MAIN ] BE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:22.685771] [INFO ] [MAIN ] BDN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:22.686202] [DEBUG] [LOCAL ] BDN: No existing data file found in primary storage. [2025-10-20T17:29:22.686259] [WARN ] [MAIN ] BDN: No existing data found, performing full refresh instead. [2025-10-20T17:29:22.686363] [INFO ] [MAIN ] BDN: Wiping existing primary data before full refresh. [2025-10-20T17:29:22.686407] [INFO ] [LOCAL ] BDN: Starting deletion of primary data. [2025-10-20T17:29:22.686455] [DEBUG] [LOCAL ] BDN: No primary data file found to delete. [2025-10-20T17:29:22.686514] [INFO ] [MAIN ] BDN: Starting full primary refresh. [2025-10-20T17:29:22.931464] [DEBUG] [LOCAL ] BDJ: Loaded 29832 existing records from primary storage. [2025-10-20T17:29:23.043203] [INFO ] [MAIN ] BDJ: Loaded 29832 existing records. [2025-10-20T17:29:23.044213] [DEBUG] [LOCAL ] BDX: Loaded 52951 existing records from primary storage. [2025-10-20T17:29:23.104579] [INFO ] [MAIN ] BDX: Loaded 52951 existing records. [2025-10-20T17:29:23.227556] [INFO ] [REMOTE ] BDN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:23.301159] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-01. [2025-10-20T17:29:23.370550] [DEBUG] [LOCAL ] BE: Loaded 54441 existing records from primary storage. [2025-10-20T17:29:23.396614] [INFO ] [MAIN ] BE: Loaded 54441 existing records. [2025-10-20T17:29:23.508447] [WARN ] [MAIN ] BDJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:23.542318] [INFO ] [MAIN ] BDJ: Wiping existing primary data before full refresh. [2025-10-20T17:29:23.549430] [INFO ] [LOCAL ] BDJ: Starting deletion of primary data. [2025-10-20T17:29:23.600460] [DEBUG] [LOCAL ] BDJ: Deleted primary data file. [2025-10-20T17:29:23.612303] [INFO ] [MAIN ] BDJ: Starting full primary refresh. [2025-10-20T17:29:23.612095] [WARN ] [MAIN ] BDX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:23.634036] [INFO ] [MAIN ] BDX: Wiping existing primary data before full refresh. [2025-10-20T17:29:23.656408] [INFO ] [LOCAL ] BDX: Starting deletion of primary data. [2025-10-20T17:29:23.684404] [DEBUG] [LOCAL ] BDX: Deleted primary data file. [2025-10-20T17:29:23.684523] [INFO ] [MAIN ] BDX: Starting full primary refresh. [2025-10-20T17:29:23.889041] [WARN ] [MAIN ] BE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:23.942468] [INFO ] [MAIN ] BE: Wiping existing primary data before full refresh. [2025-10-20T17:29:23.954144] [INFO ] [LOCAL ] BE: Starting deletion of primary data. [2025-10-20T17:29:23.990418] [DEBUG] [LOCAL ] BE: Deleted primary data file. [2025-10-20T17:29:24.015417] [INFO ] [MAIN ] BE: Starting full primary refresh. [2025-10-20T17:29:24.677136] [INFO ] [REMOTE ] BDX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:24.791372] [INFO ] [REMOTE ] BDJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:24.791942] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-01. [2025-10-20T17:29:24.819113] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-01. [2025-10-20T17:29:24.943175] [INFO ] [REMOTE ] BE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:24.943499] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-01. [2025-10-20T17:29:25.167136] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T17:29:25.176968] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-02. [2025-10-20T17:29:26.273358] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:29:26.279822] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-02. [2025-10-20T17:29:26.453128] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:29:26.463739] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-02. [2025-10-20T17:29:26.488370] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T17:29:26.651424] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-03. [2025-10-20T17:29:27.134206] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-01 (2.19 seconds). [2025-10-20T17:29:27.451403] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-02. [2025-10-20T17:29:27.521896] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T17:29:27.885825] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T17:29:28.179012] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:29:28.251381] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-03. [2025-10-20T17:29:29.051349] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-04. [2025-10-20T17:29:29.172746] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:29:29.851346] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-03. [2025-10-20T17:29:30.132623] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T17:29:30.651357] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-03. [2025-10-20T17:29:30.759303] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:29:31.011769] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T17:29:31.451393] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-05. [2025-10-20T17:29:32.251429] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-04. [2025-10-20T17:29:32.516223] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T17:29:32.760317] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T17:29:33.051394] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-04. [2025-10-20T17:29:33.291783] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T17:29:33.851419] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-06. [2025-10-20T17:29:34.651366] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-04. [2025-10-20T17:29:35.020886] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T17:29:35.451418] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-05. [2025-10-20T17:29:35.536700] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:29:36.251431] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-07. [2025-10-20T17:29:36.295159] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:29:36.410708] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T17:29:37.051372] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-06. [2025-10-20T17:29:37.851401] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-05. [2025-10-20T17:29:37.998737] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T17:29:38.314487] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T17:29:38.651408] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-07. [2025-10-20T17:29:39.451442] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-05. [2025-10-20T17:29:39.767796] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T17:29:40.040593] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:29:40.251433] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-08. [2025-10-20T17:29:41.051354] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-08. [2025-10-20T17:29:41.183310] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T17:29:41.496218] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T17:29:41.851370] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-09. [2025-10-20T17:29:42.295494] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T17:29:42.651390] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-06. [2025-10-20T17:29:42.972918] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T17:29:43.451380] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-09. [2025-10-20T17:29:44.032893] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T17:29:44.251379] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-07. [2025-10-20T17:29:45.051358] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-10. [2025-10-20T17:29:45.130736] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:29:45.851377] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-06. [2025-10-20T17:29:46.182420] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T17:29:46.344340] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T17:29:46.349126] [INFO ] [REMOTE ] BDJ: Full fetch complete (42853 records). [2025-10-20T17:29:46.349220] [DEBUG] [VALIDATOR ] BDJ: Validating primary data. [2025-10-20T17:29:46.653487] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-08. [2025-10-20T17:29:46.748926] [DEBUG] [VALIDATOR ] BDJ: All expected trading days are present. [2025-10-20T17:29:46.749006] [DEBUG] [VALIDATOR ] BDJ: Primary data validation passed. [2025-10-20T17:29:46.749555] [INFO ] [MAIN ] BDJ: Fetched primary data is valid. [2025-10-20T17:29:46.749605] [DEBUG] [LOCAL ] BDJ: Saving primary data. [2025-10-20T17:29:46.750835] [DEBUG] [LOCAL ] BDJ: Trimmed 42853 records to 29444 (keeping 200 days of data) [2025-10-20T17:29:46.922787] [DEBUG] [LOCAL ] BDJ: Primary data saved (29444 records). [2025-10-20T17:29:46.922998] [INFO ] [MAIN ] BDJ: Full history saved. [2025-10-20T17:29:46.923087] [INFO ] [MAIN ] Update Progress: 241/2367 symbols completed. BDJ took 25.0s. Estimated remaining: 4h 22m 5s. Rate: 8.1 symbols/min [2025-10-20T17:29:47.102574] [INFO ] [MAIN ] BEDU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:47.102768] [DEBUG] [LOCAL ] BEDU: No existing data file found in primary storage. [2025-10-20T17:29:47.102812] [WARN ] [MAIN ] BEDU: No existing data found, performing full refresh instead. [2025-10-20T17:29:47.102923] [INFO ] [MAIN ] BEDU: Wiping existing primary data before full refresh. [2025-10-20T17:29:47.102963] [INFO ] [LOCAL ] BEDU: Starting deletion of primary data. [2025-10-20T17:29:47.103004] [DEBUG] [LOCAL ] BEDU: No primary data file found to delete. [2025-10-20T17:29:47.103037] [INFO ] [MAIN ] BEDU: Starting full primary refresh. [2025-10-20T17:29:47.469235] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-10. [2025-10-20T17:29:47.472922] [INFO ] [REMOTE ] BEDU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:47.507627] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:29:48.251392] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-07. [2025-10-20T17:29:48.366452] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:29:48.922312] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T17:29:48.929490] [INFO ] [REMOTE ] BDN: Full fetch complete (69409 records). [2025-10-20T17:29:48.929622] [DEBUG] [VALIDATOR ] BDN: Validating primary data. [2025-10-20T17:29:49.056494] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-09. [2025-10-20T17:29:49.361542] [DEBUG] [VALIDATOR ] BDN: All expected trading days are present. [2025-10-20T17:29:49.361626] [DEBUG] [VALIDATOR ] BDN: Primary data validation passed. [2025-10-20T17:29:49.362384] [INFO ] [MAIN ] BDN: Fetched primary data is valid. [2025-10-20T17:29:49.362438] [DEBUG] [LOCAL ] BDN: Saving primary data. [2025-10-20T17:29:49.363967] [DEBUG] [LOCAL ] BDN: Trimmed 69409 records to 47639 (keeping 200 days of data) [2025-10-20T17:29:49.640869] [DEBUG] [LOCAL ] BDN: Primary data saved (47639 records). [2025-10-20T17:29:49.641014] [INFO ] [MAIN ] BDN: Full history saved. [2025-10-20T17:29:49.641086] [INFO ] [MAIN ] Update Progress: 242/2367 symbols completed. BDN took 27.7s. Estimated remaining: 4h 21m 16s. Rate: 8.1 symbols/min [2025-10-20T17:29:49.844521] [INFO ] [MAIN ] BEKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:49.851394] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-01. [2025-10-20T17:29:49.910030] [DEBUG] [LOCAL ] BEKE: Loaded 53980 existing records from primary storage. [2025-10-20T17:29:49.910540] [INFO ] [MAIN ] BEKE: Loaded 53980 existing records. [2025-10-20T17:29:50.176075] [WARN ] [MAIN ] BEKE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:50.176744] [INFO ] [MAIN ] BEKE: Wiping existing primary data before full refresh. [2025-10-20T17:29:50.176938] [INFO ] [LOCAL ] BEKE: Starting deletion of primary data. [2025-10-20T17:29:50.177369] [DEBUG] [LOCAL ] BEKE: Deleted primary data file. [2025-10-20T17:29:50.177434] [INFO ] [MAIN ] BEKE: Starting full primary refresh. [2025-10-20T17:29:50.378499] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-10-20T17:29:50.657434] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-02. [2025-10-20T17:29:50.673361] [INFO ] [REMOTE ] BEKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:50.724087] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-07 (2.47 seconds). [2025-10-20T17:29:50.785055] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T17:29:51.027560] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T17:29:51.451443] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-10. [2025-10-20T17:29:52.251440] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-01. [2025-10-20T17:29:52.985770] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T17:29:52.993214] [INFO ] [REMOTE ] BDX: Full fetch complete (75508 records). [2025-10-20T17:29:52.993344] [DEBUG] [VALIDATOR ] BDX: Validating primary data. [2025-10-20T17:29:53.056523] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-08. [2025-10-20T17:29:53.404230] [DEBUG] [VALIDATOR ] BDX: All expected trading days are present. [2025-10-20T17:29:53.404308] [DEBUG] [VALIDATOR ] BDX: Primary data validation passed. [2025-10-20T17:29:53.405069] [INFO ] [MAIN ] BDX: Fetched primary data is valid. [2025-10-20T17:29:53.405131] [DEBUG] [LOCAL ] BDX: Saving primary data. [2025-10-20T17:29:53.406750] [DEBUG] [LOCAL ] BDX: Trimmed 75508 records to 52166 (keeping 200 days of data) [2025-10-20T17:29:53.729475] [DEBUG] [LOCAL ] BDX: Primary data saved (52166 records). [2025-10-20T17:29:53.729601] [INFO ] [MAIN ] BDX: Full history saved. [2025-10-20T17:29:53.729699] [INFO ] [MAIN ] Update Progress: 243/2367 symbols completed. BDX took 31.8s. Estimated remaining: 4h 20m 40s. Rate: 8.1 symbols/min [2025-10-20T17:29:53.854088] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-03. [2025-10-20T17:29:53.904442] [INFO ] [MAIN ] BEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:53.954352] [DEBUG] [LOCAL ] BEN: Loaded 53896 existing records from primary storage. [2025-10-20T17:29:53.954697] [INFO ] [MAIN ] BEN: Loaded 53896 existing records. [2025-10-20T17:29:54.131779] [WARN ] [MAIN ] BEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:54.132153] [INFO ] [MAIN ] BEN: Wiping existing primary data before full refresh. [2025-10-20T17:29:54.132242] [INFO ] [LOCAL ] BEN: Starting deletion of primary data. [2025-10-20T17:29:54.148055] [DEBUG] [LOCAL ] BEN: Deleted primary data file. [2025-10-20T17:29:54.161218] [INFO ] [MAIN ] BEN: Starting full primary refresh. [2025-10-20T17:29:54.238407] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T17:29:54.417823] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T17:29:54.530940] [INFO ] [REMOTE ] BEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:54.651338] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-01. [2025-10-20T17:29:55.451357] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-04. [2025-10-20T17:29:55.621581] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-08 (2.57 seconds). [2025-10-20T17:29:55.879819] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T17:29:56.251346] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-02. [2025-10-20T17:29:56.515612] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:29:57.051459] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-05. [2025-10-20T17:29:57.387291] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-05 (0.34 seconds). [2025-10-20T17:29:57.851419] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-09. [2025-10-20T17:29:58.147177] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T17:29:58.651355] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-03. [2025-10-20T17:29:59.451397] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-02. [2025-10-20T17:30:00.071550] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T17:30:00.251421] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-10. [2025-10-20T17:30:00.603361] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T17:30:00.835361] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T17:30:01.051406] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-06. [2025-10-20T17:30:01.533463] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T17:30:01.851371] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-07. [2025-10-20T17:30:02.104687] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-07 (0.25 seconds). [2025-10-20T17:30:02.651389] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-08. [2025-10-20T17:30:02.857926] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-08 (0.21 seconds). [2025-10-20T17:30:03.451405] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-03. [2025-10-20T17:30:03.561632] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-10 (3.31 seconds). [2025-10-20T17:30:03.570713] [INFO ] [REMOTE ] BE: Full fetch complete (77426 records). [2025-10-20T17:30:03.570960] [DEBUG] [VALIDATOR ] BE: Validating primary data. [2025-10-20T17:30:03.572174] [FATAL] [VALIDATOR ] BE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:30:03.572292] [FATAL] [MAIN ] BE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:30:03.572369] [WARN ] [MAIN ] BE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:30:03.572461] [INFO ] [LOCAL ] BE: Starting deletion of primary data. [2025-10-20T17:30:03.572572] [DEBUG] [LOCAL ] BE: No primary data file found to delete. [2025-10-20T17:30:03.572700] [INFO ] [LOCAL ] BE: Starting deletion of derivative data. [2025-10-20T17:30:03.572894] [DEBUG] [LOCAL ] BE: Deleted derivative data file. [2025-10-20T17:30:03.573005] [INFO ] [MAIN ] Update Progress: 244/2367 symbols completed. BE took 41.6s. Estimated remaining: 4h 20m 54s. Rate: 8.1 symbols/min [2025-10-20T17:30:03.760820] [INFO ] [MAIN ] BEP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:03.806077] [DEBUG] [LOCAL ] BEP: Loaded 39645 existing records from primary storage. [2025-10-20T17:30:03.806409] [INFO ] [MAIN ] BEP: Loaded 39645 existing records. [2025-10-20T17:30:03.990742] [WARN ] [MAIN ] BEP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:03.991021] [INFO ] [MAIN ] BEP: Wiping existing primary data before full refresh. [2025-10-20T17:30:03.991077] [INFO ] [LOCAL ] BEP: Starting deletion of primary data. [2025-10-20T17:30:03.991304] [DEBUG] [LOCAL ] BEP: Deleted primary data file. [2025-10-20T17:30:03.991360] [INFO ] [MAIN ] BEP: Starting full primary refresh. [2025-10-20T17:30:04.256694] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-04. [2025-10-20T17:30:04.363561] [INFO ] [REMOTE ] BEP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:05.051369] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-01. [2025-10-20T17:30:05.425523] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T17:30:05.851408] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-04. [2025-10-20T17:30:06.401724] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T17:30:06.651357] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-05. [2025-10-20T17:30:06.813973] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T17:30:07.451345] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-02. [2025-10-20T17:30:07.682566] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T17:30:08.251360] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-05. [2025-10-20T17:30:08.487012] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T17:30:08.901534] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T17:30:09.051375] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-09. [2025-10-20T17:30:09.368229] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-10-20T17:30:09.851415] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-06. [2025-10-20T17:30:10.268592] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-05 (2.02 seconds). [2025-10-20T17:30:10.651397] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-03. [2025-10-20T17:30:11.451376] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-06. [2025-10-20T17:30:11.649919] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T17:30:12.062599] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T17:30:12.251447] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-07. [2025-10-20T17:30:13.051368] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-10. [2025-10-20T17:30:13.224230] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T17:30:13.605575] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T17:30:13.606931] [INFO ] [REMOTE ] BEDU: Full fetch complete (1320 records). [2025-10-20T17:30:13.607022] [DEBUG] [VALIDATOR ] BEDU: Validating primary data. [2025-10-20T17:30:13.797840] [WARN ] [VALIDATOR ] BEDU: Validation failed - insufficient historical data. Have 169 unique days, need 200 days [2025-10-20T17:30:13.798071] [FATAL] [MAIN ] BEDU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:30:13.798140] [WARN ] [MAIN ] BEDU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:30:13.798227] [INFO ] [LOCAL ] BEDU: Starting deletion of primary data. [2025-10-20T17:30:13.798289] [DEBUG] [LOCAL ] BEDU: No primary data file found to delete. [2025-10-20T17:30:13.798324] [INFO ] [LOCAL ] BEDU: Starting deletion of derivative data. [2025-10-20T17:30:13.798411] [DEBUG] [LOCAL ] BEDU: No derivative data file found to delete. [2025-10-20T17:30:13.798473] [INFO ] [MAIN ] Update Progress: 245/2367 symbols completed. BEDU took 26.9s. Estimated remaining: 4h 21m 12s. Rate: 8.1 symbols/min [2025-10-20T17:30:13.859560] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-04. [2025-10-20T17:30:13.978126] [INFO ] [MAIN ] BEPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:14.021218] [DEBUG] [LOCAL ] BEPC: Loaded 47230 existing records from primary storage. [2025-10-20T17:30:14.021556] [INFO ] [MAIN ] BEPC: Loaded 47230 existing records. [2025-10-20T17:30:14.186599] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T17:30:14.254632] [WARN ] [MAIN ] BEPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:14.254997] [INFO ] [MAIN ] BEPC: Wiping existing primary data before full refresh. [2025-10-20T17:30:14.255358] [INFO ] [LOCAL ] BEPC: Starting deletion of primary data. [2025-10-20T17:30:14.255577] [DEBUG] [LOCAL ] BEPC: Deleted primary data file. [2025-10-20T17:30:14.255616] [INFO ] [MAIN ] BEPC: Starting full primary refresh. [2025-10-20T17:30:14.629190] [INFO ] [REMOTE ] BEPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:14.651383] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-01. [2025-10-20T17:30:15.456609] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-07. [2025-10-20T17:30:15.469543] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:30:16.251448] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-08. [2025-10-20T17:30:16.580753] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:30:17.051521] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-02. [2025-10-20T17:30:17.335885] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T17:30:17.851465] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-08. [2025-10-20T17:30:18.651357] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-05. [2025-10-20T17:30:18.751590] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T17:30:19.451486] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-03. [2025-10-20T17:30:19.699684] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-10-20T17:30:20.222389] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-08 (3.97 seconds). [2025-10-20T17:30:20.256849] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-09. [2025-10-20T17:30:20.261022] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:30:20.843959] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:30:21.051389] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-06. [2025-10-20T17:30:21.851368] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-04. [2025-10-20T17:30:22.106554] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:30:22.602463] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:30:22.651397] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-07. [2025-10-20T17:30:23.451406] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-10. [2025-10-20T17:30:23.541155] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:30:24.075602] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:30:24.251373] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-08. [2025-10-20T17:30:25.051375] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-09. [2025-10-20T17:30:25.192436] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:30:25.200295] [INFO ] [REMOTE ] BEKE: Full fetch complete (77332 records). [2025-10-20T17:30:25.200439] [DEBUG] [VALIDATOR ] BEKE: Validating primary data. [2025-10-20T17:30:25.594879] [DEBUG] [VALIDATOR ] BEKE: All expected trading days are present. [2025-10-20T17:30:25.595073] [DEBUG] [VALIDATOR ] BEKE: Primary data validation passed. [2025-10-20T17:30:25.596051] [INFO ] [MAIN ] BEKE: Fetched primary data is valid. [2025-10-20T17:30:25.596136] [DEBUG] [LOCAL ] BEKE: Saving primary data. [2025-10-20T17:30:25.597706] [DEBUG] [LOCAL ] BEKE: Trimmed 77332 records to 53198 (keeping 200 days of data) [2025-10-20T17:30:25.713356] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T17:30:25.856494] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-09. [2025-10-20T17:30:25.955500] [DEBUG] [LOCAL ] BEKE: Primary data saved (53198 records). [2025-10-20T17:30:25.955679] [INFO ] [MAIN ] BEKE: Full history saved. [2025-10-20T17:30:25.955781] [INFO ] [MAIN ] Update Progress: 246/2367 symbols completed. BEKE took 36.3s. Estimated remaining: 4h 21m 45s. Rate: 8.1 symbols/min [2025-10-20T17:30:26.141313] [INFO ] [MAIN ] BEPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:26.155419] [DEBUG] [LOCAL ] BEPH: Loaded 5233 existing records from primary storage. [2025-10-20T17:30:26.155689] [INFO ] [MAIN ] BEPH: Loaded 5233 existing records. [2025-10-20T17:30:26.358604] [WARN ] [MAIN ] BEPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:26.359251] [INFO ] [MAIN ] BEPH: Wiping existing primary data before full refresh. [2025-10-20T17:30:26.359425] [INFO ] [LOCAL ] BEPH: Starting deletion of primary data. [2025-10-20T17:30:26.359616] [DEBUG] [LOCAL ] BEPH: Deleted primary data file. [2025-10-20T17:30:26.359712] [INFO ] [MAIN ] BEPH: Starting full primary refresh. [2025-10-20T17:30:26.653416] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-05. [2025-10-20T17:30:26.720912] [INFO ] [REMOTE ] BEPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:26.751271] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:30:27.416261] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T17:30:27.451376] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-10. [2025-10-20T17:30:27.913685] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T17:30:28.251426] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-10. [2025-10-20T17:30:29.051425] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-01. [2025-10-20T17:30:29.317149] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T17:30:29.324758] [INFO ] [REMOTE ] BEN: Full fetch complete (76695 records). [2025-10-20T17:30:29.324946] [DEBUG] [VALIDATOR ] BEN: Validating primary data. [2025-10-20T17:30:29.325836] [FATAL] [VALIDATOR ] BEN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:30:29.325937] [FATAL] [MAIN ] BEN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:30:29.325981] [WARN ] [MAIN ] BEN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:30:29.326021] [INFO ] [LOCAL ] BEN: Starting deletion of primary data. [2025-10-20T17:30:29.326101] [DEBUG] [LOCAL ] BEN: No primary data file found to delete. [2025-10-20T17:30:29.326137] [INFO ] [LOCAL ] BEN: Starting deletion of derivative data. [2025-10-20T17:30:29.326268] [DEBUG] [LOCAL ] BEN: Deleted derivative data file. [2025-10-20T17:30:29.326339] [INFO ] [MAIN ] Update Progress: 247/2367 symbols completed. BEN took 35.6s. Estimated remaining: 4h 21m 3s. Rate: 8.1 symbols/min [2025-10-20T17:30:29.501883] [INFO ] [MAIN ] BEPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:29.507921] [DEBUG] [LOCAL ] BEPI: Loaded 3648 existing records from primary storage. [2025-10-20T17:30:29.508139] [INFO ] [MAIN ] BEPI: Loaded 3648 existing records. [2025-10-20T17:30:29.689235] [WARN ] [MAIN ] BEPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:29.689539] [INFO ] [MAIN ] BEPI: Wiping existing primary data before full refresh. [2025-10-20T17:30:29.689593] [INFO ] [LOCAL ] BEPI: Starting deletion of primary data. [2025-10-20T17:30:29.689766] [DEBUG] [LOCAL ] BEPI: Deleted primary data file. [2025-10-20T17:30:29.689820] [INFO ] [MAIN ] BEPI: Starting full primary refresh. [2025-10-20T17:30:29.867746] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-06. [2025-10-20T17:30:29.948067] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T17:30:29.988270] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:30:30.032068] [INFO ] [REMOTE ] BEP: Full fetch complete (59297 records). [2025-10-20T17:30:30.042579] [DEBUG] [VALIDATOR ] BEP: Validating primary data. [2025-10-20T17:30:30.134106] [INFO ] [REMOTE ] BEPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:30.488242] [DEBUG] [VALIDATOR ] BEP: All expected trading days are present. [2025-10-20T17:30:30.488353] [DEBUG] [VALIDATOR ] BEP: Primary data validation passed. [2025-10-20T17:30:30.489020] [INFO ] [MAIN ] BEP: Fetched primary data is valid. [2025-10-20T17:30:30.489085] [DEBUG] [LOCAL ] BEP: Saving primary data. [2025-10-20T17:30:30.490323] [DEBUG] [LOCAL ] BEP: Trimmed 59297 records to 39006 (keeping 200 days of data) [2025-10-20T17:30:30.654370] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-01. [2025-10-20T17:30:30.720866] [DEBUG] [LOCAL ] BEP: Primary data saved (39006 records). [2025-10-20T17:30:30.721005] [INFO ] [MAIN ] BEP: Full history saved. [2025-10-20T17:30:30.721374] [INFO ] [MAIN ] Update Progress: 248/2367 symbols completed. BEP took 27.1s. Estimated remaining: 4h 20m 5s. Rate: 8.1 symbols/min [2025-10-20T17:30:30.887549] [INFO ] [MAIN ] BEPJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:30.893285] [DEBUG] [LOCAL ] BEPJ: Loaded 3122 existing records from primary storage. [2025-10-20T17:30:30.893476] [INFO ] [MAIN ] BEPJ: Loaded 3122 existing records. [2025-10-20T17:30:31.067140] [WARN ] [MAIN ] BEPJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:31.067370] [INFO ] [MAIN ] BEPJ: Wiping existing primary data before full refresh. [2025-10-20T17:30:31.067424] [INFO ] [LOCAL ] BEPJ: Starting deletion of primary data. [2025-10-20T17:30:31.067585] [DEBUG] [LOCAL ] BEPJ: Deleted primary data file. [2025-10-20T17:30:31.067639] [INFO ] [MAIN ] BEPJ: Starting full primary refresh. [2025-10-20T17:30:31.254896] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T17:30:31.430322] [INFO ] [REMOTE ] BEPJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:31.451330] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-01. [2025-10-20T17:30:31.609716] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T17:30:32.234861] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T17:30:32.251346] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-02. [2025-10-20T17:30:32.690147] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:30:33.051370] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-02. [2025-10-20T17:30:33.630705] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T17:30:33.851377] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-07. [2025-10-20T17:30:34.651388] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-02. [2025-10-20T17:30:35.122736] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T17:30:35.451409] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-03. [2025-10-20T17:30:35.577800] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:30:35.740821] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-10-20T17:30:36.251453] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-08. [2025-10-20T17:30:37.051451] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-04. [2025-10-20T17:30:37.761092] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:30:37.814246] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T17:30:37.851362] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-05. [2025-10-20T17:30:38.529723] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T17:30:38.651372] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-03. [2025-10-20T17:30:39.058806] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-10-20T17:30:39.451398] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-09. [2025-10-20T17:30:40.251387] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-03. [2025-10-20T17:30:40.708243] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T17:30:41.051343] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-06. [2025-10-20T17:30:41.156186] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:30:41.746837] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T17:30:41.851377] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-04. [2025-10-20T17:30:42.280625] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T17:30:42.651432] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-04. [2025-10-20T17:30:43.281950] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T17:30:43.451474] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-10. [2025-10-20T17:30:44.251380] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-07. [2025-10-20T17:30:44.811490] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T17:30:45.051374] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-05. [2025-10-20T17:30:45.088037] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T17:30:45.096587] [INFO ] [REMOTE ] BEPC: Full fetch complete (68882 records). [2025-10-20T17:30:45.096867] [DEBUG] [VALIDATOR ] BEPC: Validating primary data. [2025-10-20T17:30:45.521787] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T17:30:45.547634] [DEBUG] [VALIDATOR ] BEPC: All expected trading days are present. [2025-10-20T17:30:45.547781] [DEBUG] [VALIDATOR ] BEPC: Primary data validation passed. [2025-10-20T17:30:45.548817] [INFO ] [MAIN ] BEPC: Fetched primary data is valid. [2025-10-20T17:30:45.548913] [DEBUG] [LOCAL ] BEPC: Saving primary data. [2025-10-20T17:30:45.550855] [DEBUG] [LOCAL ] BEPC: Trimmed 68882 records to 46428 (keeping 200 days of data) [2025-10-20T17:30:45.854999] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-06. [2025-10-20T17:30:45.870207] [DEBUG] [LOCAL ] BEPC: Primary data saved (46428 records). [2025-10-20T17:30:45.870449] [INFO ] [MAIN ] BEPC: Full history saved. [2025-10-20T17:30:45.870600] [INFO ] [MAIN ] Update Progress: 249/2367 symbols completed. BEPC took 32.1s. Estimated remaining: 4h 21m 4s. Rate: 8.1 symbols/min [2025-10-20T17:30:46.052367] [INFO ] [MAIN ] BFAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:46.052634] [DEBUG] [LOCAL ] BFAM: No existing data file found in primary storage. [2025-10-20T17:30:46.052711] [WARN ] [MAIN ] BFAM: No existing data found, performing full refresh instead. [2025-10-20T17:30:46.052868] [INFO ] [MAIN ] BFAM: Wiping existing primary data before full refresh. [2025-10-20T17:30:46.052927] [INFO ] [LOCAL ] BFAM: Starting deletion of primary data. [2025-10-20T17:30:46.052971] [DEBUG] [LOCAL ] BFAM: No primary data file found to delete. [2025-10-20T17:30:46.053005] [INFO ] [MAIN ] BFAM: Starting full primary refresh. [2025-10-20T17:30:46.418716] [INFO ] [REMOTE ] BFAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:46.651391] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-05. [2025-10-20T17:30:46.710049] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T17:30:47.253374] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T17:30:47.451368] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-07. [2025-10-20T17:30:48.036773] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T17:30:48.251357] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-08. [2025-10-20T17:30:48.686388] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T17:30:49.051374] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-01. [2025-10-20T17:30:49.851385] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-06. [2025-10-20T17:30:50.443606] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T17:30:50.475798] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:30:50.651404] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-07. [2025-10-20T17:30:51.225772] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T17:30:51.451618] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-08. [2025-10-20T17:30:52.035748] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T17:30:52.251448] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-09. [2025-10-20T17:30:52.830581] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T17:30:53.051421] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-09. [2025-10-20T17:30:53.610629] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T17:30:53.851632] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-02. [2025-10-20T17:30:54.651411] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-10. [2025-10-20T17:30:55.209857] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T17:30:55.429874] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T17:30:55.431142] [INFO ] [REMOTE ] BEPH: Full fetch complete (7524 records). [2025-10-20T17:30:55.431214] [DEBUG] [VALIDATOR ] BEPH: Validating primary data. [2025-10-20T17:30:55.460178] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-10. [2025-10-20T17:30:55.793165] [DEBUG] [VALIDATOR ] BEPH: All expected trading days are present. [2025-10-20T17:30:55.793245] [DEBUG] [VALIDATOR ] BEPH: Primary data validation passed. [2025-10-20T17:30:55.793478] [INFO ] [MAIN ] BEPH: Fetched primary data is valid. [2025-10-20T17:30:55.793516] [DEBUG] [LOCAL ] BEPH: Saving primary data. [2025-10-20T17:30:55.793932] [DEBUG] [LOCAL ] BEPH: Trimmed 7524 records to 4950 (keeping 200 days of data) [2025-10-20T17:30:55.824724] [DEBUG] [LOCAL ] BEPH: Primary data saved (4950 records). [2025-10-20T17:30:55.824868] [INFO ] [MAIN ] BEPH: Full history saved. [2025-10-20T17:30:55.824948] [INFO ] [MAIN ] Update Progress: 250/2367 symbols completed. BEPH took 29.9s. Estimated remaining: 4h 21m 18s. Rate: 8.1 symbols/min [2025-10-20T17:30:55.994394] [INFO ] [MAIN ] BFH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:56.032812] [DEBUG] [LOCAL ] BFH: Loaded 42165 existing records from primary storage. [2025-10-20T17:30:56.033149] [INFO ] [MAIN ] BFH: Loaded 42165 existing records. [2025-10-20T17:30:56.116754] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T17:30:56.139892] [INFO ] [REMOTE ] BEPJ: Full fetch complete (4367 records). [2025-10-20T17:30:56.140017] [DEBUG] [VALIDATOR ] BEPJ: Validating primary data. [2025-10-20T17:30:56.282577] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-08. [2025-10-20T17:30:56.315216] [WARN ] [MAIN ] BFH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:56.328816] [INFO ] [MAIN ] BFH: Wiping existing primary data before full refresh. [2025-10-20T17:30:56.328949] [INFO ] [LOCAL ] BFH: Starting deletion of primary data. [2025-10-20T17:30:56.339927] [DEBUG] [LOCAL ] BFH: Deleted primary data file. [2025-10-20T17:30:56.345220] [INFO ] [MAIN ] BFH: Starting full primary refresh. [2025-10-20T17:30:56.906210] [DEBUG] [VALIDATOR ] BEPJ: All expected trading days are present. [2025-10-20T17:30:56.916873] [DEBUG] [VALIDATOR ] BEPJ: Primary data validation passed. [2025-10-20T17:30:56.917123] [INFO ] [MAIN ] BEPJ: Fetched primary data is valid. [2025-10-20T17:30:56.917539] [DEBUG] [LOCAL ] BEPJ: Saving primary data. [2025-10-20T17:30:56.939569] [DEBUG] [LOCAL ] BEPJ: Trimmed 4367 records to 3084 (keeping 200 days of data) [2025-10-20T17:30:57.007753] [INFO ] [REMOTE ] BFH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:57.021430] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T17:30:57.026702] [DEBUG] [LOCAL ] BEPJ: Primary data saved (3084 records). [2025-10-20T17:30:57.026771] [INFO ] [MAIN ] BEPJ: Full history saved. [2025-10-20T17:30:57.026844] [INFO ] [MAIN ] Update Progress: 251/2367 symbols completed. BEPJ took 26.3s. Estimated remaining: 4h 20m 18s. Rate: 8.1 symbols/min [2025-10-20T17:30:57.058595] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-03. [2025-10-20T17:30:57.206339] [INFO ] [MAIN ] BFK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:57.224259] [DEBUG] [LOCAL ] BFK: Loaded 13862 existing records from primary storage. [2025-10-20T17:30:57.224599] [INFO ] [MAIN ] BFK: Loaded 13862 existing records. [2025-10-20T17:30:57.410892] [WARN ] [MAIN ] BFK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:57.411190] [INFO ] [MAIN ] BFK: Wiping existing primary data before full refresh. [2025-10-20T17:30:57.411252] [INFO ] [LOCAL ] BFK: Starting deletion of primary data. [2025-10-20T17:30:57.411435] [DEBUG] [LOCAL ] BFK: Deleted primary data file. [2025-10-20T17:30:57.411475] [INFO ] [MAIN ] BFK: Starting full primary refresh. [2025-10-20T17:30:57.792404] [INFO ] [REMOTE ] BFK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:57.851445] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-01. [2025-10-20T17:30:58.460420] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T17:30:58.618411] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T17:30:58.651397] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-02. [2025-10-20T17:30:59.067530] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T17:30:59.451409] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-09. [2025-10-20T17:30:59.961798] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-10-20T17:31:00.251424] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-03. [2025-10-20T17:31:01.051480] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-10. [2025-10-20T17:31:01.310732] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T17:31:01.851382] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-04. [2025-10-20T17:31:01.923158] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T17:31:01.924600] [INFO ] [REMOTE ] BEPI: Full fetch complete (4683 records). [2025-10-20T17:31:01.924773] [DEBUG] [VALIDATOR ] BEPI: Validating primary data. [2025-10-20T17:31:02.363815] [DEBUG] [VALIDATOR ] BEPI: All expected trading days are present. [2025-10-20T17:31:02.363901] [DEBUG] [VALIDATOR ] BEPI: Primary data validation passed. [2025-10-20T17:31:02.364056] [INFO ] [MAIN ] BEPI: Fetched primary data is valid. [2025-10-20T17:31:02.364096] [DEBUG] [LOCAL ] BEPI: Saving primary data. [2025-10-20T17:31:02.364562] [DEBUG] [LOCAL ] BEPI: Trimmed 4683 records to 3594 (keeping 200 days of data) [2025-10-20T17:31:02.393934] [DEBUG] [LOCAL ] BEPI: Primary data saved (3594 records). [2025-10-20T17:31:02.394107] [INFO ] [MAIN ] BEPI: Full history saved. [2025-10-20T17:31:02.394203] [INFO ] [MAIN ] Update Progress: 252/2367 symbols completed. BEPI took 33.1s. Estimated remaining: 4h 19m 54s. Rate: 8.1 symbols/min [2025-10-20T17:31:02.614575] [INFO ] [MAIN ] BFLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:02.651388] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-04. [2025-10-20T17:31:02.672157] [DEBUG] [LOCAL ] BFLY: Loaded 52605 existing records from primary storage. [2025-10-20T17:31:02.672522] [INFO ] [MAIN ] BFLY: Loaded 52605 existing records. [2025-10-20T17:31:02.881614] [WARN ] [MAIN ] BFLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:31:02.881856] [INFO ] [MAIN ] BFLY: Wiping existing primary data before full refresh. [2025-10-20T17:31:02.881902] [INFO ] [LOCAL ] BFLY: Starting deletion of primary data. [2025-10-20T17:31:02.882100] [DEBUG] [LOCAL ] BFLY: Deleted primary data file. [2025-10-20T17:31:02.882138] [INFO ] [MAIN ] BFLY: Starting full primary refresh. [2025-10-20T17:31:03.263569] [INFO ] [REMOTE ] BFLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:03.431763] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:31:03.451351] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-05. [2025-10-20T17:31:04.060768] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T17:31:04.251367] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-01. [2025-10-20T17:31:04.320362] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T17:31:05.051377] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-05. [2025-10-20T17:31:05.851413] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-06. [2025-10-20T17:31:05.881071] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:31:06.413179] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T17:31:06.536096] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T17:31:06.651418] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-07. [2025-10-20T17:31:07.365589] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T17:31:07.451428] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-02. [2025-10-20T17:31:08.251368] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-06. [2025-10-20T17:31:08.912900] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:31:09.051436] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-03. [2025-10-20T17:31:09.601830] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:31:09.851399] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-07. [2025-10-20T17:31:10.651432] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-08. [2025-10-20T17:31:10.729731] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T17:31:11.248296] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T17:31:11.451381] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-04. [2025-10-20T17:31:11.486855] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T17:31:12.251383] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-09. [2025-10-20T17:31:12.886613] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T17:31:12.939528] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T17:31:13.051387] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-08. [2025-10-20T17:31:13.851441] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-05. [2025-10-20T17:31:14.656518] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-01. [2025-10-20T17:31:14.673622] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T17:31:15.347830] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T17:31:15.451385] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-09. [2025-10-20T17:31:16.251381] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-06. [2025-10-20T17:31:16.632397] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T17:31:16.833939] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:31:17.051425] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-10. [2025-10-20T17:31:17.832622] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T17:31:17.835812] [INFO ] [REMOTE ] BFK: Full fetch complete (18924 records). [2025-10-20T17:31:17.836059] [DEBUG] [VALIDATOR ] BFK: Validating primary data. [2025-10-20T17:31:17.856577] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-02. [2025-10-20T17:31:17.860800] [FATAL] [VALIDATOR ] BFK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:31:17.862523] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T17:31:17.870960] [FATAL] [MAIN ] BFK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:31:17.871749] [WARN ] [MAIN ] BFK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:31:17.871833] [INFO ] [LOCAL ] BFK: Starting deletion of primary data. [2025-10-20T17:31:17.871947] [DEBUG] [LOCAL ] BFK: No primary data file found to delete. [2025-10-20T17:31:17.872356] [INFO ] [LOCAL ] BFK: Starting deletion of derivative data. [2025-10-20T17:31:17.872545] [DEBUG] [LOCAL ] BFK: Deleted derivative data file. [2025-10-20T17:31:17.872685] [INFO ] [MAIN ] Update Progress: 253/2367 symbols completed. BFK took 20.8s. Estimated remaining: 4h 20m 54s. Rate: 8.1 symbols/min [2025-10-20T17:31:18.042565] [INFO ] [MAIN ] BFS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:18.042808] [DEBUG] [LOCAL ] BFS: No existing data file found in primary storage. [2025-10-20T17:31:18.042860] [WARN ] [MAIN ] BFS: No existing data found, performing full refresh instead. [2025-10-20T17:31:18.042970] [INFO ] [MAIN ] BFS: Wiping existing primary data before full refresh. [2025-10-20T17:31:18.043010] [INFO ] [LOCAL ] BFS: Starting deletion of primary data. [2025-10-20T17:31:18.043050] [DEBUG] [LOCAL ] BFS: No primary data file found to delete. [2025-10-20T17:31:18.043081] [INFO ] [MAIN ] BFS: Starting full primary refresh. [2025-10-20T17:31:18.396145] [INFO ] [REMOTE ] BFS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:18.651467] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-07. [2025-10-20T17:31:19.396778] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:31:19.451466] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-03. [2025-10-20T17:31:20.154598] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:31:20.251362] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-08. [2025-10-20T17:31:20.948405] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T17:31:21.051385] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-01. [2025-10-20T17:31:21.289456] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T17:31:21.758748] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T17:31:21.851396] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-10. [2025-10-20T17:31:22.651437] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-04. [2025-10-20T17:31:23.308138] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T17:31:23.313689] [INFO ] [REMOTE ] BFAM: Full fetch complete (47340 records). [2025-10-20T17:31:23.313806] [DEBUG] [VALIDATOR ] BFAM: Validating primary data. [2025-10-20T17:31:23.461269] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-09. [2025-10-20T17:31:23.709224] [DEBUG] [VALIDATOR ] BFAM: All expected trading days are present. [2025-10-20T17:31:23.709342] [DEBUG] [VALIDATOR ] BFAM: Primary data validation passed. [2025-10-20T17:31:23.710106] [INFO ] [MAIN ] BFAM: Fetched primary data is valid. [2025-10-20T17:31:23.710172] [DEBUG] [LOCAL ] BFAM: Saving primary data. [2025-10-20T17:31:23.711433] [DEBUG] [LOCAL ] BFAM: Trimmed 47340 records to 33897 (keeping 200 days of data) [2025-10-20T17:31:23.918625] [DEBUG] [LOCAL ] BFAM: Primary data saved (33897 records). [2025-10-20T17:31:23.918816] [INFO ] [MAIN ] BFAM: Full history saved. [2025-10-20T17:31:23.918904] [INFO ] [MAIN ] Update Progress: 254/2367 symbols completed. BFAM took 38.0s. Estimated remaining: 4h 20m 35s. Rate: 8.1 symbols/min [2025-10-20T17:31:24.092418] [INFO ] [MAIN ] BFZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:24.107081] [DEBUG] [LOCAL ] BFZ: Loaded 12201 existing records from primary storage. [2025-10-20T17:31:24.107347] [INFO ] [MAIN ] BFZ: Loaded 12201 existing records. [2025-10-20T17:31:24.265671] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-02. [2025-10-20T17:31:24.301045] [WARN ] [MAIN ] BFZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:31:24.301324] [INFO ] [MAIN ] BFZ: Wiping existing primary data before full refresh. [2025-10-20T17:31:24.301416] [INFO ] [LOCAL ] BFZ: Starting deletion of primary data. [2025-10-20T17:31:24.301579] [DEBUG] [LOCAL ] BFZ: Deleted primary data file. [2025-10-20T17:31:24.301857] [INFO ] [MAIN ] BFZ: Starting full primary refresh. [2025-10-20T17:31:24.453091] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T17:31:24.710319] [INFO ] [REMOTE ] BFZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:25.025811] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T17:31:25.060384] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-05. [2025-10-20T17:31:25.065614] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:31:25.851401] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-10. [2025-10-20T17:31:26.651403] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-03. [2025-10-20T17:31:27.027488] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T17:31:27.451360] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-06. [2025-10-20T17:31:27.480418] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T17:31:27.710065] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T17:31:27.716329] [INFO ] [REMOTE ] BFH: Full fetch complete (60198 records). [2025-10-20T17:31:27.716447] [DEBUG] [VALIDATOR ] BFH: Validating primary data. [2025-10-20T17:31:28.098498] [DEBUG] [VALIDATOR ] BFH: All expected trading days are present. [2025-10-20T17:31:28.098590] [DEBUG] [VALIDATOR ] BFH: Primary data validation passed. [2025-10-20T17:31:28.099254] [INFO ] [MAIN ] BFH: Fetched primary data is valid. [2025-10-20T17:31:28.099319] [DEBUG] [LOCAL ] BFH: Saving primary data. [2025-10-20T17:31:28.100590] [DEBUG] [LOCAL ] BFH: Trimmed 60198 records to 41581 (keeping 200 days of data) [2025-10-20T17:31:28.267387] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-04. [2025-10-20T17:31:28.344619] [DEBUG] [LOCAL ] BFH: Primary data saved (41581 records). [2025-10-20T17:31:28.344776] [INFO ] [MAIN ] BFH: Full history saved. [2025-10-20T17:31:28.344856] [INFO ] [MAIN ] Update Progress: 255/2367 symbols completed. BFH took 32.5s. Estimated remaining: 4h 20m 3s. Rate: 8.1 symbols/min [2025-10-20T17:31:28.515960] [INFO ] [MAIN ] BG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:28.562028] [DEBUG] [LOCAL ] BG: Loaded 51286 existing records from primary storage. [2025-10-20T17:31:28.562335] [INFO ] [MAIN ] BG: Loaded 51286 existing records. [2025-10-20T17:31:28.753249] [WARN ] [MAIN ] BG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:31:28.753502] [INFO ] [MAIN ] BG: Wiping existing primary data before full refresh. [2025-10-20T17:31:28.753561] [INFO ] [LOCAL ] BG: Starting deletion of primary data. [2025-10-20T17:31:28.753779] [DEBUG] [LOCAL ] BG: Deleted primary data file. [2025-10-20T17:31:28.753835] [INFO ] [MAIN ] BG: Starting full primary refresh. [2025-10-20T17:31:29.056117] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-01. [2025-10-20T17:31:29.122915] [INFO ] [REMOTE ] BG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:29.130563] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:31:29.176399] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:31:29.851342] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-05. [2025-10-20T17:31:29.976009] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:31:30.365638] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T17:31:30.651421] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-02. [2025-10-20T17:31:31.425004] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T17:31:31.451386] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-07. [2025-10-20T17:31:32.251374] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-06. [2025-10-20T17:31:33.051377] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-01. [2025-10-20T17:31:33.238750] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T17:31:33.240863] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:31:33.851377] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-08. [2025-10-20T17:31:34.402359] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T17:31:34.651408] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-07. [2025-10-20T17:31:35.439300] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T17:31:35.451352] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-08. [2025-10-20T17:31:35.757694] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:31:35.979352] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T17:31:36.251387] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-02. [2025-10-20T17:31:37.051439] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-03. [2025-10-20T17:31:37.649616] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T17:31:37.775895] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:31:37.851423] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-03. [2025-10-20T17:31:38.651393] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-09. [2025-10-20T17:31:39.345602] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T17:31:39.451399] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-10. [2025-10-20T17:31:39.600776] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:31:40.179955] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T17:31:40.182217] [INFO ] [REMOTE ] BFS: Full fetch complete (13095 records). [2025-10-20T17:31:40.182295] [DEBUG] [VALIDATOR ] BFS: Validating primary data. [2025-10-20T17:31:40.256510] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-04. [2025-10-20T17:31:40.551290] [DEBUG] [VALIDATOR ] BFS: All expected trading days are present. [2025-10-20T17:31:40.551368] [DEBUG] [VALIDATOR ] BFS: Primary data validation passed. [2025-10-20T17:31:40.551692] [INFO ] [MAIN ] BFS: Fetched primary data is valid. [2025-10-20T17:31:40.551752] [DEBUG] [LOCAL ] BFS: Saving primary data. [2025-10-20T17:31:40.552243] [DEBUG] [LOCAL ] BFS: Trimmed 13095 records to 9614 (keeping 200 days of data) [2025-10-20T17:31:40.609623] [DEBUG] [LOCAL ] BFS: Primary data saved (9614 records). [2025-10-20T17:31:40.609788] [INFO ] [MAIN ] BFS: Full history saved. [2025-10-20T17:31:40.609861] [INFO ] [MAIN ] Update Progress: 256/2367 symbols completed. BFS took 22.7s. Estimated remaining: 4h 20m 36s. Rate: 8.1 symbols/min [2025-10-20T17:31:40.775874] [INFO ] [MAIN ] BGB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:40.794801] [DEBUG] [LOCAL ] BGB: Loaded 18257 existing records from primary storage. [2025-10-20T17:31:40.795056] [INFO ] [MAIN ] BGB: Loaded 18257 existing records. [2025-10-20T17:31:40.968994] [WARN ] [MAIN ] BGB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:31:40.969330] [INFO ] [MAIN ] BGB: Wiping existing primary data before full refresh. [2025-10-20T17:31:40.969402] [INFO ] [LOCAL ] BGB: Starting deletion of primary data. [2025-10-20T17:31:40.969565] [DEBUG] [LOCAL ] BGB: Deleted primary data file. [2025-10-20T17:31:40.969604] [INFO ] [MAIN ] BGB: Starting full primary refresh. [2025-10-20T17:31:41.051964] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-09. [2025-10-20T17:31:41.260925] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T17:31:41.352153] [INFO ] [REMOTE ] BGB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:41.851352] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-05. [2025-10-20T17:31:42.281176] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T17:31:42.651382] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-04. [2025-10-20T17:31:42.818827] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:31:43.451375] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-10. [2025-10-20T17:31:43.979258] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:31:44.251381] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-05. [2025-10-20T17:31:45.051392] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-06. [2025-10-20T17:31:45.685802] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-10 (2.23 seconds). [2025-10-20T17:31:45.693828] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:31:45.697531] [INFO ] [REMOTE ] BFLY: Full fetch complete (75525 records). [2025-10-20T17:31:45.697960] [DEBUG] [VALIDATOR ] BFLY: Validating primary data. [2025-10-20T17:31:45.857489] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-07. [2025-10-20T17:31:46.095988] [DEBUG] [VALIDATOR ] BFLY: All expected trading days are present. [2025-10-20T17:31:46.096089] [DEBUG] [VALIDATOR ] BFLY: Primary data validation passed. [2025-10-20T17:31:46.096861] [INFO ] [MAIN ] BFLY: Fetched primary data is valid. [2025-10-20T17:31:46.096921] [DEBUG] [LOCAL ] BFLY: Saving primary data. [2025-10-20T17:31:46.099174] [DEBUG] [LOCAL ] BFLY: Trimmed 75525 records to 51857 (keeping 200 days of data) [2025-10-20T17:31:46.339928] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-05 (2.09 seconds). [2025-10-20T17:31:46.425386] [DEBUG] [LOCAL ] BFLY: Primary data saved (51857 records). [2025-10-20T17:31:46.425535] [INFO ] [MAIN ] BFLY: Full history saved. [2025-10-20T17:31:46.425636] [INFO ] [MAIN ] Update Progress: 257/2367 symbols completed. BFLY took 44.0s. Estimated remaining: 4h 20m 16s. Rate: 8.1 symbols/min [2025-10-20T17:31:46.599196] [INFO ] [MAIN ] BGH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:46.599457] [DEBUG] [LOCAL ] BGH: No existing data file found in primary storage. [2025-10-20T17:31:46.599508] [WARN ] [MAIN ] BGH: No existing data found, performing full refresh instead. [2025-10-20T17:31:46.599660] [INFO ] [MAIN ] BGH: Wiping existing primary data before full refresh. [2025-10-20T17:31:46.599717] [INFO ] [LOCAL ] BGH: Starting deletion of primary data. [2025-10-20T17:31:46.599763] [DEBUG] [LOCAL ] BGH: No primary data file found to delete. [2025-10-20T17:31:46.599796] [INFO ] [MAIN ] BGH: Starting full primary refresh. [2025-10-20T17:31:46.655678] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-06. [2025-10-20T17:31:46.710944] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T17:31:46.968078] [INFO ] [REMOTE ] BGH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:47.451346] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-01. [2025-10-20T17:31:48.225260] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T17:31:48.251343] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-01. [2025-10-20T17:31:48.639953] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:31:49.051412] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-02. [2025-10-20T17:31:49.104398] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T17:31:49.835230] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T17:31:49.851335] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-02. [2025-10-20T17:31:50.416227] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T17:31:50.651383] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-07. [2025-10-20T17:31:51.451387] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-03. [2025-10-20T17:31:52.150778] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T17:31:52.188147] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:31:52.251398] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-04. [2025-10-20T17:31:53.041722] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T17:31:53.051360] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-05. [2025-10-20T17:31:53.851393] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-03. [2025-10-20T17:31:53.991586] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T17:31:54.651373] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-06. [2025-10-20T17:31:54.852032] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T17:31:55.411392] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T17:31:55.451344] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-08. [2025-10-20T17:31:56.251422] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-07. [2025-10-20T17:31:57.057744] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-04. [2025-10-20T17:31:57.060251] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T17:31:57.101705] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:31:57.851417] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-08. [2025-10-20T17:31:58.232186] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T17:31:58.651345] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-08. [2025-10-20T17:31:58.728278] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T17:31:59.451345] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-09. [2025-10-20T17:31:59.766333] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:32:00.251423] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-09. [2025-10-20T17:32:00.897269] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T17:32:01.051436] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-05. [2025-10-20T17:32:01.218785] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T17:32:01.851379] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-10. [2025-10-20T17:32:02.004845] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T17:32:02.651348] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-06. [2025-10-20T17:32:02.816933] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T17:32:02.819097] [INFO ] [REMOTE ] BFZ: Full fetch complete (17496 records). [2025-10-20T17:32:02.819171] [DEBUG] [VALIDATOR ] BFZ: Validating primary data. [2025-10-20T17:32:03.184775] [DEBUG] [VALIDATOR ] BFZ: All expected trading days are present. [2025-10-20T17:32:03.184876] [DEBUG] [VALIDATOR ] BFZ: Primary data validation passed. [2025-10-20T17:32:03.185193] [INFO ] [MAIN ] BFZ: Fetched primary data is valid. [2025-10-20T17:32:03.185231] [DEBUG] [LOCAL ] BFZ: Saving primary data. [2025-10-20T17:32:03.185805] [DEBUG] [LOCAL ] BFZ: Trimmed 17496 records to 12050 (keeping 200 days of data) [2025-10-20T17:32:03.261289] [DEBUG] [LOCAL ] BFZ: Primary data saved (12050 records). [2025-10-20T17:32:03.261529] [INFO ] [MAIN ] BFZ: Full history saved. [2025-10-20T17:32:03.261626] [INFO ] [MAIN ] Update Progress: 258/2367 symbols completed. BFZ took 39.3s. Estimated remaining: 4h 21m 26s. Rate: 8.1 symbols/min [2025-10-20T17:32:03.431903] [INFO ] [MAIN ] BGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:03.448478] [DEBUG] [LOCAL ] BGR: Loaded 13773 existing records from primary storage. [2025-10-20T17:32:03.448759] [INFO ] [MAIN ] BGR: Loaded 13773 existing records. [2025-10-20T17:32:03.460738] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-10. [2025-10-20T17:32:03.644459] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T17:32:03.663872] [WARN ] [MAIN ] BGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:03.667513] [INFO ] [MAIN ] BGR: Wiping existing primary data before full refresh. [2025-10-20T17:32:03.667882] [INFO ] [LOCAL ] BGR: Starting deletion of primary data. [2025-10-20T17:32:03.668121] [DEBUG] [LOCAL ] BGR: Deleted primary data file. [2025-10-20T17:32:03.668178] [INFO ] [MAIN ] BGR: Starting full primary refresh. [2025-10-20T17:32:04.027216] [INFO ] [REMOTE ] BGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:04.251392] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-01. [2025-10-20T17:32:05.051359] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-09. [2025-10-20T17:32:05.113988] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T17:32:05.138475] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T17:32:05.146282] [INFO ] [REMOTE ] BG: Full fetch complete (73050 records). [2025-10-20T17:32:05.146422] [DEBUG] [VALIDATOR ] BG: Validating primary data. [2025-10-20T17:32:05.566230] [DEBUG] [VALIDATOR ] BG: All expected trading days are present. [2025-10-20T17:32:05.566335] [DEBUG] [VALIDATOR ] BG: Primary data validation passed. [2025-10-20T17:32:05.567104] [INFO ] [MAIN ] BG: Fetched primary data is valid. [2025-10-20T17:32:05.567167] [DEBUG] [LOCAL ] BG: Saving primary data. [2025-10-20T17:32:05.568867] [DEBUG] [LOCAL ] BG: Trimmed 73050 records to 50585 (keeping 200 days of data) [2025-10-20T17:32:05.867128] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-07. [2025-10-20T17:32:05.905875] [DEBUG] [LOCAL ] BG: Primary data saved (50585 records). [2025-10-20T17:32:05.906243] [INFO ] [MAIN ] BG: Full history saved. [2025-10-20T17:32:05.906339] [INFO ] [MAIN ] Update Progress: 259/2367 symbols completed. BG took 37.6s. Estimated remaining: 4h 20m 39s. Rate: 8.1 symbols/min [2025-10-20T17:32:06.093630] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:32:06.097105] [INFO ] [MAIN ] BGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:06.149612] [DEBUG] [LOCAL ] BGS: Loaded 51299 existing records from primary storage. [2025-10-20T17:32:06.149972] [INFO ] [MAIN ] BGS: Loaded 51299 existing records. [2025-10-20T17:32:06.334741] [WARN ] [MAIN ] BGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:06.334988] [INFO ] [MAIN ] BGS: Wiping existing primary data before full refresh. [2025-10-20T17:32:06.335058] [INFO ] [LOCAL ] BGS: Starting deletion of primary data. [2025-10-20T17:32:06.335266] [DEBUG] [LOCAL ] BGS: Deleted primary data file. [2025-10-20T17:32:06.335323] [INFO ] [MAIN ] BGS: Starting full primary refresh. [2025-10-20T17:32:06.665131] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-10. [2025-10-20T17:32:06.711286] [INFO ] [REMOTE ] BGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:06.991481] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T17:32:07.451396] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-01. [2025-10-20T17:32:07.819798] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:32:07.823024] [INFO ] [REMOTE ] BGH: Full fetch complete (22894 records). [2025-10-20T17:32:07.823104] [DEBUG] [VALIDATOR ] BGH: Validating primary data. [2025-10-20T17:32:08.221888] [DEBUG] [VALIDATOR ] BGH: All expected trading days are present. [2025-10-20T17:32:08.221993] [DEBUG] [VALIDATOR ] BGH: Primary data validation passed. [2025-10-20T17:32:08.222586] [INFO ] [MAIN ] BGH: Fetched primary data is valid. [2025-10-20T17:32:08.222638] [DEBUG] [LOCAL ] BGH: Saving primary data. [2025-10-20T17:32:08.223382] [DEBUG] [LOCAL ] BGH: Trimmed 22894 records to 17201 (keeping 200 days of data) [2025-10-20T17:32:08.268004] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-08. [2025-10-20T17:32:08.334343] [DEBUG] [LOCAL ] BGH: Primary data saved (17201 records). [2025-10-20T17:32:08.334541] [INFO ] [MAIN ] BGH: Full history saved. [2025-10-20T17:32:08.334629] [INFO ] [MAIN ] Update Progress: 260/2367 symbols completed. BGH took 21.9s. Estimated remaining: 4h 19m 51s. Rate: 8.1 symbols/min [2025-10-20T17:32:08.514707] [INFO ] [MAIN ] BGSF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:08.524709] [DEBUG] [LOCAL ] BGSF: Loaded 8117 existing records from primary storage. [2025-10-20T17:32:08.524992] [INFO ] [MAIN ] BGSF: Loaded 8117 existing records. [2025-10-20T17:32:08.718042] [WARN ] [MAIN ] BGSF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:08.718352] [INFO ] [MAIN ] BGSF: Wiping existing primary data before full refresh. [2025-10-20T17:32:08.718419] [INFO ] [LOCAL ] BGSF: Starting deletion of primary data. [2025-10-20T17:32:08.718605] [DEBUG] [LOCAL ] BGSF: Deleted primary data file. [2025-10-20T17:32:08.718694] [INFO ] [MAIN ] BGSF: Starting full primary refresh. [2025-10-20T17:32:09.062108] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-02. [2025-10-20T17:32:09.107557] [INFO ] [REMOTE ] BGSF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:09.229446] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:32:09.378751] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:32:09.851391] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-09. [2025-10-20T17:32:10.134166] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T17:32:10.651348] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-03. [2025-10-20T17:32:10.807503] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T17:32:11.451443] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-01. [2025-10-20T17:32:11.474868] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T17:32:12.117511] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T17:32:12.251431] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-02. [2025-10-20T17:32:12.694090] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:32:13.051447] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-04. [2025-10-20T17:32:13.851388] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-10. [2025-10-20T17:32:14.089412] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T17:32:14.651357] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-02. [2025-10-20T17:32:15.024332] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T17:32:15.028473] [INFO ] [REMOTE ] BGB: Full fetch complete (26786 records). [2025-10-20T17:32:15.028614] [DEBUG] [VALIDATOR ] BGB: Validating primary data. [2025-10-20T17:32:15.395431] [DEBUG] [VALIDATOR ] BGB: All expected trading days are present. [2025-10-20T17:32:15.395505] [DEBUG] [VALIDATOR ] BGB: Primary data validation passed. [2025-10-20T17:32:15.395894] [INFO ] [MAIN ] BGB: Fetched primary data is valid. [2025-10-20T17:32:15.395954] [DEBUG] [LOCAL ] BGB: Saving primary data. [2025-10-20T17:32:15.396582] [DEBUG] [LOCAL ] BGB: Trimmed 26786 records to 17876 (keeping 200 days of data) [2025-10-20T17:32:15.472435] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-05. [2025-10-20T17:32:15.504758] [DEBUG] [LOCAL ] BGB: Primary data saved (17876 records). [2025-10-20T17:32:15.504898] [INFO ] [MAIN ] BGB: Full history saved. [2025-10-20T17:32:15.504977] [INFO ] [MAIN ] Update Progress: 261/2367 symbols completed. BGB took 34.9s. Estimated remaining: 4h 19m 42s. Rate: 8.1 symbols/min [2025-10-20T17:32:15.683093] [INFO ] [MAIN ] BGT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:15.699332] [DEBUG] [LOCAL ] BGT: Loaded 16635 existing records from primary storage. [2025-10-20T17:32:15.699555] [INFO ] [MAIN ] BGT: Loaded 16635 existing records. [2025-10-20T17:32:15.883152] [WARN ] [MAIN ] BGT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:15.883554] [INFO ] [MAIN ] BGT: Wiping existing primary data before full refresh. [2025-10-20T17:32:15.883721] [INFO ] [LOCAL ] BGT: Starting deletion of primary data. [2025-10-20T17:32:15.884070] [DEBUG] [LOCAL ] BGT: Deleted primary data file. [2025-10-20T17:32:15.884255] [INFO ] [MAIN ] BGT: Starting full primary refresh. [2025-10-20T17:32:16.217392] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T17:32:16.278195] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-03. [2025-10-20T17:32:16.303303] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T17:32:16.325258] [INFO ] [REMOTE ] BGT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:16.946312] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T17:32:17.051364] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-06. [2025-10-20T17:32:17.851380] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-01. [2025-10-20T17:32:18.006695] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T17:32:18.651356] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-07. [2025-10-20T17:32:19.035837] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:32:19.233090] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T17:32:19.451413] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-02. [2025-10-20T17:32:20.251431] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-08. [2025-10-20T17:32:20.357705] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T17:32:21.051398] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-04. [2025-10-20T17:32:21.141751] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T17:32:21.526228] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-10-20T17:32:21.851384] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-03. [2025-10-20T17:32:22.651353] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-09. [2025-10-20T17:32:22.722030] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T17:32:23.451371] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-04. [2025-10-20T17:32:23.496543] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T17:32:24.103634] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:32:24.251379] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-03. [2025-10-20T17:32:25.051440] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-05. [2025-10-20T17:32:25.851416] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-05. [2025-10-20T17:32:25.984006] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T17:32:26.036993] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T17:32:26.362629] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T17:32:26.651397] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-04. [2025-10-20T17:32:27.451377] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-06. [2025-10-20T17:32:28.251431] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-06. [2025-10-20T17:32:28.422448] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T17:32:28.465618] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:32:29.051383] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-07. [2025-10-20T17:32:29.059181] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T17:32:29.851365] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-07. [2025-10-20T17:32:30.017035] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T17:32:30.600309] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-10-20T17:32:30.651338] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-08. [2025-10-20T17:32:31.184992] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T17:32:31.451362] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-09. [2025-10-20T17:32:32.251433] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-10. [2025-10-20T17:32:32.424563] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T17:32:32.917235] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:32:32.919507] [INFO ] [REMOTE ] BGR: Full fetch complete (19891 records). [2025-10-20T17:32:32.919580] [DEBUG] [VALIDATOR ] BGR: Validating primary data. [2025-10-20T17:32:33.056465] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-10. [2025-10-20T17:32:33.283499] [DEBUG] [VALIDATOR ] BGR: All expected trading days are present. [2025-10-20T17:32:33.283575] [DEBUG] [VALIDATOR ] BGR: Primary data validation passed. [2025-10-20T17:32:33.283912] [INFO ] [MAIN ] BGR: Fetched primary data is valid. [2025-10-20T17:32:33.283974] [DEBUG] [LOCAL ] BGR: Saving primary data. [2025-10-20T17:32:33.284506] [DEBUG] [LOCAL ] BGR: Trimmed 19891 records to 13505 (keeping 200 days of data) [2025-10-20T17:32:33.365176] [DEBUG] [LOCAL ] BGR: Primary data saved (13505 records). [2025-10-20T17:32:33.365311] [INFO ] [MAIN ] BGR: Full history saved. [2025-10-20T17:32:33.365386] [INFO ] [MAIN ] Update Progress: 262/2367 symbols completed. BGR took 30.1s. Estimated remaining: 4h 20m 59s. Rate: 8.1 symbols/min [2025-10-20T17:32:33.534790] [INFO ] [MAIN ] BGX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:33.548112] [DEBUG] [LOCAL ] BGX: Loaded 11350 existing records from primary storage. [2025-10-20T17:32:33.548399] [INFO ] [MAIN ] BGX: Loaded 11350 existing records. [2025-10-20T17:32:33.721768] [WARN ] [MAIN ] BGX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:33.721975] [INFO ] [MAIN ] BGX: Wiping existing primary data before full refresh. [2025-10-20T17:32:33.722022] [INFO ] [LOCAL ] BGX: Starting deletion of primary data. [2025-10-20T17:32:33.722177] [DEBUG] [LOCAL ] BGX: Deleted primary data file. [2025-10-20T17:32:33.722215] [INFO ] [MAIN ] BGX: Starting full primary refresh. [2025-10-20T17:32:33.860022] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-08. [2025-10-20T17:32:34.079957] [INFO ] [REMOTE ] BGX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:34.357135] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T17:32:34.359787] [INFO ] [REMOTE ] BGSF: Full fetch complete (9556 records). [2025-10-20T17:32:34.359858] [DEBUG] [VALIDATOR ] BGSF: Validating primary data. [2025-10-20T17:32:34.658896] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-05. [2025-10-20T17:32:34.659847] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T17:32:34.745444] [DEBUG] [VALIDATOR ] BGSF: All expected trading days are present. [2025-10-20T17:32:34.745544] [DEBUG] [VALIDATOR ] BGSF: Primary data validation passed. [2025-10-20T17:32:34.745841] [INFO ] [MAIN ] BGSF: Fetched primary data is valid. [2025-10-20T17:32:34.745902] [DEBUG] [LOCAL ] BGSF: Saving primary data. [2025-10-20T17:32:34.746362] [DEBUG] [LOCAL ] BGSF: Trimmed 9556 records to 8128 (keeping 200 days of data) [2025-10-20T17:32:34.792017] [DEBUG] [LOCAL ] BGSF: Primary data saved (8128 records). [2025-10-20T17:32:34.792138] [INFO ] [MAIN ] BGSF: Full history saved. [2025-10-20T17:32:34.792214] [INFO ] [MAIN ] Update Progress: 263/2367 symbols completed. BGSF took 26.5s. Estimated remaining: 4h 20m 3s. Rate: 8.1 symbols/min [2025-10-20T17:32:34.963353] [INFO ] [MAIN ] BGY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:34.963524] [DEBUG] [LOCAL ] BGY: No existing data file found in primary storage. [2025-10-20T17:32:34.963562] [WARN ] [MAIN ] BGY: No existing data found, performing full refresh instead. [2025-10-20T17:32:34.963699] [INFO ] [MAIN ] BGY: Wiping existing primary data before full refresh. [2025-10-20T17:32:34.963753] [INFO ] [LOCAL ] BGY: Starting deletion of primary data. [2025-10-20T17:32:34.963800] [DEBUG] [LOCAL ] BGY: No primary data file found to delete. [2025-10-20T17:32:34.963833] [INFO ] [MAIN ] BGY: Starting full primary refresh. [2025-10-20T17:32:35.321982] [INFO ] [REMOTE ] BGY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:35.451352] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-01. [2025-10-20T17:32:36.251354] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-01. [2025-10-20T17:32:36.424887] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:32:36.665788] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:32:37.051413] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-02. [2025-10-20T17:32:37.140608] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T17:32:37.851384] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-09. [2025-10-20T17:32:38.050739] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T17:32:38.531129] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-10-20T17:32:38.651396] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-02. [2025-10-20T17:32:39.260755] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:32:39.451369] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-03. [2025-10-20T17:32:40.148409] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T17:32:40.251357] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-03. [2025-10-20T17:32:41.036869] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T17:32:41.051320] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-04. [2025-10-20T17:32:41.851348] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-06. [2025-10-20T17:32:42.165229] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T17:32:42.651369] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-04. [2025-10-20T17:32:43.153773] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T17:32:43.451381] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-05. [2025-10-20T17:32:43.530843] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T17:32:44.251447] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-10. [2025-10-20T17:32:44.465289] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T17:32:45.051344] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-06. [2025-10-20T17:32:45.236114] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T17:32:45.238786] [INFO ] [REMOTE ] BGT: Full fetch complete (24259 records). [2025-10-20T17:32:45.238870] [DEBUG] [VALIDATOR ] BGT: Validating primary data. [2025-10-20T17:32:45.612718] [DEBUG] [VALIDATOR ] BGT: All expected trading days are present. [2025-10-20T17:32:45.612800] [DEBUG] [VALIDATOR ] BGT: Primary data validation passed. [2025-10-20T17:32:45.613175] [INFO ] [MAIN ] BGT: Fetched primary data is valid. [2025-10-20T17:32:45.613214] [DEBUG] [LOCAL ] BGT: Saving primary data. [2025-10-20T17:32:45.613902] [DEBUG] [LOCAL ] BGT: Trimmed 24259 records to 16314 (keeping 200 days of data) [2025-10-20T17:32:45.714226] [DEBUG] [LOCAL ] BGT: Primary data saved (16314 records). [2025-10-20T17:32:45.714367] [INFO ] [MAIN ] BGT: Full history saved. [2025-10-20T17:32:45.714443] [INFO ] [MAIN ] Update Progress: 264/2367 symbols completed. BGT took 30.2s. Estimated remaining: 4h 20m 24s. Rate: 8.1 symbols/min [2025-10-20T17:32:45.860328] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-05. [2025-10-20T17:32:45.891607] [INFO ] [MAIN ] BH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:45.914833] [DEBUG] [LOCAL ] BH: Loaded 16442 existing records from primary storage. [2025-10-20T17:32:45.915104] [INFO ] [MAIN ] BH: Loaded 16442 existing records. [2025-10-20T17:32:46.145573] [WARN ] [MAIN ] BH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:46.145897] [INFO ] [MAIN ] BH: Wiping existing primary data before full refresh. [2025-10-20T17:32:46.145978] [INFO ] [LOCAL ] BH: Starting deletion of primary data. [2025-10-20T17:32:46.146185] [DEBUG] [LOCAL ] BH: Deleted primary data file. [2025-10-20T17:32:46.146230] [INFO ] [MAIN ] BH: Starting full primary refresh. [2025-10-20T17:32:46.337396] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:32:46.655303] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-07. [2025-10-20T17:32:46.741943] [INFO ] [REMOTE ] BH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:46.743724] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T17:32:47.451402] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-07. [2025-10-20T17:32:48.251376] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-01. [2025-10-20T17:32:48.710107] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-10-20T17:32:48.738231] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T17:32:48.838753] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T17:32:49.051394] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-02. [2025-10-20T17:32:49.460796] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T17:32:49.851422] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-03. [2025-10-20T17:32:50.618559] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T17:32:50.651395] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-04. [2025-10-20T17:32:51.392525] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T17:32:51.451413] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-06. [2025-10-20T17:32:52.251434] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-08. [2025-10-20T17:32:52.424976] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T17:32:53.051431] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-08. [2025-10-20T17:32:53.258896] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:32:53.851428] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-09. [2025-10-20T17:32:54.651459] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-07. [2025-10-20T17:32:54.873291] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:32:54.877455] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T17:32:55.350137] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T17:32:55.451386] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-09. [2025-10-20T17:32:56.251359] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-08. [2025-10-20T17:32:57.009488] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T17:32:57.051324] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-09. [2025-10-20T17:32:57.312421] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T17:32:57.642061] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T17:32:57.851402] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-05. [2025-10-20T17:32:58.651367] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-10. [2025-10-20T17:32:58.799461] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T17:32:59.451422] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-10. [2025-10-20T17:32:59.937593] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T17:32:59.941168] [INFO ] [REMOTE ] BGY: Full fetch complete (33055 records). [2025-10-20T17:32:59.941247] [DEBUG] [VALIDATOR ] BGY: Validating primary data. [2025-10-20T17:33:00.256594] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-06. [2025-10-20T17:33:00.331715] [DEBUG] [VALIDATOR ] BGY: All expected trading days are present. [2025-10-20T17:33:00.331975] [DEBUG] [VALIDATOR ] BGY: Primary data validation passed. [2025-10-20T17:33:00.332679] [INFO ] [MAIN ] BGY: Fetched primary data is valid. [2025-10-20T17:33:00.332761] [DEBUG] [LOCAL ] BGY: Saving primary data. [2025-10-20T17:33:00.334198] [DEBUG] [LOCAL ] BGY: Trimmed 33055 records to 21632 (keeping 200 days of data) [2025-10-20T17:33:00.477600] [DEBUG] [LOCAL ] BGY: Primary data saved (21632 records). [2025-10-20T17:33:00.477806] [INFO ] [MAIN ] BGY: Full history saved. [2025-10-20T17:33:00.477906] [INFO ] [MAIN ] Update Progress: 265/2367 symbols completed. BGY took 25.7s. Estimated remaining: 4h 21m 14s. Rate: 8.0 symbols/min [2025-10-20T17:33:00.679029] [INFO ] [MAIN ] BHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:00.720750] [DEBUG] [LOCAL ] BHC: Loaded 47402 existing records from primary storage. [2025-10-20T17:33:00.721137] [INFO ] [MAIN ] BHC: Loaded 47402 existing records. [2025-10-20T17:33:00.922306] [WARN ] [MAIN ] BHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:00.922571] [INFO ] [MAIN ] BHC: Wiping existing primary data before full refresh. [2025-10-20T17:33:00.922639] [INFO ] [LOCAL ] BHC: Starting deletion of primary data. [2025-10-20T17:33:00.923155] [DEBUG] [LOCAL ] BHC: Deleted primary data file. [2025-10-20T17:33:00.923227] [INFO ] [MAIN ] BHC: Starting full primary refresh. [2025-10-20T17:33:01.056617] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-10. [2025-10-20T17:33:01.327770] [INFO ] [REMOTE ] BHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:01.467883] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:33:01.681276] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-10 (2.23 seconds). [2025-10-20T17:33:01.689433] [INFO ] [REMOTE ] BGS: Full fetch complete (72666 records). [2025-10-20T17:33:01.689590] [DEBUG] [VALIDATOR ] BGS: Validating primary data. [2025-10-20T17:33:01.856466] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-01. [2025-10-20T17:33:02.132433] [DEBUG] [VALIDATOR ] BGS: All expected trading days are present. [2025-10-20T17:33:02.133184] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T17:33:02.132967] [DEBUG] [VALIDATOR ] BGS: Primary data validation passed. [2025-10-20T17:33:02.137152] [INFO ] [MAIN ] BGS: Fetched primary data is valid. [2025-10-20T17:33:02.137271] [DEBUG] [LOCAL ] BGS: Saving primary data. [2025-10-20T17:33:02.137540] [INFO ] [REMOTE ] BGX: Full fetch complete (16417 records). [2025-10-20T17:33:02.137957] [DEBUG] [VALIDATOR ] BGX: Validating primary data. [2025-10-20T17:33:02.139255] [FATAL] [VALIDATOR ] BGX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:33:02.139414] [FATAL] [MAIN ] BGX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:33:02.139525] [WARN ] [MAIN ] BGX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:33:02.139755] [INFO ] [LOCAL ] BGX: Starting deletion of primary data. [2025-10-20T17:33:02.139921] [DEBUG] [LOCAL ] BGX: No primary data file found to delete. [2025-10-20T17:33:02.140034] [INFO ] [LOCAL ] BGX: Starting deletion of derivative data. [2025-10-20T17:33:02.140421] [DEBUG] [LOCAL ] BGX: Deleted derivative data file. [2025-10-20T17:33:02.140540] [INFO ] [MAIN ] Update Progress: 266/2367 symbols completed. BGX took 28.8s. Estimated remaining: 4h 20m 21s. Rate: 8.1 symbols/min [2025-10-20T17:33:02.173775] [DEBUG] [LOCAL ] BGS: Trimmed 72666 records to 50571 (keeping 200 days of data) [2025-10-20T17:33:02.514226] [INFO ] [MAIN ] BHE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:02.657051] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-07. [2025-10-20T17:33:02.671917] [DEBUG] [LOCAL ] BHE: Loaded 24249 existing records from primary storage. [2025-10-20T17:33:02.677129] [INFO ] [MAIN ] BHE: Loaded 24249 existing records. [2025-10-20T17:33:02.700542] [DEBUG] [LOCAL ] BGS: Primary data saved (50571 records). [2025-10-20T17:33:02.748944] [INFO ] [MAIN ] BGS: Full history saved. [2025-10-20T17:33:02.759983] [INFO ] [MAIN ] Update Progress: 267/2367 symbols completed. BGS took 56.9s. Estimated remaining: 4h 19m 20s. Rate: 8.1 symbols/min [2025-10-20T17:33:03.013116] [WARN ] [MAIN ] BHE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:03.036601] [INFO ] [MAIN ] BHE: Wiping existing primary data before full refresh. [2025-10-20T17:33:03.037507] [INFO ] [LOCAL ] BHE: Starting deletion of primary data. [2025-10-20T17:33:03.047856] [DEBUG] [LOCAL ] BHE: Deleted primary data file. [2025-10-20T17:33:03.056896] [INFO ] [MAIN ] BHK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:03.058811] [INFO ] [MAIN ] BHE: Starting full primary refresh. [2025-10-20T17:33:03.333088] [DEBUG] [LOCAL ] BHK: Loaded 23994 existing records from primary storage. [2025-10-20T17:33:03.348634] [INFO ] [MAIN ] BHK: Loaded 23994 existing records. [2025-10-20T17:33:03.632568] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T17:33:03.636731] [INFO ] [REMOTE ] BHE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:03.657975] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-01. [2025-10-20T17:33:03.738179] [WARN ] [MAIN ] BHK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:03.738739] [INFO ] [MAIN ] BHK: Wiping existing primary data before full refresh. [2025-10-20T17:33:03.738832] [INFO ] [LOCAL ] BHK: Starting deletion of primary data. [2025-10-20T17:33:03.739309] [DEBUG] [LOCAL ] BHK: Deleted primary data file. [2025-10-20T17:33:03.739375] [INFO ] [MAIN ] BHK: Starting full primary refresh. [2025-10-20T17:33:03.885319] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T17:33:04.146457] [INFO ] [REMOTE ] BHK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:04.251425] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-01. [2025-10-20T17:33:04.774565] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:33:05.051360] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-02. [2025-10-20T17:33:05.227015] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T17:33:05.851364] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-08. [2025-10-20T17:33:06.627104] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:33:06.651334] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-03. [2025-10-20T17:33:06.689955] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T17:33:07.451371] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-09. [2025-10-20T17:33:08.251443] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-02. [2025-10-20T17:33:08.343025] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T17:33:08.395638] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T17:33:08.875862] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T17:33:09.051359] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-04. [2025-10-20T17:33:09.851370] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-10. [2025-10-20T17:33:10.651355] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-02. [2025-10-20T17:33:10.840846] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:33:10.857773] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T17:33:10.860246] [INFO ] [REMOTE ] BH: Full fetch complete (19462 records). [2025-10-20T17:33:10.860328] [DEBUG] [VALIDATOR ] BH: Validating primary data. [2025-10-20T17:33:10.860833] [FATAL] [VALIDATOR ] BH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:33:10.860936] [FATAL] [MAIN ] BH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:33:10.861003] [WARN ] [MAIN ] BH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:33:10.861075] [INFO ] [LOCAL ] BH: Starting deletion of primary data. [2025-10-20T17:33:10.861137] [DEBUG] [LOCAL ] BH: No primary data file found to delete. [2025-10-20T17:33:10.861173] [INFO ] [LOCAL ] BH: Starting deletion of derivative data. [2025-10-20T17:33:10.861331] [DEBUG] [LOCAL ] BH: Deleted derivative data file. [2025-10-20T17:33:10.861419] [INFO ] [MAIN ] Update Progress: 268/2367 symbols completed. BH took 25.1s. Estimated remaining: 4h 19m 18s. Rate: 8.1 symbols/min [2025-10-20T17:33:11.035957] [INFO ] [MAIN ] BHP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:11.082739] [DEBUG] [LOCAL ] BHP: Loaded 54152 existing records from primary storage. [2025-10-20T17:33:11.083071] [INFO ] [MAIN ] BHP: Loaded 54152 existing records. [2025-10-20T17:33:11.265837] [WARN ] [MAIN ] BHP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:11.266113] [INFO ] [MAIN ] BHP: Wiping existing primary data before full refresh. [2025-10-20T17:33:11.266161] [INFO ] [LOCAL ] BHP: Starting deletion of primary data. [2025-10-20T17:33:11.266606] [DEBUG] [LOCAL ] BHP: Deleted primary data file. [2025-10-20T17:33:11.266698] [INFO ] [MAIN ] BHP: Starting full primary refresh. [2025-10-20T17:33:11.458700] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-05. [2025-10-20T17:33:11.643208] [INFO ] [REMOTE ] BHP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:11.904352] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:33:12.251410] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-01. [2025-10-20T17:33:12.981396] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T17:33:13.051340] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-06. [2025-10-20T17:33:13.851366] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-03. [2025-10-20T17:33:14.272662] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T17:33:14.598381] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:33:14.645911] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T17:33:14.651326] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-04. [2025-10-20T17:33:15.366191] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T17:33:15.451382] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-05. [2025-10-20T17:33:16.251343] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-03. [2025-10-20T17:33:16.356029] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T17:33:17.051378] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-02. [2025-10-20T17:33:17.734346] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T17:33:17.851391] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-04. [2025-10-20T17:33:18.499015] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T17:33:18.651415] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-06. [2025-10-20T17:33:19.211871] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T17:33:19.451426] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-07. [2025-10-20T17:33:19.809282] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T17:33:20.251379] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-03. [2025-10-20T17:33:20.839954] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:33:21.051355] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-07. [2025-10-20T17:33:21.851396] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-05. [2025-10-20T17:33:22.094984] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T17:33:22.278239] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T17:33:22.651352] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-08. [2025-10-20T17:33:23.003530] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T17:33:23.451354] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-04. [2025-10-20T17:33:23.656635] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:33:24.251387] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-08. [2025-10-20T17:33:25.051465] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-06. [2025-10-20T17:33:25.591147] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T17:33:25.652684] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:33:25.851428] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-09. [2025-10-20T17:33:26.276118] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T17:33:26.651467] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-07. [2025-10-20T17:33:27.451361] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-09. [2025-10-20T17:33:27.765421] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T17:33:27.771678] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:33:28.251361] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-05. [2025-10-20T17:33:28.725018] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T17:33:29.051396] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-08. [2025-10-20T17:33:29.851364] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-10. [2025-10-20T17:33:30.040354] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T17:33:30.372977] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T17:33:30.651372] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-09. [2025-10-20T17:33:31.207026] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T17:33:31.213833] [INFO ] [REMOTE ] BHC: Full fetch complete (67520 records). [2025-10-20T17:33:31.213984] [DEBUG] [VALIDATOR ] BHC: Validating primary data. [2025-10-20T17:33:31.461304] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-06. [2025-10-20T17:33:31.608727] [DEBUG] [VALIDATOR ] BHC: All expected trading days are present. [2025-10-20T17:33:31.608848] [DEBUG] [VALIDATOR ] BHC: Primary data validation passed. [2025-10-20T17:33:31.609579] [INFO ] [MAIN ] BHC: Fetched primary data is valid. [2025-10-20T17:33:31.609726] [DEBUG] [LOCAL ] BHC: Saving primary data. [2025-10-20T17:33:31.611097] [DEBUG] [LOCAL ] BHC: Trimmed 67520 records to 46695 (keeping 200 days of data) [2025-10-20T17:33:31.875693] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T17:33:31.903047] [DEBUG] [LOCAL ] BHC: Primary data saved (46695 records). [2025-10-20T17:33:31.903358] [INFO ] [MAIN ] BHC: Full history saved. [2025-10-20T17:33:31.903456] [INFO ] [MAIN ] Update Progress: 269/2367 symbols completed. BHC took 31.4s. Estimated remaining: 4h 20m 57s. Rate: 8.0 symbols/min [2025-10-20T17:33:32.070570] [INFO ] [MAIN ] BHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:32.090710] [DEBUG] [LOCAL ] BHR: Loaded 22426 existing records from primary storage. [2025-10-20T17:33:32.090985] [INFO ] [MAIN ] BHR: Loaded 22426 existing records. [2025-10-20T17:33:32.265034] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-10. [2025-10-20T17:33:32.269944] [WARN ] [MAIN ] BHR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:32.271321] [INFO ] [MAIN ] BHR: Wiping existing primary data before full refresh. [2025-10-20T17:33:32.271408] [INFO ] [LOCAL ] BHR: Starting deletion of primary data. [2025-10-20T17:33:32.271688] [DEBUG] [LOCAL ] BHR: Deleted primary data file. [2025-10-20T17:33:32.271747] [INFO ] [MAIN ] BHR: Starting full primary refresh. [2025-10-20T17:33:32.652010] [INFO ] [REMOTE ] BHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:33.051369] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-01. [2025-10-20T17:33:33.356359] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T17:33:33.571382] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:33:33.574890] [INFO ] [REMOTE ] BHE: Full fetch complete (36362 records). [2025-10-20T17:33:33.574970] [DEBUG] [VALIDATOR ] BHE: Validating primary data. [2025-10-20T17:33:33.855479] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-10. [2025-10-20T17:33:33.973389] [DEBUG] [VALIDATOR ] BHE: All expected trading days are present. [2025-10-20T17:33:33.973534] [DEBUG] [VALIDATOR ] BHE: Primary data validation passed. [2025-10-20T17:33:33.974084] [INFO ] [MAIN ] BHE: Fetched primary data is valid. [2025-10-20T17:33:33.974166] [DEBUG] [LOCAL ] BHE: Saving primary data. [2025-10-20T17:33:33.988305] [DEBUG] [LOCAL ] BHE: Trimmed 36362 records to 23853 (keeping 200 days of data) [2025-10-20T17:33:33.989878] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T17:33:34.134973] [DEBUG] [LOCAL ] BHE: Primary data saved (23853 records). [2025-10-20T17:33:34.135107] [INFO ] [MAIN ] BHE: Full history saved. [2025-10-20T17:33:34.135184] [INFO ] [MAIN ] Update Progress: 270/2367 symbols completed. BHE took 32.0s. Estimated remaining: 4h 20m 9s. Rate: 8.1 symbols/min [2025-10-20T17:33:34.302031] [INFO ] [MAIN ] BHV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:34.302246] [DEBUG] [LOCAL ] BHV: No existing data file found in primary storage. [2025-10-20T17:33:34.302295] [WARN ] [MAIN ] BHV: No existing data found, performing full refresh instead. [2025-10-20T17:33:34.302428] [INFO ] [MAIN ] BHV: Wiping existing primary data before full refresh. [2025-10-20T17:33:34.302472] [INFO ] [LOCAL ] BHV: Starting deletion of primary data. [2025-10-20T17:33:34.302514] [DEBUG] [LOCAL ] BHV: No primary data file found to delete. [2025-10-20T17:33:34.302550] [INFO ] [MAIN ] BHV: Starting full primary refresh. [2025-10-20T17:33:34.652145] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-02. [2025-10-20T17:33:34.652082] [INFO ] [REMOTE ] BHV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:34.975356] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T17:33:34.978806] [INFO ] [REMOTE ] BHK: Full fetch complete (30338 records). [2025-10-20T17:33:34.978877] [DEBUG] [VALIDATOR ] BHK: Validating primary data. [2025-10-20T17:33:34.979323] [FATAL] [VALIDATOR ] BHK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:33:34.979405] [FATAL] [MAIN ] BHK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:33:34.979447] [WARN ] [MAIN ] BHK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:33:34.979528] [INFO ] [LOCAL ] BHK: Starting deletion of primary data. [2025-10-20T17:33:34.979590] [DEBUG] [LOCAL ] BHK: No primary data file found to delete. [2025-10-20T17:33:34.979625] [INFO ] [LOCAL ] BHK: Starting deletion of derivative data. [2025-10-20T17:33:34.979792] [DEBUG] [LOCAL ] BHK: Deleted derivative data file. [2025-10-20T17:33:34.979868] [INFO ] [MAIN ] Update Progress: 271/2367 symbols completed. BHK took 32.2s. Estimated remaining: 4h 19m 10s. Rate: 8.1 symbols/min [2025-10-20T17:33:35.146283] [INFO ] [MAIN ] BHVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:35.192139] [DEBUG] [LOCAL ] BHVN: Loaded 51375 existing records from primary storage. [2025-10-20T17:33:35.192468] [INFO ] [MAIN ] BHVN: Loaded 51375 existing records. [2025-10-20T17:33:35.390893] [WARN ] [MAIN ] BHVN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:35.391130] [INFO ] [MAIN ] BHVN: Wiping existing primary data before full refresh. [2025-10-20T17:33:35.391183] [INFO ] [LOCAL ] BHVN: Starting deletion of primary data. [2025-10-20T17:33:35.391615] [DEBUG] [LOCAL ] BHVN: Deleted primary data file. [2025-10-20T17:33:35.391706] [INFO ] [MAIN ] BHVN: Starting full primary refresh. [2025-10-20T17:33:35.456940] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-07. [2025-10-20T17:33:35.794039] [INFO ] [REMOTE ] BHVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:35.806169] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T17:33:36.251400] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-01. [2025-10-20T17:33:37.051431] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-01. [2025-10-20T17:33:37.373760] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T17:33:37.388017] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-10-20T17:33:37.851406] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-08. [2025-10-20T17:33:37.904458] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:33:38.651384] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-03. [2025-10-20T17:33:39.451413] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-02. [2025-10-20T17:33:39.644260] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:33:39.747038] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T17:33:39.801588] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-10-20T17:33:40.251419] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-02. [2025-10-20T17:33:41.051372] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-04. [2025-10-20T17:33:41.609630] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T17:33:41.851392] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-03. [2025-10-20T17:33:42.143695] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T17:33:42.651429] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-03. [2025-10-20T17:33:43.056126] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T17:33:43.405840] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:33:43.451391] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-04. [2025-10-20T17:33:44.251482] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-05. [2025-10-20T17:33:45.051462] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-04. [2025-10-20T17:33:45.223155] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T17:33:45.251962] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T17:33:45.407708] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-10-20T17:33:45.851432] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-05. [2025-10-20T17:33:46.651443] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-09. [2025-10-20T17:33:47.451390] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-05. [2025-10-20T17:33:47.709588] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:33:48.142896] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T17:33:48.251362] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-06. [2025-10-20T17:33:48.429026] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:33:49.051390] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-10. [2025-10-20T17:33:49.851466] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-06. [2025-10-20T17:33:49.923697] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:33:50.651382] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-06. [2025-10-20T17:33:50.891426] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:33:50.908106] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-06 (0.26 seconds). [2025-10-20T17:33:51.045673] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T17:33:51.053096] [INFO ] [REMOTE ] BHP: Full fetch complete (77052 records). [2025-10-20T17:33:51.053234] [DEBUG] [VALIDATOR ] BHP: Validating primary data. [2025-10-20T17:33:51.053975] [FATAL] [VALIDATOR ] BHP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:33:51.054078] [FATAL] [MAIN ] BHP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:33:51.054149] [WARN ] [MAIN ] BHP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:33:51.054244] [INFO ] [LOCAL ] BHP: Starting deletion of primary data. [2025-10-20T17:33:51.054322] [DEBUG] [LOCAL ] BHP: No primary data file found to delete. [2025-10-20T17:33:51.054358] [INFO ] [LOCAL ] BHP: Starting deletion of derivative data. [2025-10-20T17:33:51.054481] [DEBUG] [LOCAL ] BHP: Deleted derivative data file. [2025-10-20T17:33:51.054767] [INFO ] [MAIN ] Update Progress: 272/2367 symbols completed. BHP took 40.2s. Estimated remaining: 4h 20m 10s. Rate: 8.1 symbols/min [2025-10-20T17:33:51.239598] [INFO ] [MAIN ] BILL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:51.290797] [DEBUG] [LOCAL ] BILL: Loaded 53277 existing records from primary storage. [2025-10-20T17:33:51.291196] [INFO ] [MAIN ] BILL: Loaded 53277 existing records. [2025-10-20T17:33:51.451870] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-07. [2025-10-20T17:33:51.475059] [WARN ] [MAIN ] BILL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:51.475464] [INFO ] [MAIN ] BILL: Wiping existing primary data before full refresh. [2025-10-20T17:33:51.475534] [INFO ] [LOCAL ] BILL: Starting deletion of primary data. [2025-10-20T17:33:51.476060] [DEBUG] [LOCAL ] BILL: Deleted primary data file. [2025-10-20T17:33:51.476113] [INFO ] [MAIN ] BILL: Starting full primary refresh. [2025-10-20T17:33:51.827190] [INFO ] [REMOTE ] BILL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:52.251391] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-01. [2025-10-20T17:33:53.051416] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-07. [2025-10-20T17:33:53.176689] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T17:33:53.851434] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-08. [2025-10-20T17:33:53.954953] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T17:33:53.991815] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:33:54.651353] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-08. [2025-10-20T17:33:55.205712] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:33:55.451374] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-02. [2025-10-20T17:33:55.775737] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:33:56.251402] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-09. [2025-10-20T17:33:57.051382] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-07. [2025-10-20T17:33:57.163524] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T17:33:57.395231] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T17:33:57.851422] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-09. [2025-10-20T17:33:58.096957] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:33:58.651406] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-08. [2025-10-20T17:33:58.834413] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T17:33:59.019975] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-10-20T17:33:59.451373] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-10. [2025-10-20T17:34:00.251441] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-09. [2025-10-20T17:34:00.643403] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T17:34:00.647437] [INFO ] [REMOTE ] BHR: Full fetch complete (31338 records). [2025-10-20T17:34:00.647515] [DEBUG] [VALIDATOR ] BHR: Validating primary data. [2025-10-20T17:34:00.808998] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T17:34:01.027806] [DEBUG] [VALIDATOR ] BHR: All expected trading days are present. [2025-10-20T17:34:01.027888] [DEBUG] [VALIDATOR ] BHR: Primary data validation passed. [2025-10-20T17:34:01.028346] [INFO ] [MAIN ] BHR: Fetched primary data is valid. [2025-10-20T17:34:01.028412] [DEBUG] [LOCAL ] BHR: Saving primary data. [2025-10-20T17:34:01.029159] [DEBUG] [LOCAL ] BHR: Trimmed 31338 records to 22056 (keeping 200 days of data) [2025-10-20T17:34:01.076307] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-10. [2025-10-20T17:34:01.155227] [DEBUG] [LOCAL ] BHR: Primary data saved (22056 records). [2025-10-20T17:34:01.155355] [INFO ] [MAIN ] BHR: Full history saved. [2025-10-20T17:34:01.155428] [INFO ] [MAIN ] Update Progress: 273/2367 symbols completed. BHR took 29.3s. Estimated remaining: 4h 20m 23s. Rate: 8.0 symbols/min [2025-10-20T17:34:01.333337] [INFO ] [MAIN ] BIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:01.355892] [DEBUG] [LOCAL ] BIO: Loaded 24345 existing records from primary storage. [2025-10-20T17:34:01.356159] [INFO ] [MAIN ] BIO: Loaded 24345 existing records. [2025-10-20T17:34:01.528363] [WARN ] [MAIN ] BIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:01.528601] [INFO ] [MAIN ] BIO: Wiping existing primary data before full refresh. [2025-10-20T17:34:01.528678] [INFO ] [LOCAL ] BIO: Starting deletion of primary data. [2025-10-20T17:34:01.529021] [DEBUG] [LOCAL ] BIO: Deleted primary data file. [2025-10-20T17:34:01.529076] [INFO ] [MAIN ] BIO: Starting full primary refresh. [2025-10-20T17:34:01.856541] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-03. [2025-10-20T17:34:01.892272] [INFO ] [REMOTE ] BIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:02.651445] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-01. [2025-10-20T17:34:03.277817] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T17:34:03.285761] [INFO ] [REMOTE ] BHVN: Full fetch complete (70178 records). [2025-10-20T17:34:03.285959] [DEBUG] [VALIDATOR ] BHVN: Validating primary data. [2025-10-20T17:34:03.286910] [FATAL] [VALIDATOR ] BHVN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:34:03.287003] [FATAL] [MAIN ] BHVN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:34:03.287047] [WARN ] [MAIN ] BHVN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:34:03.287114] [INFO ] [LOCAL ] BHVN: Starting deletion of primary data. [2025-10-20T17:34:03.287178] [DEBUG] [LOCAL ] BHVN: No primary data file found to delete. [2025-10-20T17:34:03.287217] [INFO ] [LOCAL ] BHVN: Starting deletion of derivative data. [2025-10-20T17:34:03.287427] [DEBUG] [LOCAL ] BHVN: Deleted derivative data file. [2025-10-20T17:34:03.287519] [INFO ] [MAIN ] Update Progress: 274/2367 symbols completed. BHVN took 28.3s. Estimated remaining: 4h 19m 34s. Rate: 8.1 symbols/min [2025-10-20T17:34:03.459305] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-10. [2025-10-20T17:34:03.491843] [INFO ] [MAIN ] BIP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:03.526428] [DEBUG] [LOCAL ] BIP: Loaded 39830 existing records from primary storage. [2025-10-20T17:34:03.526739] [INFO ] [MAIN ] BIP: Loaded 39830 existing records. [2025-10-20T17:34:03.709701] [WARN ] [MAIN ] BIP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:03.709961] [INFO ] [MAIN ] BIP: Wiping existing primary data before full refresh. [2025-10-20T17:34:03.710031] [INFO ] [LOCAL ] BIP: Starting deletion of primary data. [2025-10-20T17:34:03.710422] [DEBUG] [LOCAL ] BIP: Deleted primary data file. [2025-10-20T17:34:03.710539] [INFO ] [MAIN ] BIP: Starting full primary refresh. [2025-10-20T17:34:03.863899] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T17:34:03.988415] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:34:04.056374] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T17:34:04.073031] [INFO ] [REMOTE ] BHV: Full fetch complete (1373 records). [2025-10-20T17:34:04.078266] [DEBUG] [VALIDATOR ] BHV: Validating primary data. [2025-10-20T17:34:04.083948] [FATAL] [VALIDATOR ] BHV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:34:04.113431] [FATAL] [MAIN ] BHV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:34:04.117466] [WARN ] [MAIN ] BHV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:34:04.134637] [INFO ] [LOCAL ] BHV: Starting deletion of primary data. [2025-10-20T17:34:04.138766] [INFO ] [REMOTE ] BIP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:04.139167] [DEBUG] [LOCAL ] BHV: No primary data file found to delete. [2025-10-20T17:34:04.139231] [INFO ] [LOCAL ] BHV: Starting deletion of derivative data. [2025-10-20T17:34:04.139281] [DEBUG] [LOCAL ] BHV: No derivative data file found to delete. [2025-10-20T17:34:04.139341] [INFO ] [MAIN ] Update Progress: 275/2367 symbols completed. BHV took 30.0s. Estimated remaining: 4h 18m 37s. Rate: 8.1 symbols/min [2025-10-20T17:34:04.256549] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-02. [2025-10-20T17:34:04.306499] [INFO ] [MAIN ] BIPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:04.342255] [DEBUG] [LOCAL ] BIPC: Loaded 41904 existing records from primary storage. [2025-10-20T17:34:04.342544] [INFO ] [MAIN ] BIPC: Loaded 41904 existing records. [2025-10-20T17:34:04.514311] [WARN ] [MAIN ] BIPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:04.514519] [INFO ] [MAIN ] BIPC: Wiping existing primary data before full refresh. [2025-10-20T17:34:04.514566] [INFO ] [LOCAL ] BIPC: Starting deletion of primary data. [2025-10-20T17:34:04.514929] [DEBUG] [LOCAL ] BIPC: Deleted primary data file. [2025-10-20T17:34:04.514984] [INFO ] [MAIN ] BIPC: Starting full primary refresh. [2025-10-20T17:34:04.859124] [INFO ] [REMOTE ] BIPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:05.051366] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-01. [2025-10-20T17:34:05.391851] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T17:34:05.851382] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-03. [2025-10-20T17:34:06.610174] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:34:06.651417] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-04. [2025-10-20T17:34:06.838501] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:34:07.451414] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-04. [2025-10-20T17:34:08.251450] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-02. [2025-10-20T17:34:08.612166] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T17:34:08.669559] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T17:34:09.051438] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-01. [2025-10-20T17:34:09.671488] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:34:09.851400] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-05. [2025-10-20T17:34:10.311216] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T17:34:10.651417] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-03. [2025-10-20T17:34:11.451381] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-05. [2025-10-20T17:34:11.710269] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:34:12.017195] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:34:12.251392] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-02. [2025-10-20T17:34:12.667726] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T17:34:13.051427] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-04. [2025-10-20T17:34:13.501565] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:34:13.851429] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-03. [2025-10-20T17:34:14.575375] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T17:34:14.651431] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-05. [2025-10-20T17:34:15.451396] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-06. [2025-10-20T17:34:15.697712] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T17:34:16.252841] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-06. [2025-10-20T17:34:16.256279] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T17:34:16.744082] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:34:17.051436] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-07. [2025-10-20T17:34:17.706639] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T17:34:17.851411] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-04. [2025-10-20T17:34:18.228446] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T17:34:18.651380] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-08. [2025-10-20T17:34:19.195933] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:34:19.451339] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-05. [2025-10-20T17:34:19.469537] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T17:34:20.251366] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-09. [2025-10-20T17:34:20.956557] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T17:34:20.997558] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T17:34:21.051348] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-06. [2025-10-20T17:34:21.851353] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-07. [2025-10-20T17:34:22.430401] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T17:34:22.651391] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-07. [2025-10-20T17:34:23.451368] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-06. [2025-10-20T17:34:23.859919] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T17:34:24.229201] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T17:34:24.251395] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-08. [2025-10-20T17:34:24.627311] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T17:34:25.051378] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-10. [2025-10-20T17:34:25.430775] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T17:34:25.851393] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-09. [2025-10-20T17:34:26.149449] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T17:34:26.152502] [INFO ] [REMOTE ] BIO: Full fetch complete (34239 records). [2025-10-20T17:34:26.152633] [DEBUG] [VALIDATOR ] BIO: Validating primary data. [2025-10-20T17:34:26.153261] [FATAL] [VALIDATOR ] BIO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:34:26.153348] [FATAL] [MAIN ] BIO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:34:26.153393] [WARN ] [MAIN ] BIO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:34:26.153431] [INFO ] [LOCAL ] BIO: Starting deletion of primary data. [2025-10-20T17:34:26.153517] [DEBUG] [LOCAL ] BIO: No primary data file found to delete. [2025-10-20T17:34:26.153572] [INFO ] [LOCAL ] BIO: Starting deletion of derivative data. [2025-10-20T17:34:26.153881] [DEBUG] [LOCAL ] BIO: Deleted derivative data file. [2025-10-20T17:34:26.153971] [INFO ] [MAIN ] Update Progress: 276/2367 symbols completed. BIO took 25.0s. Estimated remaining: 4h 20m 20s. Rate: 8.0 symbols/min [2025-10-20T17:34:26.342344] [INFO ] [MAIN ] BIPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:26.348336] [DEBUG] [LOCAL ] BIPH: Loaded 2538 existing records from primary storage. [2025-10-20T17:34:26.348551] [INFO ] [MAIN ] BIPH: Loaded 2538 existing records. [2025-10-20T17:34:26.532242] [WARN ] [MAIN ] BIPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:26.532504] [INFO ] [MAIN ] BIPH: Wiping existing primary data before full refresh. [2025-10-20T17:34:26.532552] [INFO ] [LOCAL ] BIPH: Starting deletion of primary data. [2025-10-20T17:34:26.532734] [DEBUG] [LOCAL ] BIPH: Deleted primary data file. [2025-10-20T17:34:26.532790] [INFO ] [MAIN ] BIPH: Starting full primary refresh. [2025-10-20T17:34:26.656730] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-07. [2025-10-20T17:34:26.909556] [INFO ] [REMOTE ] BIPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:27.280245] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:34:27.451352] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-10. [2025-10-20T17:34:27.928997] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T17:34:28.251381] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-08. [2025-10-20T17:34:28.975798] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T17:34:28.982598] [INFO ] [REMOTE ] BIP: Full fetch complete (57336 records). [2025-10-20T17:34:28.982795] [DEBUG] [VALIDATOR ] BIP: Validating primary data. [2025-10-20T17:34:28.984082] [FATAL] [VALIDATOR ] BIP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:34:28.984190] [FATAL] [MAIN ] BIP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:34:28.984235] [WARN ] [MAIN ] BIP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:34:28.984280] [INFO ] [LOCAL ] BIP: Starting deletion of primary data. [2025-10-20T17:34:28.984345] [DEBUG] [LOCAL ] BIP: No primary data file found to delete. [2025-10-20T17:34:28.984382] [INFO ] [LOCAL ] BIP: Starting deletion of derivative data. [2025-10-20T17:34:28.984531] [DEBUG] [LOCAL ] BIP: Deleted derivative data file. [2025-10-20T17:34:28.984612] [INFO ] [MAIN ] Update Progress: 277/2367 symbols completed. BIP took 25.7s. Estimated remaining: 4h 19m 38s. Rate: 8.1 symbols/min [2025-10-20T17:34:29.051388] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-01. [2025-10-20T17:34:29.504461] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T17:34:29.851398] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-02. [2025-10-20T17:34:29.921590] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T17:34:30.222937] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T17:34:30.651380] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-08. [2025-10-20T17:34:31.451408] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-03. [2025-10-20T17:34:31.820468] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-10-20T17:34:32.159416] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:34:32.251415] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-09. [2025-10-20T17:34:33.051440] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-09. [2025-10-20T17:34:33.394752] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T17:34:33.851387] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-10. [2025-10-20T17:34:34.652624] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-04. [2025-10-20T17:34:34.660282] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T17:34:35.277289] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T17:34:35.451384] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-10. [2025-10-20T17:34:35.518568] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T17:34:35.525377] [INFO ] [REMOTE ] BIPC: Full fetch complete (60293 records). [2025-10-20T17:34:35.525551] [DEBUG] [VALIDATOR ] BIPC: Validating primary data. [2025-10-20T17:34:35.934759] [DEBUG] [VALIDATOR ] BIPC: All expected trading days are present. [2025-10-20T17:34:35.934846] [DEBUG] [VALIDATOR ] BIPC: Primary data validation passed. [2025-10-20T17:34:35.935485] [INFO ] [MAIN ] BIPC: Fetched primary data is valid. [2025-10-20T17:34:35.935526] [DEBUG] [LOCAL ] BIPC: Saving primary data. [2025-10-20T17:34:35.936798] [DEBUG] [LOCAL ] BIPC: Trimmed 60293 records to 41246 (keeping 200 days of data) [2025-10-20T17:34:36.180288] [DEBUG] [LOCAL ] BIPC: Primary data saved (41246 records). [2025-10-20T17:34:36.180425] [INFO ] [MAIN ] BIPC: Full history saved. [2025-10-20T17:34:36.180499] [INFO ] [MAIN ] Update Progress: 278/2367 symbols completed. BIPC took 32.0s. Estimated remaining: 4h 19m 28s. Rate: 8.1 symbols/min [2025-10-20T17:34:36.251395] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-05. [2025-10-20T17:34:36.699169] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T17:34:37.051415] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-06. [2025-10-20T17:34:37.498775] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T17:34:37.506587] [INFO ] [REMOTE ] BILL: Full fetch complete (76124 records). [2025-10-20T17:34:37.506758] [DEBUG] [VALIDATOR ] BILL: Validating primary data. [2025-10-20T17:34:37.603834] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T17:34:37.856700] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-07. [2025-10-20T17:34:37.901357] [DEBUG] [VALIDATOR ] BILL: All expected trading days are present. [2025-10-20T17:34:37.901486] [DEBUG] [VALIDATOR ] BILL: Primary data validation passed. [2025-10-20T17:34:37.902374] [INFO ] [MAIN ] BILL: Fetched primary data is valid. [2025-10-20T17:34:37.902443] [DEBUG] [LOCAL ] BILL: Saving primary data. [2025-10-20T17:34:37.904573] [DEBUG] [LOCAL ] BILL: Trimmed 76124 records to 52513 (keeping 200 days of data) [2025-10-20T17:34:38.234979] [DEBUG] [LOCAL ] BILL: Primary data saved (52513 records). [2025-10-20T17:34:38.235192] [INFO ] [MAIN ] BILL: Full history saved. [2025-10-20T17:34:38.235620] [INFO ] [MAIN ] Update Progress: 279/2367 symbols completed. BILL took 47.2s. Estimated remaining: 4h 18m 40s. Rate: 8.1 symbols/min [2025-10-20T17:34:38.359230] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-10-20T17:34:38.651402] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-08. [2025-10-20T17:34:39.147818] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-10-20T17:34:39.451485] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-09. [2025-10-20T17:34:39.824020] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-10-20T17:34:40.251414] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-10. [2025-10-20T17:34:40.656271] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T17:34:40.657148] [INFO ] [REMOTE ] BIPH: Full fetch complete (3896 records). [2025-10-20T17:34:40.657216] [DEBUG] [VALIDATOR ] BIPH: Validating primary data. [2025-10-20T17:34:41.022490] [DEBUG] [VALIDATOR ] BIPH: All expected trading days are present. [2025-10-20T17:34:41.022570] [DEBUG] [VALIDATOR ] BIPH: Primary data validation passed. [2025-10-20T17:34:41.022725] [INFO ] [MAIN ] BIPH: Fetched primary data is valid. [2025-10-20T17:34:41.022785] [DEBUG] [LOCAL ] BIPH: Saving primary data. [2025-10-20T17:34:41.023129] [DEBUG] [LOCAL ] BIPH: Trimmed 3896 records to 2395 (keeping 200 days of data) [2025-10-20T17:34:41.037580] [DEBUG] [LOCAL ] BIPH: Primary data saved (2395 records). [2025-10-20T17:34:41.037684] [INFO ] [MAIN ] BIPH: Full history saved. [2025-10-20T17:34:41.037768] [INFO ] [MAIN ] Update Progress: 280/2367 symbols completed. BIPH took 14.9s. Estimated remaining: 4h 17m 58s. Rate: 8.1 symbols/min [2025-10-20T17:34:41.715123] [INFO ] [MAIN ] BIPJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:41.753614] [INFO ] [MAIN ] BIT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:41.762329] [INFO ] [MAIN ] BIPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:41.840542] [INFO ] [MAIN ] BIRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:41.845985] [DEBUG] [LOCAL ] BIPJ: Loaded 3197 existing records from primary storage. [2025-10-20T17:34:41.849781] [INFO ] [MAIN ] BIPJ: Loaded 3197 existing records. [2025-10-20T17:34:41.849460] [DEBUG] [LOCAL ] BIPI: Loaded 3038 existing records from primary storage. [2025-10-20T17:34:41.863956] [INFO ] [MAIN ] BIPI: Loaded 3038 existing records. [2025-10-20T17:34:42.096042] [DEBUG] [LOCAL ] BIT: Loaded 24078 existing records from primary storage. [2025-10-20T17:34:42.117967] [INFO ] [MAIN ] BIT: Loaded 24078 existing records. [2025-10-20T17:34:42.303679] [WARN ] [MAIN ] BIPJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:42.334864] [WARN ] [MAIN ] BIPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:42.335149] [INFO ] [MAIN ] BIPJ: Wiping existing primary data before full refresh. [2025-10-20T17:34:42.361511] [INFO ] [LOCAL ] BIPJ: Starting deletion of primary data. [2025-10-20T17:34:42.356213] [INFO ] [MAIN ] BIPI: Wiping existing primary data before full refresh. [2025-10-20T17:34:42.361906] [INFO ] [LOCAL ] BIPI: Starting deletion of primary data. [2025-10-20T17:34:42.367216] [DEBUG] [LOCAL ] BIPJ: Deleted primary data file. [2025-10-20T17:34:42.382418] [DEBUG] [LOCAL ] BIPI: Deleted primary data file. [2025-10-20T17:34:42.367448] [INFO ] [MAIN ] BIPJ: Starting full primary refresh. [2025-10-20T17:34:42.383730] [INFO ] [MAIN ] BIPI: Starting full primary refresh. [2025-10-20T17:34:42.467299] [DEBUG] [LOCAL ] BIRK: Loaded 51882 existing records from primary storage. [2025-10-20T17:34:42.657750] [INFO ] [MAIN ] BIRK: Loaded 51882 existing records. [2025-10-20T17:34:42.793173] [WARN ] [MAIN ] BIT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:42.829200] [INFO ] [MAIN ] BIT: Wiping existing primary data before full refresh. [2025-10-20T17:34:42.856853] [INFO ] [LOCAL ] BIT: Starting deletion of primary data. [2025-10-20T17:34:42.877931] [DEBUG] [LOCAL ] BIT: Deleted primary data file. [2025-10-20T17:34:42.903922] [INFO ] [MAIN ] BIT: Starting full primary refresh. [2025-10-20T17:34:43.434702] [WARN ] [MAIN ] BIRK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:43.491043] [INFO ] [MAIN ] BIRK: Wiping existing primary data before full refresh. [2025-10-20T17:34:43.530889] [INFO ] [LOCAL ] BIRK: Starting deletion of primary data. [2025-10-20T17:34:43.664083] [DEBUG] [LOCAL ] BIRK: Deleted primary data file. [2025-10-20T17:34:43.691268] [INFO ] [REMOTE ] BIPJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:43.740372] [INFO ] [REMOTE ] BIPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:43.763502] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-01. [2025-10-20T17:34:43.764180] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-01. [2025-10-20T17:34:43.763676] [INFO ] [MAIN ] BIRK: Starting full primary refresh. [2025-10-20T17:34:44.150700] [INFO ] [REMOTE ] BIT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:44.169445] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-01. [2025-10-20T17:34:44.367909] [INFO ] [REMOTE ] BIRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:44.368189] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T17:34:44.368823] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-01. [2025-10-20T17:34:44.370601] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-02. [2025-10-20T17:34:44.781557] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T17:34:44.847140] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T17:34:45.051420] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-02. [2025-10-20T17:34:45.301615] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T17:34:45.647093] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T17:34:45.662395] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:34:45.851399] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-02. [2025-10-20T17:34:46.651376] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-03. [2025-10-20T17:34:46.973168] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:34:47.230193] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T17:34:47.451392] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-03. [2025-10-20T17:34:47.871259] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T17:34:48.251414] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-03. [2025-10-20T17:34:49.051363] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-04. [2025-10-20T17:34:49.457958] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T17:34:49.583362] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T17:34:49.851392] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-04. [2025-10-20T17:34:50.651397] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-02. [2025-10-20T17:34:51.463027] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-05. [2025-10-20T17:34:51.484554] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T17:34:51.496008] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T17:34:51.774187] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T17:34:52.251376] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-03. [2025-10-20T17:34:53.051443] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-05. [2025-10-20T17:34:53.262874] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:34:53.851402] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-04. [2025-10-20T17:34:54.471223] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T17:34:54.651418] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-04. [2025-10-20T17:34:54.805007] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T17:34:55.055051] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-10-20T17:34:55.451417] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-05. [2025-10-20T17:34:55.880349] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T17:34:56.251446] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-06. [2025-10-20T17:34:56.867685] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T17:34:57.051445] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-05. [2025-10-20T17:34:57.851375] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-06. [2025-10-20T17:34:58.092669] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T17:34:58.651397] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-06. [2025-10-20T17:34:59.103231] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T17:34:59.183355] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:34:59.451388] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-06. [2025-10-20T17:35:00.056434] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T17:35:00.251364] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-07. [2025-10-20T17:35:01.051351] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-07. [2025-10-20T17:35:01.697069] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T17:35:01.851367] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-07. [2025-10-20T17:35:02.092282] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T17:35:02.413668] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T17:35:02.651362] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-07. [2025-10-20T17:35:03.451404] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-08. [2025-10-20T17:35:03.469736] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T17:35:04.044085] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T17:35:04.251447] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-08. [2025-10-20T17:35:05.051432] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-08. [2025-10-20T17:35:05.747949] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T17:35:05.746165] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:35:05.851380] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-08. [2025-10-20T17:35:06.651393] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-09. [2025-10-20T17:35:07.451388] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-09. [2025-10-20T17:35:08.251454] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-09. [2025-10-20T17:35:08.966175] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T17:35:09.051408] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-10. [2025-10-20T17:35:09.266620] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:35:09.562795] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-08 (3.71 seconds). [2025-10-20T17:35:09.600544] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-09 (2.95 seconds). [2025-10-20T17:35:09.603442] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T17:35:09.612160] [INFO ] [REMOTE ] BIPJ: Full fetch complete (4204 records). [2025-10-20T17:35:09.612235] [DEBUG] [VALIDATOR ] BIPJ: Validating primary data. [2025-10-20T17:35:09.856584] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-10. [2025-10-20T17:35:10.003283] [DEBUG] [VALIDATOR ] BIPJ: All expected trading days are present. [2025-10-20T17:35:10.003519] [DEBUG] [VALIDATOR ] BIPJ: Primary data validation passed. [2025-10-20T17:35:10.003914] [INFO ] [MAIN ] BIPJ: Fetched primary data is valid. [2025-10-20T17:35:10.004032] [DEBUG] [LOCAL ] BIPJ: Saving primary data. [2025-10-20T17:35:10.004925] [DEBUG] [LOCAL ] BIPJ: Trimmed 4204 records to 3133 (keeping 200 days of data) [2025-10-20T17:35:10.023638] [DEBUG] [LOCAL ] BIPJ: Primary data saved (3133 records). [2025-10-20T17:35:10.023732] [INFO ] [MAIN ] BIPJ: Full history saved. [2025-10-20T17:35:10.023816] [INFO ] [MAIN ] Update Progress: 281/2367 symbols completed. BIPJ took 29.0s. Estimated remaining: 4h 20m 31s. Rate: 8.0 symbols/min [2025-10-20T17:35:10.189244] [INFO ] [MAIN ] BJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:10.235562] [DEBUG] [LOCAL ] BJ: Loaded 52353 existing records from primary storage. [2025-10-20T17:35:10.235894] [INFO ] [MAIN ] BJ: Loaded 52353 existing records. [2025-10-20T17:35:10.409072] [WARN ] [MAIN ] BJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:10.409300] [INFO ] [MAIN ] BJ: Wiping existing primary data before full refresh. [2025-10-20T17:35:10.409346] [INFO ] [LOCAL ] BJ: Starting deletion of primary data. [2025-10-20T17:35:10.409931] [DEBUG] [LOCAL ] BJ: Deleted primary data file. [2025-10-20T17:35:10.409986] [INFO ] [MAIN ] BJ: Starting full primary refresh. [2025-10-20T17:35:10.494775] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T17:35:10.517960] [INFO ] [REMOTE ] BIPI: Full fetch complete (4623 records). [2025-10-20T17:35:10.524089] [DEBUG] [VALIDATOR ] BIPI: Validating primary data. [2025-10-20T17:35:10.545541] [FATAL] [VALIDATOR ] BIPI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:35:10.568845] [FATAL] [MAIN ] BIPI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:35:10.579051] [WARN ] [MAIN ] BIPI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:35:10.628961] [INFO ] [LOCAL ] BIPI: Starting deletion of primary data. [2025-10-20T17:35:10.664438] [DEBUG] [LOCAL ] BIPI: No primary data file found to delete. [2025-10-20T17:35:10.658932] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-09. [2025-10-20T17:35:10.688239] [INFO ] [LOCAL ] BIPI: Starting deletion of derivative data. [2025-10-20T17:35:10.700714] [DEBUG] [LOCAL ] BIPI: Deleted derivative data file. [2025-10-20T17:35:10.722040] [INFO ] [MAIN ] Update Progress: 282/2367 symbols completed. BIPI took 29.7s. Estimated remaining: 4h 19m 33s. Rate: 8.0 symbols/min [2025-10-20T17:35:10.842378] [INFO ] [REMOTE ] BJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:10.985710] [INFO ] [MAIN ] BK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:11.042297] [DEBUG] [LOCAL ] BK: Loaded 54323 existing records from primary storage. [2025-10-20T17:35:11.042887] [INFO ] [MAIN ] BK: Loaded 54323 existing records. [2025-10-20T17:35:11.245056] [WARN ] [MAIN ] BK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:11.245365] [INFO ] [MAIN ] BK: Wiping existing primary data before full refresh. [2025-10-20T17:35:11.245415] [INFO ] [LOCAL ] BK: Starting deletion of primary data. [2025-10-20T17:35:11.245619] [DEBUG] [LOCAL ] BK: Deleted primary data file. [2025-10-20T17:35:11.245708] [INFO ] [MAIN ] BK: Starting full primary refresh. [2025-10-20T17:35:11.463711] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-10. [2025-10-20T17:35:11.624894] [INFO ] [REMOTE ] BK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:12.251403] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-01. [2025-10-20T17:35:12.582166] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:35:12.935683] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T17:35:12.940252] [INFO ] [REMOTE ] BIT: Full fetch complete (32123 records). [2025-10-20T17:35:12.940329] [DEBUG] [VALIDATOR ] BIT: Validating primary data. [2025-10-20T17:35:13.056512] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-10. [2025-10-20T17:35:13.314271] [DEBUG] [VALIDATOR ] BIT: All expected trading days are present. [2025-10-20T17:35:13.314352] [DEBUG] [VALIDATOR ] BIT: Primary data validation passed. [2025-10-20T17:35:13.314796] [INFO ] [MAIN ] BIT: Fetched primary data is valid. [2025-10-20T17:35:13.314856] [DEBUG] [LOCAL ] BIT: Saving primary data. [2025-10-20T17:35:13.315534] [DEBUG] [LOCAL ] BIT: Trimmed 32123 records to 23780 (keeping 200 days of data) [2025-10-20T17:35:13.457378] [DEBUG] [LOCAL ] BIT: Primary data saved (23780 records). [2025-10-20T17:35:13.457518] [INFO ] [MAIN ] BIT: Full history saved. [2025-10-20T17:35:13.457599] [INFO ] [MAIN ] Update Progress: 283/2367 symbols completed. BIT took 32.4s. Estimated remaining: 4h 18m 51s. Rate: 8.1 symbols/min [2025-10-20T17:35:13.631991] [INFO ] [MAIN ] BKD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:13.677383] [DEBUG] [LOCAL ] BKD: Loaded 51205 existing records from primary storage. [2025-10-20T17:35:13.677714] [INFO ] [MAIN ] BKD: Loaded 51205 existing records. [2025-10-20T17:35:13.851749] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-01. [2025-10-20T17:35:13.854887] [WARN ] [MAIN ] BKD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:13.856093] [INFO ] [MAIN ] BKD: Wiping existing primary data before full refresh. [2025-10-20T17:35:13.856144] [INFO ] [LOCAL ] BKD: Starting deletion of primary data. [2025-10-20T17:35:13.856360] [DEBUG] [LOCAL ] BKD: Deleted primary data file. [2025-10-20T17:35:13.856430] [INFO ] [MAIN ] BKD: Starting full primary refresh. [2025-10-20T17:35:14.216715] [INFO ] [REMOTE ] BKD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:14.247880] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:35:14.651418] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-01. [2025-10-20T17:35:14.934590] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T17:35:14.942308] [INFO ] [REMOTE ] BIRK: Full fetch complete (73389 records). [2025-10-20T17:35:14.942428] [DEBUG] [VALIDATOR ] BIRK: Validating primary data. [2025-10-20T17:35:15.335988] [DEBUG] [VALIDATOR ] BIRK: All expected trading days are present. [2025-10-20T17:35:15.336102] [DEBUG] [VALIDATOR ] BIRK: Primary data validation passed. [2025-10-20T17:35:15.337160] [INFO ] [MAIN ] BIRK: Fetched primary data is valid. [2025-10-20T17:35:15.337230] [DEBUG] [LOCAL ] BIRK: Saving primary data. [2025-10-20T17:35:15.338815] [DEBUG] [LOCAL ] BIRK: Trimmed 73389 records to 51198 (keeping 200 days of data) [2025-10-20T17:35:15.452953] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-02. [2025-10-20T17:35:15.484570] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:35:15.671459] [DEBUG] [LOCAL ] BIRK: Primary data saved (51198 records). [2025-10-20T17:35:15.671614] [INFO ] [MAIN ] BIRK: Full history saved. [2025-10-20T17:35:15.671732] [INFO ] [MAIN ] Update Progress: 284/2367 symbols completed. BIRK took 34.6s. Estimated remaining: 4h 18m 5s. Rate: 8.1 symbols/min [2025-10-20T17:35:15.845669] [INFO ] [MAIN ] BKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:15.845859] [DEBUG] [LOCAL ] BKE: No existing data file found in primary storage. [2025-10-20T17:35:15.845896] [WARN ] [MAIN ] BKE: No existing data found, performing full refresh instead. [2025-10-20T17:35:15.846013] [INFO ] [MAIN ] BKE: Wiping existing primary data before full refresh. [2025-10-20T17:35:15.846088] [INFO ] [LOCAL ] BKE: Starting deletion of primary data. [2025-10-20T17:35:15.846150] [DEBUG] [LOCAL ] BKE: No primary data file found to delete. [2025-10-20T17:35:15.846183] [INFO ] [MAIN ] BKE: Starting full primary refresh. [2025-10-20T17:35:16.198917] [INFO ] [REMOTE ] BKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:16.224821] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:35:16.251332] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-01. [2025-10-20T17:35:16.972138] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:35:17.051442] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-02. [2025-10-20T17:35:17.458378] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:35:17.851377] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-03. [2025-10-20T17:35:18.595344] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:35:18.651357] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-03. [2025-10-20T17:35:19.451406] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-02. [2025-10-20T17:35:19.721207] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T17:35:19.986444] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T17:35:20.251427] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-02. [2025-10-20T17:35:21.051484] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-04. [2025-10-20T17:35:21.687983] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-10-20T17:35:21.851375] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-03. [2025-10-20T17:35:22.465985] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-10-20T17:35:22.651451] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-03. [2025-10-20T17:35:23.451369] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-04. [2025-10-20T17:35:23.778538] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-04 (2.73 seconds). [2025-10-20T17:35:23.961843] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T17:35:24.051958] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T17:35:24.251366] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-04. [2025-10-20T17:35:25.051346] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-05. [2025-10-20T17:35:25.442993] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T17:35:25.755115] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T17:35:25.851346] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-05. [2025-10-20T17:35:26.617129] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:35:26.651337] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-06. [2025-10-20T17:35:27.236438] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T17:35:27.451397] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-06. [2025-10-20T17:35:28.256496] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-05. [2025-10-20T17:35:28.269794] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:35:28.757342] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:35:29.051387] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-07. [2025-10-20T17:35:29.626345] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:35:29.851423] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-04. [2025-10-20T17:35:30.440736] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:35:30.651365] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-07. [2025-10-20T17:35:31.451385] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-08. [2025-10-20T17:35:31.844732] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T17:35:32.251346] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-05. [2025-10-20T17:35:32.339187] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T17:35:32.569302] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:35:33.051422] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-08. [2025-10-20T17:35:33.716980] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:35:33.851435] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-06. [2025-10-20T17:35:34.651433] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-09. [2025-10-20T17:35:34.797666] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T17:35:35.168397] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T17:35:35.451386] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-09. [2025-10-20T17:35:36.082755] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:35:36.251374] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-07. [2025-10-20T17:35:37.051364] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-06. [2025-10-20T17:35:37.107463] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T17:35:37.820505] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T17:35:37.851402] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-10. [2025-10-20T17:35:38.651412] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-10. [2025-10-20T17:35:38.966525] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:35:39.451356] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-07. [2025-10-20T17:35:39.545591] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:35:39.552699] [INFO ] [REMOTE ] BKD: Full fetch complete (71776 records). [2025-10-20T17:35:39.552835] [DEBUG] [VALIDATOR ] BKD: Validating primary data. [2025-10-20T17:35:39.553570] [FATAL] [VALIDATOR ] BKD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:35:39.553690] [FATAL] [MAIN ] BKD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:35:39.553758] [WARN ] [MAIN ] BKD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:35:39.553844] [INFO ] [LOCAL ] BKD: Starting deletion of primary data. [2025-10-20T17:35:39.553923] [DEBUG] [LOCAL ] BKD: No primary data file found to delete. [2025-10-20T17:35:39.553972] [INFO ] [LOCAL ] BKD: Starting deletion of derivative data. [2025-10-20T17:35:39.554206] [DEBUG] [LOCAL ] BKD: Deleted derivative data file. [2025-10-20T17:35:39.554316] [INFO ] [MAIN ] Update Progress: 285/2367 symbols completed. BKD took 26.1s. Estimated remaining: 4h 19m 58s. Rate: 8.0 symbols/min [2025-10-20T17:35:39.723457] [INFO ] [MAIN ] BKH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:39.756103] [DEBUG] [LOCAL ] BKH: Loaded 37744 existing records from primary storage. [2025-10-20T17:35:39.756411] [INFO ] [MAIN ] BKH: Loaded 37744 existing records. [2025-10-20T17:35:39.940943] [WARN ] [MAIN ] BKH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:39.941216] [INFO ] [MAIN ] BKH: Wiping existing primary data before full refresh. [2025-10-20T17:35:39.941266] [INFO ] [LOCAL ] BKH: Starting deletion of primary data. [2025-10-20T17:35:39.941561] [DEBUG] [LOCAL ] BKH: Deleted primary data file. [2025-10-20T17:35:39.941617] [INFO ] [MAIN ] BKH: Starting full primary refresh. [2025-10-20T17:35:40.261504] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-08. [2025-10-20T17:35:40.278854] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:35:40.338814] [INFO ] [REMOTE ] BKH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:40.339373] [INFO ] [REMOTE ] BKE: Full fetch complete (50414 records). [2025-10-20T17:35:40.339452] [DEBUG] [VALIDATOR ] BKE: Validating primary data. [2025-10-20T17:35:40.734827] [DEBUG] [VALIDATOR ] BKE: All expected trading days are present. [2025-10-20T17:35:40.734935] [DEBUG] [VALIDATOR ] BKE: Primary data validation passed. [2025-10-20T17:35:40.735613] [INFO ] [MAIN ] BKE: Fetched primary data is valid. [2025-10-20T17:35:40.735686] [DEBUG] [LOCAL ] BKE: Saving primary data. [2025-10-20T17:35:40.736907] [DEBUG] [LOCAL ] BKE: Trimmed 50414 records to 33953 (keeping 200 days of data) [2025-10-20T17:35:40.949778] [DEBUG] [LOCAL ] BKE: Primary data saved (33953 records). [2025-10-20T17:35:40.949922] [INFO ] [MAIN ] BKE: Full history saved. [2025-10-20T17:35:40.949997] [INFO ] [MAIN ] Update Progress: 286/2367 symbols completed. BKE took 25.3s. Estimated remaining: 4h 19m 6s. Rate: 8.0 symbols/min [2025-10-20T17:35:41.059591] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-01. [2025-10-20T17:35:41.135894] [INFO ] [MAIN ] BKKT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:41.171008] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T17:35:41.209673] [DEBUG] [LOCAL ] BKKT: Loaded 37334 existing records from primary storage. [2025-10-20T17:35:41.209951] [INFO ] [MAIN ] BKKT: Loaded 37334 existing records. [2025-10-20T17:35:41.398756] [WARN ] [MAIN ] BKKT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:41.398962] [INFO ] [MAIN ] BKKT: Wiping existing primary data before full refresh. [2025-10-20T17:35:41.399007] [INFO ] [LOCAL ] BKKT: Starting deletion of primary data. [2025-10-20T17:35:41.399219] [DEBUG] [LOCAL ] BKKT: Deleted primary data file. [2025-10-20T17:35:41.399275] [INFO ] [MAIN ] BKKT: Starting full primary refresh. [2025-10-20T17:35:41.794083] [INFO ] [REMOTE ] BKKT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:41.851427] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-08. [2025-10-20T17:35:41.936738] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:35:42.624823] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:35:42.651463] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-01. [2025-10-20T17:35:43.451393] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-02. [2025-10-20T17:35:43.764869] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:35:44.094587] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:35:44.251358] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-02. [2025-10-20T17:35:44.796103] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T17:35:45.051461] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-03. [2025-10-20T17:35:45.341736] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T17:35:45.851412] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-03. [2025-10-20T17:35:46.123008] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T17:35:46.651357] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-09. [2025-10-20T17:35:47.201549] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T17:35:47.451421] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-04. [2025-10-20T17:35:48.251386] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-04. [2025-10-20T17:35:48.431121] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:35:48.851192] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:35:49.051433] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-09. [2025-10-20T17:35:49.482830] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T17:35:49.851421] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-10. [2025-10-20T17:35:50.651435] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-05. [2025-10-20T17:35:51.211869] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-10-20T17:35:51.451466] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-10. [2025-10-20T17:35:51.826937] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T17:35:51.835995] [INFO ] [REMOTE ] BK: Full fetch complete (77678 records). [2025-10-20T17:35:51.836228] [DEBUG] [VALIDATOR ] BK: Validating primary data. [2025-10-20T17:35:52.225088] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:35:52.279688] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-05. [2025-10-20T17:35:52.311513] [DEBUG] [VALIDATOR ] BK: All expected trading days are present. [2025-10-20T17:35:52.312568] [DEBUG] [VALIDATOR ] BK: Primary data validation passed. [2025-10-20T17:35:52.313411] [INFO ] [MAIN ] BK: Fetched primary data is valid. [2025-10-20T17:35:52.313475] [DEBUG] [LOCAL ] BK: Saving primary data. [2025-10-20T17:35:52.315063] [DEBUG] [LOCAL ] BK: Trimmed 77678 records to 53546 (keeping 200 days of data) [2025-10-20T17:35:52.642054] [DEBUG] [LOCAL ] BK: Primary data saved (53546 records). [2025-10-20T17:35:52.642189] [INFO ] [MAIN ] BK: Full history saved. [2025-10-20T17:35:52.642261] [INFO ] [MAIN ] Update Progress: 287/2367 symbols completed. BK took 41.9s. Estimated remaining: 4h 19m 29s. Rate: 8.0 symbols/min [2025-10-20T17:35:52.813022] [INFO ] [MAIN ] BKN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:52.823814] [DEBUG] [LOCAL ] BKN: Loaded 7394 existing records from primary storage. [2025-10-20T17:35:52.823986] [INFO ] [MAIN ] BKN: Loaded 7394 existing records. [2025-10-20T17:35:53.006410] [WARN ] [MAIN ] BKN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:53.006707] [INFO ] [MAIN ] BKN: Wiping existing primary data before full refresh. [2025-10-20T17:35:53.006853] [INFO ] [LOCAL ] BKN: Starting deletion of primary data. [2025-10-20T17:35:53.007066] [DEBUG] [LOCAL ] BKN: Deleted primary data file. [2025-10-20T17:35:53.007200] [INFO ] [MAIN ] BKN: Starting full primary refresh. [2025-10-20T17:35:53.056779] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-06. [2025-10-20T17:35:53.210395] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:35:53.302985] [INFO ] [REMOTE ] BJ: Full fetch complete (74444 records). [2025-10-20T17:35:53.323693] [DEBUG] [VALIDATOR ] BJ: Validating primary data. [2025-10-20T17:35:53.504811] [INFO ] [REMOTE ] BKN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:53.836985] [DEBUG] [VALIDATOR ] BJ: All expected trading days are present. [2025-10-20T17:35:53.837136] [DEBUG] [VALIDATOR ] BJ: Primary data validation passed. [2025-10-20T17:35:53.838018] [INFO ] [MAIN ] BJ: Fetched primary data is valid. [2025-10-20T17:35:53.838118] [DEBUG] [LOCAL ] BJ: Saving primary data. [2025-10-20T17:35:53.842760] [DEBUG] [LOCAL ] BJ: Trimmed 74444 records to 51581 (keeping 200 days of data) [2025-10-20T17:35:53.892327] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-01. [2025-10-20T17:35:53.985399] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:35:54.181271] [DEBUG] [LOCAL ] BJ: Primary data saved (51581 records). [2025-10-20T17:35:54.181414] [INFO ] [MAIN ] BJ: Full history saved. [2025-10-20T17:35:54.181492] [INFO ] [MAIN ] Update Progress: 288/2367 symbols completed. BJ took 44.2s. Estimated remaining: 4h 18m 39s. Rate: 8.0 symbols/min [2025-10-20T17:35:54.353098] [INFO ] [MAIN ] BKSY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:54.398690] [DEBUG] [LOCAL ] BKSY: Loaded 49470 existing records from primary storage. [2025-10-20T17:35:54.399046] [INFO ] [MAIN ] BKSY: Loaded 49470 existing records. [2025-10-20T17:35:54.582086] [WARN ] [MAIN ] BKSY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:54.582479] [INFO ] [MAIN ] BKSY: Wiping existing primary data before full refresh. [2025-10-20T17:35:54.582734] [INFO ] [LOCAL ] BKSY: Starting deletion of primary data. [2025-10-20T17:35:54.589921] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:35:54.593006] [DEBUG] [LOCAL ] BKSY: Deleted primary data file. [2025-10-20T17:35:54.593133] [INFO ] [MAIN ] BKSY: Starting full primary refresh. [2025-10-20T17:35:54.658350] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-06. [2025-10-20T17:35:54.720977] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:35:54.976456] [INFO ] [REMOTE ] BKSY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:55.451354] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-01. [2025-10-20T17:35:56.108542] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T17:35:56.251337] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-07. [2025-10-20T17:35:56.904879] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:35:57.051373] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-02. [2025-10-20T17:35:57.663632] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:35:57.851352] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-03. [2025-10-20T17:35:57.941769] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T17:35:58.581403] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T17:35:58.651368] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-02. [2025-10-20T17:35:59.451482] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-08. [2025-10-20T17:36:00.251392] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-04. [2025-10-20T17:36:00.528990] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T17:36:01.051427] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-03. [2025-10-20T17:36:01.071825] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T17:36:01.285694] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T17:36:01.851372] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-05. [2025-10-20T17:36:02.591895] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T17:36:02.651393] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-06. [2025-10-20T17:36:02.974687] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T17:36:03.153188] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-10-20T17:36:03.451399] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-07. [2025-10-20T17:36:04.251417] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-04. [2025-10-20T17:36:05.051410] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-07. [2025-10-20T17:36:05.135012] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T17:36:05.639072] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:36:05.717447] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-10-20T17:36:05.851370] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-08. [2025-10-20T17:36:06.522638] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T17:36:06.651373] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-09. [2025-10-20T17:36:07.241348] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T17:36:07.451436] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-10. [2025-10-20T17:36:08.251399] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-05. [2025-10-20T17:36:08.300258] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T17:36:08.302137] [INFO ] [REMOTE ] BKN: Full fetch complete (11139 records). [2025-10-20T17:36:08.302224] [DEBUG] [VALIDATOR ] BKN: Validating primary data. [2025-10-20T17:36:08.684195] [DEBUG] [VALIDATOR ] BKN: All expected trading days are present. [2025-10-20T17:36:08.684284] [DEBUG] [VALIDATOR ] BKN: Primary data validation passed. [2025-10-20T17:36:08.684562] [INFO ] [MAIN ] BKN: Fetched primary data is valid. [2025-10-20T17:36:08.684601] [DEBUG] [LOCAL ] BKN: Saving primary data. [2025-10-20T17:36:08.685056] [DEBUG] [LOCAL ] BKN: Trimmed 11139 records to 7285 (keeping 200 days of data) [2025-10-20T17:36:08.729956] [DEBUG] [LOCAL ] BKN: Primary data saved (7285 records). [2025-10-20T17:36:08.730089] [INFO ] [MAIN ] BKN: Full history saved. [2025-10-20T17:36:08.730164] [INFO ] [MAIN ] Update Progress: 289/2367 symbols completed. BKN took 16.1s. Estimated remaining: 4h 19m 22s. Rate: 8.0 symbols/min [2025-10-20T17:36:08.906887] [INFO ] [MAIN ] BKT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:08.919847] [DEBUG] [LOCAL ] BKT: Loaded 12781 existing records from primary storage. [2025-10-20T17:36:08.920100] [INFO ] [MAIN ] BKT: Loaded 12781 existing records. [2025-10-20T17:36:09.056524] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-08. [2025-10-20T17:36:09.115844] [WARN ] [MAIN ] BKT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:09.116102] [INFO ] [MAIN ] BKT: Wiping existing primary data before full refresh. [2025-10-20T17:36:09.116150] [INFO ] [LOCAL ] BKT: Starting deletion of primary data. [2025-10-20T17:36:09.116360] [DEBUG] [LOCAL ] BKT: Deleted primary data file. [2025-10-20T17:36:09.116414] [INFO ] [MAIN ] BKT: Starting full primary refresh. [2025-10-20T17:36:09.480295] [INFO ] [REMOTE ] BKT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:09.851406] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-01. [2025-10-20T17:36:10.201301] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T17:36:10.396483] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T17:36:10.599907] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T17:36:10.651381] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-09. [2025-10-20T17:36:11.451392] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-09. [2025-10-20T17:36:11.672023] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:36:12.251469] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-02. [2025-10-20T17:36:13.051435] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-10. [2025-10-20T17:36:13.075750] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T17:36:13.591000] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-10-20T17:36:13.851411] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-10. [2025-10-20T17:36:14.543226] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:36:14.549670] [INFO ] [REMOTE ] BKH: Full fetch complete (52726 records). [2025-10-20T17:36:14.549864] [DEBUG] [VALIDATOR ] BKH: Validating primary data. [2025-10-20T17:36:14.550805] [FATAL] [VALIDATOR ] BKH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:36:14.550900] [FATAL] [MAIN ] BKH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:36:14.550946] [WARN ] [MAIN ] BKH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:36:14.550995] [INFO ] [LOCAL ] BKH: Starting deletion of primary data. [2025-10-20T17:36:14.551059] [DEBUG] [LOCAL ] BKH: No primary data file found to delete. [2025-10-20T17:36:14.551093] [INFO ] [LOCAL ] BKH: Starting deletion of derivative data. [2025-10-20T17:36:14.551276] [DEBUG] [LOCAL ] BKH: Deleted derivative data file. [2025-10-20T17:36:14.551380] [INFO ] [MAIN ] Update Progress: 290/2367 symbols completed. BKH took 35.0s. Estimated remaining: 4h 19m 3s. Rate: 8.0 symbols/min [2025-10-20T17:36:14.657094] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-03. [2025-10-20T17:36:14.741345] [INFO ] [MAIN ] BKU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:14.787369] [DEBUG] [LOCAL ] BKU: Loaded 42897 existing records from primary storage. [2025-10-20T17:36:14.787962] [INFO ] [MAIN ] BKU: Loaded 42897 existing records. [2025-10-20T17:36:15.000143] [WARN ] [MAIN ] BKU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:15.002049] [INFO ] [MAIN ] BKU: Wiping existing primary data before full refresh. [2025-10-20T17:36:15.002139] [INFO ] [LOCAL ] BKU: Starting deletion of primary data. [2025-10-20T17:36:15.003157] [DEBUG] [LOCAL ] BKU: Deleted primary data file. [2025-10-20T17:36:15.003219] [INFO ] [MAIN ] BKU: Starting full primary refresh. [2025-10-20T17:36:15.382208] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T17:36:15.415321] [INFO ] [REMOTE ] BKU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:15.451443] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-01. [2025-10-20T17:36:15.867123] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-10 (2.02 seconds). [2025-10-20T17:36:15.878244] [INFO ] [REMOTE ] BKKT: Full fetch complete (51700 records). [2025-10-20T17:36:15.878499] [DEBUG] [VALIDATOR ] BKKT: Validating primary data. [2025-10-20T17:36:16.256793] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-04. [2025-10-20T17:36:16.302444] [DEBUG] [VALIDATOR ] BKKT: All expected trading days are present. [2025-10-20T17:36:16.302676] [DEBUG] [VALIDATOR ] BKKT: Primary data validation passed. [2025-10-20T17:36:16.303609] [INFO ] [MAIN ] BKKT: Fetched primary data is valid. [2025-10-20T17:36:16.303708] [DEBUG] [LOCAL ] BKKT: Saving primary data. [2025-10-20T17:36:16.305376] [DEBUG] [LOCAL ] BKKT: Trimmed 51700 records to 37052 (keeping 200 days of data) [2025-10-20T17:36:16.538896] [DEBUG] [LOCAL ] BKKT: Primary data saved (37052 records). [2025-10-20T17:36:16.539089] [INFO ] [MAIN ] BKKT: Full history saved. [2025-10-20T17:36:16.539662] [INFO ] [MAIN ] Update Progress: 291/2367 symbols completed. BKKT took 35.6s. Estimated remaining: 4h 18m 16s. Rate: 8.0 symbols/min [2025-10-20T17:36:16.723369] [INFO ] [MAIN ] BKV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:16.757019] [DEBUG] [LOCAL ] BKV: Loaded 30117 existing records from primary storage. [2025-10-20T17:36:16.757372] [INFO ] [MAIN ] BKV: Loaded 30117 existing records. [2025-10-20T17:36:16.982092] [WARN ] [MAIN ] BKV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:16.982505] [INFO ] [MAIN ] BKV: Wiping existing primary data before full refresh. [2025-10-20T17:36:16.982592] [INFO ] [LOCAL ] BKV: Starting deletion of primary data. [2025-10-20T17:36:16.983089] [DEBUG] [LOCAL ] BKV: Deleted primary data file. [2025-10-20T17:36:16.983286] [INFO ] [MAIN ] BKV: Starting full primary refresh. [2025-10-20T17:36:17.056277] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-06. [2025-10-20T17:36:17.122078] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-10-20T17:36:17.250960] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:36:17.454367] [INFO ] [REMOTE ] BKV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:17.851405] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-02. [2025-10-20T17:36:18.651392] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-05. [2025-10-20T17:36:18.954972] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T17:36:19.305050] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T17:36:19.353495] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:36:19.451388] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-03. [2025-10-20T17:36:20.251384] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-06. [2025-10-20T17:36:21.051385] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-07. [2025-10-20T17:36:21.108125] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T17:36:21.108475] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T17:36:21.851384] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-04. [2025-10-20T17:36:22.651413] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-07. [2025-10-20T17:36:23.184836] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T17:36:23.410608] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-10-20T17:36:23.453862] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-01. [2025-10-20T17:36:23.460786] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:36:24.251448] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-05. [2025-10-20T17:36:24.913979] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T17:36:25.051413] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-02. [2025-10-20T17:36:25.646687] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:36:25.851362] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-06. [2025-10-20T17:36:26.359121] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T17:36:26.651411] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-08. [2025-10-20T17:36:27.061421] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:36:27.212901] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T17:36:27.451392] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-07. [2025-10-20T17:36:28.251365] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-08. [2025-10-20T17:36:28.977172] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T17:36:29.051383] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-08. [2025-10-20T17:36:29.803784] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T17:36:29.851385] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-09. [2025-10-20T17:36:30.007086] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:36:30.651371] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-09. [2025-10-20T17:36:31.451363] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-03. [2025-10-20T17:36:31.857365] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T17:36:32.103121] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T17:36:32.251366] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-10. [2025-10-20T17:36:32.648238] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T17:36:33.051374] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-09. [2025-10-20T17:36:33.525427] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T17:36:33.532279] [INFO ] [REMOTE ] BKU: Full fetch complete (60914 records). [2025-10-20T17:36:33.532427] [DEBUG] [VALIDATOR ] BKU: Validating primary data. [2025-10-20T17:36:33.854966] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-10. [2025-10-20T17:36:33.931625] [DEBUG] [VALIDATOR ] BKU: All expected trading days are present. [2025-10-20T17:36:33.931735] [DEBUG] [VALIDATOR ] BKU: Primary data validation passed. [2025-10-20T17:36:33.932380] [INFO ] [MAIN ] BKU: Fetched primary data is valid. [2025-10-20T17:36:33.932420] [DEBUG] [LOCAL ] BKU: Saving primary data. [2025-10-20T17:36:33.933591] [DEBUG] [LOCAL ] BKU: Trimmed 60914 records to 42263 (keeping 200 days of data) [2025-10-20T17:36:34.166547] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T17:36:34.205410] [DEBUG] [LOCAL ] BKU: Primary data saved (42263 records). [2025-10-20T17:36:34.205551] [INFO ] [MAIN ] BKU: Full history saved. [2025-10-20T17:36:34.205629] [INFO ] [MAIN ] Update Progress: 292/2367 symbols completed. BKU took 19.7s. Estimated remaining: 4h 19m 21s. Rate: 8.0 symbols/min [2025-10-20T17:36:34.371864] [INFO ] [MAIN ] BLCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:34.404484] [DEBUG] [LOCAL ] BLCO: Loaded 35655 existing records from primary storage. [2025-10-20T17:36:34.404784] [INFO ] [MAIN ] BLCO: Loaded 35655 existing records. [2025-10-20T17:36:34.580795] [WARN ] [MAIN ] BLCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:34.581014] [INFO ] [MAIN ] BLCO: Wiping existing primary data before full refresh. [2025-10-20T17:36:34.581060] [INFO ] [LOCAL ] BLCO: Starting deletion of primary data. [2025-10-20T17:36:34.581243] [DEBUG] [LOCAL ] BLCO: Deleted primary data file. [2025-10-20T17:36:34.581281] [INFO ] [MAIN ] BLCO: Starting full primary refresh. [2025-10-20T17:36:34.656470] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-04. [2025-10-20T17:36:35.027399] [INFO ] [REMOTE ] BLCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:35.451453] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-10. [2025-10-20T17:36:35.776034] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T17:36:36.063117] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T17:36:36.071533] [INFO ] [REMOTE ] BKSY: Full fetch complete (70525 records). [2025-10-20T17:36:36.071779] [DEBUG] [VALIDATOR ] BKSY: Validating primary data. [2025-10-20T17:36:36.251396] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-05. [2025-10-20T17:36:36.488992] [DEBUG] [VALIDATOR ] BKSY: All expected trading days are present. [2025-10-20T17:36:36.489079] [DEBUG] [VALIDATOR ] BKSY: Primary data validation passed. [2025-10-20T17:36:36.489849] [INFO ] [MAIN ] BKSY: Fetched primary data is valid. [2025-10-20T17:36:36.489912] [DEBUG] [LOCAL ] BKSY: Saving primary data. [2025-10-20T17:36:36.491379] [DEBUG] [LOCAL ] BKSY: Trimmed 70525 records to 48890 (keeping 200 days of data) [2025-10-20T17:36:36.712052] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T17:36:36.728606] [INFO ] [REMOTE ] BKT: Full fetch complete (16574 records). [2025-10-20T17:36:36.740234] [DEBUG] [VALIDATOR ] BKT: Validating primary data. [2025-10-20T17:36:36.912284] [DEBUG] [LOCAL ] BKSY: Primary data saved (48890 records). [2025-10-20T17:36:36.912797] [INFO ] [MAIN ] BKSY: Full history saved. [2025-10-20T17:36:36.918414] [INFO ] [MAIN ] Update Progress: 293/2367 symbols completed. BKSY took 42.7s. Estimated remaining: 4h 18m 40s. Rate: 8.0 symbols/min [2025-10-20T17:36:37.074381] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-01. [2025-10-20T17:36:37.290876] [INFO ] [MAIN ] BLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:37.379981] [DEBUG] [VALIDATOR ] BKT: All expected trading days are present. [2025-10-20T17:36:37.380075] [DEBUG] [VALIDATOR ] BKT: Primary data validation passed. [2025-10-20T17:36:37.380438] [INFO ] [MAIN ] BKT: Fetched primary data is valid. [2025-10-20T17:36:37.380476] [DEBUG] [LOCAL ] BKT: Saving primary data. [2025-10-20T17:36:37.381275] [DEBUG] [LOCAL ] BKT: Trimmed 16574 records to 12778 (keeping 200 days of data) [2025-10-20T17:36:37.484409] [DEBUG] [LOCAL ] BLD: Loaded 27717 existing records from primary storage. [2025-10-20T17:36:37.484855] [DEBUG] [LOCAL ] BKT: Primary data saved (12778 records). [2025-10-20T17:36:37.485517] [INFO ] [MAIN ] BLD: Loaded 27717 existing records. [2025-10-20T17:36:37.485191] [INFO ] [MAIN ] BKT: Full history saved. [2025-10-20T17:36:37.517759] [INFO ] [MAIN ] Update Progress: 294/2367 symbols completed. BKT took 28.8s. Estimated remaining: 4h 17m 44s. Rate: 8.0 symbols/min [2025-10-20T17:36:37.825457] [WARN ] [MAIN ] BLD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:37.853668] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T17:36:37.850721] [INFO ] [MAIN ] BLD: Wiping existing primary data before full refresh. [2025-10-20T17:36:37.877750] [INFO ] [LOCAL ] BLD: Starting deletion of primary data. [2025-10-20T17:36:37.902162] [DEBUG] [LOCAL ] BLD: Deleted primary data file. [2025-10-20T17:36:37.901323] [INFO ] [MAIN ] BLDR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:37.902293] [INFO ] [MAIN ] BLD: Starting full primary refresh. [2025-10-20T17:36:37.903005] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-06. [2025-10-20T17:36:38.277369] [INFO ] [REMOTE ] BLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:38.295071] [DEBUG] [LOCAL ] BLDR: Loaded 52629 existing records from primary storage. [2025-10-20T17:36:38.295436] [INFO ] [MAIN ] BLDR: Loaded 52629 existing records. [2025-10-20T17:36:38.488814] [WARN ] [MAIN ] BLDR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:38.489188] [INFO ] [MAIN ] BLDR: Wiping existing primary data before full refresh. [2025-10-20T17:36:38.489367] [INFO ] [LOCAL ] BLDR: Starting deletion of primary data. [2025-10-20T17:36:38.489723] [DEBUG] [LOCAL ] BLDR: Deleted primary data file. [2025-10-20T17:36:38.505623] [INFO ] [MAIN ] BLDR: Starting full primary refresh. [2025-10-20T17:36:38.557491] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T17:36:38.656624] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-01. [2025-10-20T17:36:38.913003] [INFO ] [REMOTE ] BLDR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:39.354330] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T17:36:39.451400] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-01. [2025-10-20T17:36:39.867096] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:36:40.251391] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-02. [2025-10-20T17:36:41.051425] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-02. [2025-10-20T17:36:41.093033] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T17:36:41.288251] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T17:36:41.851387] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-02. [2025-10-20T17:36:42.453858] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T17:36:42.651373] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-03. [2025-10-20T17:36:43.265255] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T17:36:43.451438] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-03. [2025-10-20T17:36:43.693158] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T17:36:44.251432] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-07. [2025-10-20T17:36:45.051431] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-03. [2025-10-20T17:36:45.282176] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T17:36:45.488367] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T17:36:45.851403] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-04. [2025-10-20T17:36:46.331532] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T17:36:46.651483] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-08. [2025-10-20T17:36:47.242018] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:36:47.451381] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-05. [2025-10-20T17:36:47.500134] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T17:36:48.251401] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-04. [2025-10-20T17:36:49.051365] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-09. [2025-10-20T17:36:49.173622] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:36:49.851366] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-06. [2025-10-20T17:36:50.290520] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T17:36:50.344539] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T17:36:50.651428] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-10. [2025-10-20T17:36:51.451401] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-05. [2025-10-20T17:36:51.473253] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:36:52.081067] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T17:36:52.086594] [INFO ] [REMOTE ] BKV: Full fetch complete (44924 records). [2025-10-20T17:36:52.086774] [DEBUG] [VALIDATOR ] BKV: Validating primary data. [2025-10-20T17:36:52.253305] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-04. [2025-10-20T17:36:52.462344] [DEBUG] [VALIDATOR ] BKV: All expected trading days are present. [2025-10-20T17:36:52.462436] [DEBUG] [VALIDATOR ] BKV: Primary data validation passed. [2025-10-20T17:36:52.462986] [INFO ] [MAIN ] BKV: Fetched primary data is valid. [2025-10-20T17:36:52.463054] [DEBUG] [LOCAL ] BKV: Saving primary data. [2025-10-20T17:36:52.464199] [DEBUG] [LOCAL ] BKV: Trimmed 44924 records to 29615 (keeping 200 days of data) [2025-10-20T17:36:52.633984] [DEBUG] [LOCAL ] BKV: Primary data saved (29615 records). [2025-10-20T17:36:52.634127] [INFO ] [MAIN ] BKV: Full history saved. [2025-10-20T17:36:52.634226] [INFO ] [MAIN ] Update Progress: 295/2367 symbols completed. BKV took 36.1s. Estimated remaining: 4h 18m 30s. Rate: 8.0 symbols/min [2025-10-20T17:36:52.801060] [INFO ] [MAIN ] BLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:52.814871] [DEBUG] [LOCAL ] BLE: Loaded 13930 existing records from primary storage. [2025-10-20T17:36:52.815083] [INFO ] [MAIN ] BLE: Loaded 13930 existing records. [2025-10-20T17:36:52.988111] [WARN ] [MAIN ] BLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:52.988353] [INFO ] [MAIN ] BLE: Wiping existing primary data before full refresh. [2025-10-20T17:36:52.988409] [INFO ] [LOCAL ] BLE: Starting deletion of primary data. [2025-10-20T17:36:52.988601] [DEBUG] [LOCAL ] BLE: Deleted primary data file. [2025-10-20T17:36:52.988638] [INFO ] [MAIN ] BLE: Starting full primary refresh. [2025-10-20T17:36:53.062050] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-07. [2025-10-20T17:36:53.346138] [INFO ] [REMOTE ] BLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:53.445155] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T17:36:53.683214] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T17:36:53.851418] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-05. [2025-10-20T17:36:54.226191] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:36:54.651426] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-06. [2025-10-20T17:36:55.067875] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T17:36:55.451412] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-01. [2025-10-20T17:36:56.251368] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-06. [2025-10-20T17:36:56.429931] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:36:56.460333] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T17:36:57.051370] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-08. [2025-10-20T17:36:57.586627] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T17:36:57.851428] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-07. [2025-10-20T17:36:58.330839] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T17:36:58.651381] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-02. [2025-10-20T17:36:59.392676] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T17:36:59.451343] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-03. [2025-10-20T17:37:00.066116] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-07 (2.21 seconds). [2025-10-20T17:37:00.251370] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-09. [2025-10-20T17:37:00.400676] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:37:01.051411] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-07. [2025-10-20T17:37:01.382076] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T17:37:01.851350] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-10. [2025-10-20T17:37:02.075600] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T17:37:02.651421] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-04. [2025-10-20T17:37:02.847210] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:37:02.852007] [INFO ] [REMOTE ] BLCO: Full fetch complete (52342 records). [2025-10-20T17:37:02.852099] [DEBUG] [VALIDATOR ] BLCO: Validating primary data. [2025-10-20T17:37:03.248765] [DEBUG] [VALIDATOR ] BLCO: All expected trading days are present. [2025-10-20T17:37:03.248842] [DEBUG] [VALIDATOR ] BLCO: Primary data validation passed. [2025-10-20T17:37:03.249438] [INFO ] [MAIN ] BLCO: Fetched primary data is valid. [2025-10-20T17:37:03.249477] [DEBUG] [LOCAL ] BLCO: Saving primary data. [2025-10-20T17:37:03.251101] [DEBUG] [LOCAL ] BLCO: Trimmed 52342 records to 35097 (keeping 200 days of data) [2025-10-20T17:37:03.448041] [DEBUG] [LOCAL ] BLCO: Primary data saved (35097 records). [2025-10-20T17:37:03.448218] [INFO ] [MAIN ] BLCO: Full history saved. [2025-10-20T17:37:03.448340] [INFO ] [MAIN ] Update Progress: 296/2367 symbols completed. BLCO took 29.2s. Estimated remaining: 4h 18m 46s. Rate: 8.0 symbols/min [2025-10-20T17:37:03.459378] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-08. [2025-10-20T17:37:03.570614] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T17:37:03.641839] [INFO ] [MAIN ] BLK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:03.673009] [DEBUG] [LOCAL ] BLK: Loaded 34882 existing records from primary storage. [2025-10-20T17:37:03.673282] [INFO ] [MAIN ] BLK: Loaded 34882 existing records. [2025-10-20T17:37:03.846745] [WARN ] [MAIN ] BLK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:03.846970] [INFO ] [MAIN ] BLK: Wiping existing primary data before full refresh. [2025-10-20T17:37:03.847017] [INFO ] [LOCAL ] BLK: Starting deletion of primary data. [2025-10-20T17:37:03.847227] [DEBUG] [LOCAL ] BLK: Deleted primary data file. [2025-10-20T17:37:03.847264] [INFO ] [MAIN ] BLK: Starting full primary refresh. [2025-10-20T17:37:04.193245] [INFO ] [REMOTE ] BLK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:04.251326] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-01. [2025-10-20T17:37:05.051421] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-05. [2025-10-20T17:37:05.652187] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-08 (2.19 seconds). [2025-10-20T17:37:05.851412] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-09. [2025-10-20T17:37:05.865537] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T17:37:06.130718] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T17:37:06.651444] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-06. [2025-10-20T17:37:07.451345] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-02. [2025-10-20T17:37:07.502823] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:37:07.767012] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T17:37:08.251409] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-10. [2025-10-20T17:37:09.056534] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-08. [2025-10-20T17:37:09.087238] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T17:37:09.851371] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-03. [2025-10-20T17:37:09.939970] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:37:09.948622] [INFO ] [REMOTE ] BLDR: Full fetch complete (74700 records). [2025-10-20T17:37:09.948840] [DEBUG] [VALIDATOR ] BLDR: Validating primary data. [2025-10-20T17:37:10.369022] [DEBUG] [VALIDATOR ] BLDR: All expected trading days are present. [2025-10-20T17:37:10.370499] [DEBUG] [VALIDATOR ] BLDR: Primary data validation passed. [2025-10-20T17:37:10.371444] [INFO ] [MAIN ] BLDR: Fetched primary data is valid. [2025-10-20T17:37:10.371528] [DEBUG] [LOCAL ] BLDR: Saving primary data. [2025-10-20T17:37:10.370408] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:37:10.379755] [DEBUG] [LOCAL ] BLDR: Trimmed 74700 records to 51892 (keeping 200 days of data) [2025-10-20T17:37:10.668413] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-07. [2025-10-20T17:37:10.691333] [DEBUG] [LOCAL ] BLDR: Primary data saved (51892 records). [2025-10-20T17:37:10.691545] [INFO ] [MAIN ] BLDR: Full history saved. [2025-10-20T17:37:10.691635] [INFO ] [MAIN ] Update Progress: 297/2367 symbols completed. BLDR took 33.1s. Estimated remaining: 4h 18m 37s. Rate: 8.0 symbols/min [2025-10-20T17:37:10.858207] [INFO ] [MAIN ] BLND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:10.900221] [DEBUG] [LOCAL ] BLND: Loaded 50014 existing records from primary storage. [2025-10-20T17:37:10.900516] [INFO ] [MAIN ] BLND: Loaded 50014 existing records. [2025-10-20T17:37:11.072883] [WARN ] [MAIN ] BLND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:11.073129] [INFO ] [MAIN ] BLND: Wiping existing primary data before full refresh. [2025-10-20T17:37:11.073176] [INFO ] [LOCAL ] BLND: Starting deletion of primary data. [2025-10-20T17:37:11.073585] [DEBUG] [LOCAL ] BLND: Deleted primary data file. [2025-10-20T17:37:11.073623] [INFO ] [MAIN ] BLND: Starting full primary refresh. [2025-10-20T17:37:11.390784] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T17:37:11.455064] [INFO ] [REMOTE ] BLND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:11.455135] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-09. [2025-10-20T17:37:11.855209] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T17:37:12.251348] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-08. [2025-10-20T17:37:12.329797] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T17:37:12.718037] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-10-20T17:37:13.051390] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-01. [2025-10-20T17:37:13.851374] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-04. [2025-10-20T17:37:14.651400] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-10. [2025-10-20T17:37:14.878975] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:37:15.451369] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-02. [2025-10-20T17:37:15.501853] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T17:37:15.539131] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T17:37:15.542881] [INFO ] [REMOTE ] BLD: Full fetch complete (39851 records). [2025-10-20T17:37:15.542966] [DEBUG] [VALIDATOR ] BLD: Validating primary data. [2025-10-20T17:37:15.918537] [DEBUG] [VALIDATOR ] BLD: All expected trading days are present. [2025-10-20T17:37:15.918637] [DEBUG] [VALIDATOR ] BLD: Primary data validation passed. [2025-10-20T17:37:15.919164] [INFO ] [MAIN ] BLD: Fetched primary data is valid. [2025-10-20T17:37:15.919206] [DEBUG] [LOCAL ] BLD: Saving primary data. [2025-10-20T17:37:15.920617] [DEBUG] [LOCAL ] BLD: Trimmed 39851 records to 27298 (keeping 200 days of data) [2025-10-20T17:37:16.081876] [DEBUG] [LOCAL ] BLD: Primary data saved (27298 records). [2025-10-20T17:37:16.082008] [INFO ] [MAIN ] BLD: Full history saved. [2025-10-20T17:37:16.082089] [INFO ] [MAIN ] Update Progress: 298/2367 symbols completed. BLD took 39.2s. Estimated remaining: 4h 18m 14s. Rate: 8.0 symbols/min [2025-10-20T17:37:16.253856] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-05. [2025-10-20T17:37:16.253733] [INFO ] [MAIN ] BLSH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:16.255114] [DEBUG] [LOCAL ] BLSH: No existing data file found in primary storage. [2025-10-20T17:37:16.255187] [WARN ] [MAIN ] BLSH: No existing data found, performing full refresh instead. [2025-10-20T17:37:16.255267] [INFO ] [MAIN ] BLSH: Wiping existing primary data before full refresh. [2025-10-20T17:37:16.255303] [INFO ] [LOCAL ] BLSH: Starting deletion of primary data. [2025-10-20T17:37:16.255345] [DEBUG] [LOCAL ] BLSH: No primary data file found to delete. [2025-10-20T17:37:16.255379] [INFO ] [MAIN ] BLSH: Starting full primary refresh. [2025-10-20T17:37:16.622137] [INFO ] [REMOTE ] BLSH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:16.866902] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:37:17.051348] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-03. [2025-10-20T17:37:17.692534] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T17:37:17.851391] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-06. [2025-10-20T17:37:18.402912] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T17:37:18.651394] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-01. [2025-10-20T17:37:18.838591] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:37:18.838930] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-01 [2025-10-20T17:37:18.839022] [FATAL] [REMOTE ] BLSH: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:37:18.839069] [FATAL] [MAIN ] BLSH: Failed to fetch primary data. [2025-10-20T17:37:18.839102] [WARN ] [MAIN ] BLSH: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:37:18.839165] [INFO ] [LOCAL ] BLSH: Starting deletion of primary data. [2025-10-20T17:37:18.839226] [DEBUG] [LOCAL ] BLSH: No primary data file found to delete. [2025-10-20T17:37:18.839260] [INFO ] [LOCAL ] BLSH: Starting deletion of derivative data. [2025-10-20T17:37:18.839298] [DEBUG] [LOCAL ] BLSH: No derivative data file found to delete. [2025-10-20T17:37:18.839344] [INFO ] [MAIN ] Update Progress: 299/2367 symbols completed. BLSH took 2.8s. Estimated remaining: 4h 17m 34s. Rate: 8.0 symbols/min [2025-10-20T17:37:19.012444] [INFO ] [MAIN ] BLW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:19.028028] [DEBUG] [LOCAL ] BLW: Loaded 13364 existing records from primary storage. [2025-10-20T17:37:19.028269] [INFO ] [MAIN ] BLW: Loaded 13364 existing records. [2025-10-20T17:37:19.207277] [WARN ] [MAIN ] BLW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:19.207580] [INFO ] [MAIN ] BLW: Wiping existing primary data before full refresh. [2025-10-20T17:37:19.207696] [INFO ] [LOCAL ] BLW: Starting deletion of primary data. [2025-10-20T17:37:19.207951] [DEBUG] [LOCAL ] BLW: Deleted primary data file. [2025-10-20T17:37:19.208008] [INFO ] [MAIN ] BLW: Starting full primary refresh. [2025-10-20T17:37:19.450684] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T17:37:19.457193] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-04. [2025-10-20T17:37:19.636607] [INFO ] [REMOTE ] BLW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:20.251419] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-07. [2025-10-20T17:37:21.051356] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-09. [2025-10-20T17:37:21.179677] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T17:37:21.851414] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-01. [2025-10-20T17:37:21.990433] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T17:37:22.060186] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T17:37:22.651396] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-10. [2025-10-20T17:37:22.781498] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T17:37:23.451361] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-08. [2025-10-20T17:37:23.489890] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T17:37:23.491984] [INFO ] [REMOTE ] BLE: Full fetch complete (19600 records). [2025-10-20T17:37:23.492051] [DEBUG] [VALIDATOR ] BLE: Validating primary data. [2025-10-20T17:37:23.856217] [DEBUG] [VALIDATOR ] BLE: All expected trading days are present. [2025-10-20T17:37:23.856298] [DEBUG] [VALIDATOR ] BLE: Primary data validation passed. [2025-10-20T17:37:23.856627] [INFO ] [MAIN ] BLE: Fetched primary data is valid. [2025-10-20T17:37:23.856697] [DEBUG] [LOCAL ] BLE: Saving primary data. [2025-10-20T17:37:23.857229] [DEBUG] [LOCAL ] BLE: Trimmed 19600 records to 13804 (keeping 200 days of data) [2025-10-20T17:37:23.935942] [DEBUG] [LOCAL ] BLE: Primary data saved (13804 records). [2025-10-20T17:37:23.936072] [INFO ] [MAIN ] BLE: Full history saved. [2025-10-20T17:37:23.936150] [INFO ] [MAIN ] Update Progress: 300/2367 symbols completed. BLE took 31.3s. Estimated remaining: 4h 17m 10s. Rate: 8.0 symbols/min [2025-10-20T17:37:24.101894] [INFO ] [MAIN ] BLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:24.118027] [DEBUG] [LOCAL ] BLX: Loaded 15775 existing records from primary storage. [2025-10-20T17:37:24.118272] [INFO ] [MAIN ] BLX: Loaded 15775 existing records. [2025-10-20T17:37:24.256459] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-05. [2025-10-20T17:37:24.298976] [WARN ] [MAIN ] BLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:24.299159] [INFO ] [MAIN ] BLX: Wiping existing primary data before full refresh. [2025-10-20T17:37:24.299215] [INFO ] [LOCAL ] BLX: Starting deletion of primary data. [2025-10-20T17:37:24.299405] [DEBUG] [LOCAL ] BLX: Deleted primary data file. [2025-10-20T17:37:24.299463] [INFO ] [MAIN ] BLX: Starting full primary refresh. [2025-10-20T17:37:24.653309] [INFO ] [REMOTE ] BLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:25.007202] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T17:37:25.051340] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-09. [2025-10-20T17:37:25.838259] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T17:37:25.851319] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-06. [2025-10-20T17:37:26.647501] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:37:26.653930] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-02. [2025-10-20T17:37:27.062292] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:37:27.183396] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T17:37:27.451425] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-03. [2025-10-20T17:37:28.251443] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-10. [2025-10-20T17:37:28.380481] [DEBUG] [REMOTE ] BLK: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T17:37:28.380918] [WARN ] [REMOTE ] BLK: No data returned for month 2025-10 [2025-10-20T17:37:28.381042] [FATAL] [REMOTE ] BLK: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:37:28.381149] [FATAL] [MAIN ] BLK: Failed to fetch primary data. [2025-10-20T17:37:28.381232] [WARN ] [MAIN ] BLK: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:37:28.381323] [INFO ] [LOCAL ] BLK: Starting deletion of primary data. [2025-10-20T17:37:28.381414] [DEBUG] [LOCAL ] BLK: No primary data file found to delete. [2025-10-20T17:37:28.381474] [INFO ] [LOCAL ] BLK: Starting deletion of derivative data. [2025-10-20T17:37:28.381715] [DEBUG] [LOCAL ] BLK: Deleted derivative data file. [2025-10-20T17:37:28.381824] [INFO ] [MAIN ] Update Progress: 301/2367 symbols completed. BLK took 24.9s. Estimated remaining: 4h 16m 42s. Rate: 8.0 symbols/min [2025-10-20T17:37:28.487275] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T17:37:28.639265] [INFO ] [MAIN ] BMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:28.692433] [DEBUG] [LOCAL ] BMA: Loaded 36733 existing records from primary storage. [2025-10-20T17:37:28.692922] [INFO ] [MAIN ] BMA: Loaded 36733 existing records. [2025-10-20T17:37:28.925042] [WARN ] [MAIN ] BMA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:28.925360] [INFO ] [MAIN ] BMA: Wiping existing primary data before full refresh. [2025-10-20T17:37:28.925431] [INFO ] [LOCAL ] BMA: Starting deletion of primary data. [2025-10-20T17:37:28.925737] [DEBUG] [LOCAL ] BMA: Deleted primary data file. [2025-10-20T17:37:28.925809] [INFO ] [MAIN ] BMA: Starting full primary refresh. [2025-10-20T17:37:29.056569] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-07. [2025-10-20T17:37:29.440131] [INFO ] [REMOTE ] BMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:29.851393] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-01. [2025-10-20T17:37:30.651377] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-01. [2025-10-20T17:37:30.763563] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T17:37:31.215630] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T17:37:31.455817] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-08. [2025-10-20T17:37:31.459228] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T17:37:32.251397] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-02. [2025-10-20T17:37:33.051455] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-04. [2025-10-20T17:37:33.118369] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T17:37:33.160473] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T17:37:33.851439] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-02. [2025-10-20T17:37:33.889971] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T17:37:34.651396] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-05. [2025-10-20T17:37:35.016066] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:37:35.109303] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T17:37:35.451402] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-03. [2025-10-20T17:37:36.251392] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-09. [2025-10-20T17:37:36.401380] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:37:37.051394] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-06. [2025-10-20T17:37:37.851388] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-04. [2025-10-20T17:37:37.917175] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T17:37:38.169483] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:37:38.651418] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-03. [2025-10-20T17:37:39.272006] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T17:37:39.451409] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-05. [2025-10-20T17:37:39.577279] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T17:37:40.251362] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-04. [2025-10-20T17:37:40.518616] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:37:41.051367] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-06. [2025-10-20T17:37:41.341204] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T17:37:41.851393] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-05. [2025-10-20T17:37:42.118698] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T17:37:42.598561] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T17:37:42.651402] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-10. [2025-10-20T17:37:43.451373] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-06. [2025-10-20T17:37:44.044945] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:37:44.052272] [INFO ] [REMOTE ] BLND: Full fetch complete (72274 records). [2025-10-20T17:37:44.052419] [DEBUG] [VALIDATOR ] BLND: Validating primary data. [2025-10-20T17:37:44.261347] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-07. [2025-10-20T17:37:44.452284] [DEBUG] [VALIDATOR ] BLND: All expected trading days are present. [2025-10-20T17:37:44.457526] [DEBUG] [VALIDATOR ] BLND: Primary data validation passed. [2025-10-20T17:37:44.458626] [INFO ] [MAIN ] BLND: Fetched primary data is valid. [2025-10-20T17:37:44.458744] [DEBUG] [LOCAL ] BLND: Saving primary data. [2025-10-20T17:37:44.460394] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:37:44.465770] [DEBUG] [LOCAL ] BLND: Trimmed 72274 records to 49271 (keeping 200 days of data) [2025-10-20T17:37:44.731191] [DEBUG] [LOCAL ] BLND: Primary data saved (49271 records). [2025-10-20T17:37:44.731342] [INFO ] [MAIN ] BLND: Full history saved. [2025-10-20T17:37:44.731434] [INFO ] [MAIN ] Update Progress: 302/2367 symbols completed. BLND took 34.0s. Estimated remaining: 4h 17m 36s. Rate: 8.0 symbols/min [2025-10-20T17:37:44.896736] [INFO ] [MAIN ] BME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:44.908272] [DEBUG] [LOCAL ] BME: Loaded 11421 existing records from primary storage. [2025-10-20T17:37:44.908475] [INFO ] [MAIN ] BME: Loaded 11421 existing records. [2025-10-20T17:37:45.056513] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-07. [2025-10-20T17:37:45.084379] [WARN ] [MAIN ] BME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:45.084799] [INFO ] [MAIN ] BME: Wiping existing primary data before full refresh. [2025-10-20T17:37:45.084857] [INFO ] [LOCAL ] BME: Starting deletion of primary data. [2025-10-20T17:37:45.085016] [DEBUG] [LOCAL ] BME: Deleted primary data file. [2025-10-20T17:37:45.085056] [INFO ] [MAIN ] BME: Starting full primary refresh. [2025-10-20T17:37:45.207886] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:37:45.456471] [INFO ] [REMOTE ] BME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:45.851424] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-01. [2025-10-20T17:37:46.229490] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T17:37:46.651357] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-08. [2025-10-20T17:37:46.892880] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:37:47.451340] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-02. [2025-10-20T17:37:47.533370] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T17:37:48.198446] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T17:37:48.251367] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-09. [2025-10-20T17:37:49.051341] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-03. [2025-10-20T17:37:49.349639] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T17:37:49.835720] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T17:37:49.851339] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-04. [2025-10-20T17:37:50.651382] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-10. [2025-10-20T17:37:50.728227] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T17:37:51.451372] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-05. [2025-10-20T17:37:51.619431] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T17:37:51.622217] [INFO ] [REMOTE ] BLW: Full fetch complete (19198 records). [2025-10-20T17:37:51.622302] [DEBUG] [VALIDATOR ] BLW: Validating primary data. [2025-10-20T17:37:51.989295] [DEBUG] [VALIDATOR ] BLW: All expected trading days are present. [2025-10-20T17:37:51.989375] [DEBUG] [VALIDATOR ] BLW: Primary data validation passed. [2025-10-20T17:37:51.989738] [INFO ] [MAIN ] BLW: Fetched primary data is valid. [2025-10-20T17:37:51.989798] [DEBUG] [LOCAL ] BLW: Saving primary data. [2025-10-20T17:37:51.990334] [DEBUG] [LOCAL ] BLW: Trimmed 19198 records to 13124 (keeping 200 days of data) [2025-10-20T17:37:52.072464] [DEBUG] [LOCAL ] BLW: Primary data saved (13124 records). [2025-10-20T17:37:52.072701] [INFO ] [MAIN ] BLW: Full history saved. [2025-10-20T17:37:52.072792] [INFO ] [MAIN ] Update Progress: 303/2367 symbols completed. BLW took 33.2s. Estimated remaining: 4h 17m 27s. Rate: 8.0 symbols/min [2025-10-20T17:37:52.239545] [INFO ] [MAIN ] BMEZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:52.239715] [DEBUG] [LOCAL ] BMEZ: No existing data file found in primary storage. [2025-10-20T17:37:52.239763] [WARN ] [MAIN ] BMEZ: No existing data found, performing full refresh instead. [2025-10-20T17:37:52.239898] [INFO ] [MAIN ] BMEZ: Wiping existing primary data before full refresh. [2025-10-20T17:37:52.239938] [INFO ] [LOCAL ] BMEZ: Starting deletion of primary data. [2025-10-20T17:37:52.239978] [DEBUG] [LOCAL ] BMEZ: No primary data file found to delete. [2025-10-20T17:37:52.240009] [INFO ] [MAIN ] BMEZ: Starting full primary refresh. [2025-10-20T17:37:52.259942] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-07. [2025-10-20T17:37:52.351562] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T17:37:52.609202] [INFO ] [REMOTE ] BMEZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:53.051371] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-06. [2025-10-20T17:37:53.114290] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T17:37:53.851431] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-01. [2025-10-20T17:37:53.885428] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T17:37:54.651426] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-07. [2025-10-20T17:37:54.951453] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T17:37:55.451511] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-02. [2025-10-20T17:37:55.451987] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T17:37:56.251377] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-08. [2025-10-20T17:37:56.669095] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:37:57.051370] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-08. [2025-10-20T17:37:57.637178] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:37:57.665003] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T17:37:57.851372] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-09. [2025-10-20T17:37:58.651377] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-08. [2025-10-20T17:37:59.335613] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T17:37:59.451621] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-09. [2025-10-20T17:37:59.452726] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:38:00.252479] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-09. [2025-10-20T17:38:00.253485] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-10-20T17:38:00.864533] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T17:38:01.051383] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-03. [2025-10-20T17:38:01.851365] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-10. [2025-10-20T17:38:02.418500] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:38:02.651406] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-04. [2025-10-20T17:38:02.756795] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T17:38:02.763547] [INFO ] [REMOTE ] BME: Full fetch complete (15804 records). [2025-10-20T17:38:02.763745] [DEBUG] [VALIDATOR ] BME: Validating primary data. [2025-10-20T17:38:03.216193] [DEBUG] [VALIDATOR ] BME: All expected trading days are present. [2025-10-20T17:38:03.216288] [DEBUG] [VALIDATOR ] BME: Primary data validation passed. [2025-10-20T17:38:03.216851] [INFO ] [MAIN ] BME: Fetched primary data is valid. [2025-10-20T17:38:03.216928] [DEBUG] [LOCAL ] BME: Saving primary data. [2025-10-20T17:38:03.217759] [DEBUG] [LOCAL ] BME: Trimmed 15804 records to 11258 (keeping 200 days of data) [2025-10-20T17:38:03.287003] [DEBUG] [LOCAL ] BME: Primary data saved (11258 records). [2025-10-20T17:38:03.287140] [INFO ] [MAIN ] BME: Full history saved. [2025-10-20T17:38:03.287217] [INFO ] [MAIN ] Update Progress: 304/2367 symbols completed. BME took 18.6s. Estimated remaining: 4h 17m 45s. Rate: 8.0 symbols/min [2025-10-20T17:38:03.460370] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-10. [2025-10-20T17:38:03.465860] [INFO ] [MAIN ] BMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:03.490803] [DEBUG] [LOCAL ] BMI: Loaded 24957 existing records from primary storage. [2025-10-20T17:38:03.491075] [INFO ] [MAIN ] BMI: Loaded 24957 existing records. [2025-10-20T17:38:03.679706] [WARN ] [MAIN ] BMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:03.680952] [INFO ] [MAIN ] BMI: Wiping existing primary data before full refresh. [2025-10-20T17:38:03.681011] [INFO ] [LOCAL ] BMI: Starting deletion of primary data. [2025-10-20T17:38:03.681741] [DEBUG] [LOCAL ] BMI: Deleted primary data file. [2025-10-20T17:38:03.681800] [INFO ] [MAIN ] BMI: Starting full primary refresh. [2025-10-20T17:38:04.043258] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:38:04.063462] [INFO ] [REMOTE ] BMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:04.251370] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-01. [2025-10-20T17:38:04.510029] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:38:04.513479] [INFO ] [REMOTE ] BLX: Full fetch complete (22539 records). [2025-10-20T17:38:04.513582] [DEBUG] [VALIDATOR ] BLX: Validating primary data. [2025-10-20T17:38:04.926505] [DEBUG] [VALIDATOR ] BLX: All expected trading days are present. [2025-10-20T17:38:04.926613] [DEBUG] [VALIDATOR ] BLX: Primary data validation passed. [2025-10-20T17:38:04.927173] [INFO ] [MAIN ] BLX: Fetched primary data is valid. [2025-10-20T17:38:04.927233] [DEBUG] [LOCAL ] BLX: Saving primary data. [2025-10-20T17:38:04.928105] [DEBUG] [LOCAL ] BLX: Trimmed 22539 records to 15575 (keeping 200 days of data) [2025-10-20T17:38:05.038715] [DEBUG] [LOCAL ] BLX: Primary data saved (15575 records). [2025-10-20T17:38:05.038958] [INFO ] [MAIN ] BLX: Full history saved. [2025-10-20T17:38:05.039088] [INFO ] [MAIN ] Update Progress: 305/2367 symbols completed. BLX took 41.1s. Estimated remaining: 4h 16m 59s. Rate: 8.0 symbols/min [2025-10-20T17:38:05.062067] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-10. [2025-10-20T17:38:05.225905] [INFO ] [MAIN ] BMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:05.230535] [DEBUG] [LOCAL ] BMN: Loaded 2635 existing records from primary storage. [2025-10-20T17:38:05.230736] [INFO ] [MAIN ] BMN: Loaded 2635 existing records. [2025-10-20T17:38:05.439887] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:38:05.447387] [WARN ] [MAIN ] BMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:05.451805] [INFO ] [MAIN ] BMN: Wiping existing primary data before full refresh. [2025-10-20T17:38:05.452563] [INFO ] [LOCAL ] BMN: Starting deletion of primary data. [2025-10-20T17:38:05.452842] [DEBUG] [LOCAL ] BMN: Deleted primary data file. [2025-10-20T17:38:05.452902] [INFO ] [MAIN ] BMN: Starting full primary refresh. [2025-10-20T17:38:05.806909] [INFO ] [REMOTE ] BMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:05.851415] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-01. [2025-10-20T17:38:06.271038] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T17:38:06.589187] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T17:38:06.595545] [INFO ] [REMOTE ] BMA: Full fetch complete (52294 records). [2025-10-20T17:38:06.595735] [DEBUG] [VALIDATOR ] BMA: Validating primary data. [2025-10-20T17:38:06.658626] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-02. [2025-10-20T17:38:06.985215] [DEBUG] [VALIDATOR ] BMA: All expected trading days are present. [2025-10-20T17:38:06.985346] [DEBUG] [VALIDATOR ] BMA: Primary data validation passed. [2025-10-20T17:38:06.985974] [INFO ] [MAIN ] BMA: Fetched primary data is valid. [2025-10-20T17:38:06.986041] [DEBUG] [LOCAL ] BMA: Saving primary data. [2025-10-20T17:38:06.987221] [DEBUG] [LOCAL ] BMA: Trimmed 52294 records to 36470 (keeping 200 days of data) [2025-10-20T17:38:07.226858] [DEBUG] [LOCAL ] BMA: Primary data saved (36470 records). [2025-10-20T17:38:07.227072] [INFO ] [MAIN ] BMA: Full history saved. [2025-10-20T17:38:07.227179] [INFO ] [MAIN ] Update Progress: 306/2367 symbols completed. BMA took 38.8s. Estimated remaining: 4h 16m 15s. Rate: 8.0 symbols/min [2025-10-20T17:38:07.409428] [INFO ] [MAIN ] BMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:07.452841] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-05. [2025-10-20T17:38:07.455262] [DEBUG] [LOCAL ] BMO: Loaded 41935 existing records from primary storage. [2025-10-20T17:38:07.455826] [INFO ] [MAIN ] BMO: Loaded 41935 existing records. [2025-10-20T17:38:07.647807] [WARN ] [MAIN ] BMO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:07.648237] [INFO ] [MAIN ] BMO: Wiping existing primary data before full refresh. [2025-10-20T17:38:07.648363] [INFO ] [LOCAL ] BMO: Starting deletion of primary data. [2025-10-20T17:38:07.648670] [DEBUG] [LOCAL ] BMO: Deleted primary data file. [2025-10-20T17:38:07.648844] [INFO ] [MAIN ] BMO: Starting full primary refresh. [2025-10-20T17:38:07.761097] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:38:08.043701] [INFO ] [REMOTE ] BMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:08.251364] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-03. [2025-10-20T17:38:08.594132] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:38:09.051370] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-06. [2025-10-20T17:38:09.257588] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:38:09.851436] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-01. [2025-10-20T17:38:10.114572] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T17:38:10.651437] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-07. [2025-10-20T17:38:11.157598] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T17:38:11.451371] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-02. [2025-10-20T17:38:11.602766] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:38:12.251391] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-02. [2025-10-20T17:38:12.673452] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:38:12.779571] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T17:38:13.051425] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-03. [2025-10-20T17:38:13.437043] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-10-20T17:38:13.851437] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-04. [2025-10-20T17:38:14.651382] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-04. [2025-10-20T17:38:15.093215] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T17:38:15.197051] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T17:38:15.451390] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-05. [2025-10-20T17:38:15.773193] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-10-20T17:38:16.251466] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-06. [2025-10-20T17:38:16.781257] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T17:38:17.051377] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-08. [2025-10-20T17:38:17.851420] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-07. [2025-10-20T17:38:18.147370] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T17:38:18.364312] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T17:38:18.651376] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-09. [2025-10-20T17:38:19.451385] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-03. [2025-10-20T17:38:19.957196] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T17:38:20.251358] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-05. [2025-10-20T17:38:20.710541] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:38:21.051426] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-04. [2025-10-20T17:38:21.158031] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T17:38:21.851453] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-06. [2025-10-20T17:38:22.651411] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-10. [2025-10-20T17:38:22.762046] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:38:22.951583] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T17:38:23.451378] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-07. [2025-10-20T17:38:23.585818] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T17:38:23.590441] [INFO ] [REMOTE ] BMEZ: Full fetch complete (39854 records). [2025-10-20T17:38:23.590618] [DEBUG] [VALIDATOR ] BMEZ: Validating primary data. [2025-10-20T17:38:23.988446] [DEBUG] [VALIDATOR ] BMEZ: All expected trading days are present. [2025-10-20T17:38:23.988568] [DEBUG] [VALIDATOR ] BMEZ: Primary data validation passed. [2025-10-20T17:38:23.989297] [INFO ] [MAIN ] BMEZ: Fetched primary data is valid. [2025-10-20T17:38:23.989363] [DEBUG] [LOCAL ] BMEZ: Saving primary data. [2025-10-20T17:38:23.990844] [DEBUG] [LOCAL ] BMEZ: Trimmed 39854 records to 25392 (keeping 200 days of data) [2025-10-20T17:38:24.169883] [DEBUG] [LOCAL ] BMEZ: Primary data saved (25392 records). [2025-10-20T17:38:24.170088] [INFO ] [MAIN ] BMEZ: Full history saved. [2025-10-20T17:38:24.170184] [INFO ] [MAIN ] Update Progress: 307/2367 symbols completed. BMEZ took 32.1s. Estimated remaining: 4h 17m 12s. Rate: 8.0 symbols/min [2025-10-20T17:38:24.254383] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-08. [2025-10-20T17:38:24.366795] [INFO ] [MAIN ] BMY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:24.413953] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T17:38:24.437459] [DEBUG] [LOCAL ] BMY: Loaded 54537 existing records from primary storage. [2025-10-20T17:38:24.437816] [INFO ] [MAIN ] BMY: Loaded 54537 existing records. [2025-10-20T17:38:24.618819] [WARN ] [MAIN ] BMY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:24.619135] [INFO ] [MAIN ] BMY: Wiping existing primary data before full refresh. [2025-10-20T17:38:24.619186] [INFO ] [LOCAL ] BMY: Starting deletion of primary data. [2025-10-20T17:38:24.619822] [DEBUG] [LOCAL ] BMY: Deleted primary data file. [2025-10-20T17:38:24.620123] [INFO ] [MAIN ] BMY: Starting full primary refresh. [2025-10-20T17:38:24.804611] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T17:38:24.989308] [INFO ] [REMOTE ] BMY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:25.051340] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-01. [2025-10-20T17:38:25.851372] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-05. [2025-10-20T17:38:26.651373] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-09. [2025-10-20T17:38:27.112623] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-01 (2.06 seconds). [2025-10-20T17:38:27.274661] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T17:38:27.299715] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T17:38:27.451362] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-08. [2025-10-20T17:38:28.251419] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-02. [2025-10-20T17:38:28.686534] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T17:38:29.051368] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-09. [2025-10-20T17:38:29.851359] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-06. [2025-10-20T17:38:30.232052] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T17:38:30.299470] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T17:38:30.651391] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-10. [2025-10-20T17:38:31.175373] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T17:38:31.176876] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:38:31.178026] [INFO ] [REMOTE ] BMN: Full fetch complete (3672 records). [2025-10-20T17:38:31.185246] [DEBUG] [VALIDATOR ] BMN: Validating primary data. [2025-10-20T17:38:31.456631] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-03. [2025-10-20T17:38:31.560890] [DEBUG] [VALIDATOR ] BMN: All expected trading days are present. [2025-10-20T17:38:31.560968] [DEBUG] [VALIDATOR ] BMN: Primary data validation passed. [2025-10-20T17:38:31.561096] [INFO ] [MAIN ] BMN: Fetched primary data is valid. [2025-10-20T17:38:31.561133] [DEBUG] [LOCAL ] BMN: Saving primary data. [2025-10-20T17:38:31.561473] [DEBUG] [LOCAL ] BMN: Trimmed 3672 records to 2592 (keeping 200 days of data) [2025-10-20T17:38:31.577498] [DEBUG] [LOCAL ] BMN: Primary data saved (2592 records). [2025-10-20T17:38:31.577603] [INFO ] [MAIN ] BMN: Full history saved. [2025-10-20T17:38:31.577701] [INFO ] [MAIN ] Update Progress: 308/2367 symbols completed. BMN took 26.5s. Estimated remaining: 4h 17m 4s. Rate: 8.0 symbols/min [2025-10-20T17:38:31.746518] [INFO ] [MAIN ] BN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:31.794701] [DEBUG] [LOCAL ] BN: Loaded 53768 existing records from primary storage. [2025-10-20T17:38:31.795086] [INFO ] [MAIN ] BN: Loaded 53768 existing records. [2025-10-20T17:38:31.984393] [WARN ] [MAIN ] BN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:31.984693] [INFO ] [MAIN ] BN: Wiping existing primary data before full refresh. [2025-10-20T17:38:31.984751] [INFO ] [LOCAL ] BN: Starting deletion of primary data. [2025-10-20T17:38:31.985210] [DEBUG] [LOCAL ] BN: Deleted primary data file. [2025-10-20T17:38:31.985266] [INFO ] [MAIN ] BN: Starting full primary refresh. [2025-10-20T17:38:32.260920] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-07. [2025-10-20T17:38:32.347048] [INFO ] [REMOTE ] BN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:33.051379] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-10. [2025-10-20T17:38:33.453333] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:38:33.557181] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T17:38:33.851411] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-08. [2025-10-20T17:38:34.470080] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T17:38:34.474429] [INFO ] [REMOTE ] BMI: Full fetch complete (33800 records). [2025-10-20T17:38:34.474566] [DEBUG] [VALIDATOR ] BMI: Validating primary data. [2025-10-20T17:38:34.660528] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-04. [2025-10-20T17:38:34.849547] [DEBUG] [VALIDATOR ] BMI: All expected trading days are present. [2025-10-20T17:38:34.849624] [DEBUG] [VALIDATOR ] BMI: Primary data validation passed. [2025-10-20T17:38:34.850108] [INFO ] [MAIN ] BMI: Fetched primary data is valid. [2025-10-20T17:38:34.850212] [DEBUG] [LOCAL ] BMI: Saving primary data. [2025-10-20T17:38:34.851017] [DEBUG] [LOCAL ] BMI: Trimmed 33800 records to 24793 (keeping 200 days of data) [2025-10-20T17:38:35.017509] [DEBUG] [LOCAL ] BMI: Primary data saved (24793 records). [2025-10-20T17:38:35.017669] [INFO ] [MAIN ] BMI: Full history saved. [2025-10-20T17:38:35.017766] [INFO ] [MAIN ] Update Progress: 309/2367 symbols completed. BMI took 31.7s. Estimated remaining: 4h 16m 29s. Rate: 8.0 symbols/min [2025-10-20T17:38:35.211550] [INFO ] [MAIN ] BNED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:35.239199] [DEBUG] [LOCAL ] BNED: Loaded 28914 existing records from primary storage. [2025-10-20T17:38:35.239478] [INFO ] [MAIN ] BNED: Loaded 28914 existing records. [2025-10-20T17:38:35.421273] [WARN ] [MAIN ] BNED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:35.421640] [INFO ] [MAIN ] BNED: Wiping existing primary data before full refresh. [2025-10-20T17:38:35.421835] [INFO ] [LOCAL ] BNED: Starting deletion of primary data. [2025-10-20T17:38:35.422061] [DEBUG] [LOCAL ] BNED: Deleted primary data file. [2025-10-20T17:38:35.422214] [INFO ] [MAIN ] BNED: Starting full primary refresh. [2025-10-20T17:38:35.453621] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-01. [2025-10-20T17:38:35.557375] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:38:35.854142] [INFO ] [REMOTE ] BNED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:36.251363] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-09. [2025-10-20T17:38:36.644247] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T17:38:37.051494] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-05. [2025-10-20T17:38:37.146161] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:38:37.783929] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:38:37.851473] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-02. [2025-10-20T17:38:38.651387] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-10. [2025-10-20T17:38:39.451440] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-01. [2025-10-20T17:38:39.615538] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-05 (2.56 seconds). [2025-10-20T17:38:39.658068] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T17:38:40.168239] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T17:38:40.174504] [INFO ] [REMOTE ] BMO: Full fetch complete (60042 records). [2025-10-20T17:38:40.174626] [DEBUG] [VALIDATOR ] BMO: Validating primary data. [2025-10-20T17:38:40.256505] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-06. [2025-10-20T17:38:40.570661] [DEBUG] [VALIDATOR ] BMO: All expected trading days are present. [2025-10-20T17:38:40.570835] [DEBUG] [VALIDATOR ] BMO: Primary data validation passed. [2025-10-20T17:38:40.571531] [INFO ] [MAIN ] BMO: Fetched primary data is valid. [2025-10-20T17:38:40.571576] [DEBUG] [LOCAL ] BMO: Saving primary data. [2025-10-20T17:38:40.572879] [DEBUG] [LOCAL ] BMO: Trimmed 60042 records to 41223 (keeping 200 days of data) [2025-10-20T17:38:40.830534] [DEBUG] [LOCAL ] BMO: Primary data saved (41223 records). [2025-10-20T17:38:40.830695] [INFO ] [MAIN ] BMO: Full history saved. [2025-10-20T17:38:40.830788] [INFO ] [MAIN ] Update Progress: 310/2367 symbols completed. BMO took 33.6s. Estimated remaining: 4h 16m 11s. Rate: 8.0 symbols/min [2025-10-20T17:38:40.990369] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:38:41.056576] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-02. [2025-10-20T17:38:41.085625] [INFO ] [MAIN ] BNH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:41.096150] [DEBUG] [LOCAL ] BNH: Loaded 5065 existing records from primary storage. [2025-10-20T17:38:41.096389] [INFO ] [MAIN ] BNH: Loaded 5065 existing records. [2025-10-20T17:38:41.336161] [WARN ] [MAIN ] BNH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:41.336389] [INFO ] [MAIN ] BNH: Wiping existing primary data before full refresh. [2025-10-20T17:38:41.336436] [INFO ] [LOCAL ] BNH: Starting deletion of primary data. [2025-10-20T17:38:41.336600] [DEBUG] [LOCAL ] BNH: Deleted primary data file. [2025-10-20T17:38:41.336693] [INFO ] [MAIN ] BNH: Starting full primary refresh. [2025-10-20T17:38:41.752532] [INFO ] [REMOTE ] BNH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:41.820271] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T17:38:41.851477] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-01. [2025-10-20T17:38:41.918091] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:38:42.540286] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-10-20T17:38:42.651441] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-03. [2025-10-20T17:38:43.451467] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-02. [2025-10-20T17:38:43.997988] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T17:38:44.251379] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-07. [2025-10-20T17:38:44.455053] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T17:38:45.051434] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-04. [2025-10-20T17:38:45.851460] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-03. [2025-10-20T17:38:46.015752] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T17:38:46.326742] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T17:38:46.651374] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-08. [2025-10-20T17:38:46.804176] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T17:38:47.451440] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-04. [2025-10-20T17:38:48.047792] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T17:38:48.251414] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-05. [2025-10-20T17:38:48.453126] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T17:38:49.028750] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T17:38:49.051403] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-03. [2025-10-20T17:38:49.851442] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-06. [2025-10-20T17:38:50.394893] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T17:38:50.479759] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T17:38:50.651439] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-09. [2025-10-20T17:38:51.451436] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-07. [2025-10-20T17:38:51.862919] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-10-20T17:38:52.251396] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-04. [2025-10-20T17:38:52.875483] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T17:38:53.051414] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-08. [2025-10-20T17:38:53.362605] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T17:38:53.683180] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T17:38:53.851454] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-05. [2025-10-20T17:38:54.651459] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-10. [2025-10-20T17:38:55.161597] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T17:38:55.451413] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-09. [2025-10-20T17:38:55.477317] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T17:38:55.486065] [INFO ] [REMOTE ] BMY: Full fetch complete (77937 records). [2025-10-20T17:38:55.486247] [DEBUG] [VALIDATOR ] BMY: Validating primary data. [2025-10-20T17:38:55.901110] [DEBUG] [VALIDATOR ] BMY: All expected trading days are present. [2025-10-20T17:38:55.901939] [DEBUG] [VALIDATOR ] BMY: Primary data validation passed. [2025-10-20T17:38:55.903315] [INFO ] [MAIN ] BMY: Fetched primary data is valid. [2025-10-20T17:38:55.903424] [DEBUG] [LOCAL ] BMY: Saving primary data. [2025-10-20T17:38:55.902231] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-10-20T17:38:55.907586] [DEBUG] [LOCAL ] BMY: Trimmed 77937 records to 53757 (keeping 200 days of data) [2025-10-20T17:38:56.249892] [DEBUG] [LOCAL ] BMY: Primary data saved (53757 records). [2025-10-20T17:38:56.250033] [INFO ] [MAIN ] BMY: Full history saved. [2025-10-20T17:38:56.250108] [INFO ] [MAIN ] Update Progress: 311/2367 symbols completed. BMY took 32.1s. Estimated remaining: 4h 16m 56s. Rate: 8.0 symbols/min [2025-10-20T17:38:56.260390] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-05. [2025-10-20T17:38:56.421166] [INFO ] [MAIN ] BNJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:56.426224] [DEBUG] [LOCAL ] BNJ: Loaded 2639 existing records from primary storage. [2025-10-20T17:38:56.426377] [INFO ] [MAIN ] BNJ: Loaded 2639 existing records. [2025-10-20T17:38:56.603067] [WARN ] [MAIN ] BNJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:56.603302] [INFO ] [MAIN ] BNJ: Wiping existing primary data before full refresh. [2025-10-20T17:38:56.603350] [INFO ] [LOCAL ] BNJ: Starting deletion of primary data. [2025-10-20T17:38:56.603507] [DEBUG] [LOCAL ] BNJ: Deleted primary data file. [2025-10-20T17:38:56.603560] [INFO ] [MAIN ] BNJ: Starting full primary refresh. [2025-10-20T17:38:56.952126] [INFO ] [REMOTE ] BNJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:57.051406] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-01. [2025-10-20T17:38:57.650518] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T17:38:57.851403] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-02. [2025-10-20T17:38:58.069755] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:38:58.299934] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T17:38:58.651441] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-06. [2025-10-20T17:38:59.451466] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-06. [2025-10-20T17:39:00.251412] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-03. [2025-10-20T17:39:00.399324] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T17:39:00.442210] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T17:39:00.640570] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-10-20T17:39:01.051426] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-07. [2025-10-20T17:39:01.851455] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-04. [2025-10-20T17:39:02.338954] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T17:39:02.424158] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T17:39:02.651443] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-07. [2025-10-20T17:39:03.451441] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-10. [2025-10-20T17:39:04.251403] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-05. [2025-10-20T17:39:04.296724] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T17:39:04.298267] [INFO ] [REMOTE ] BNH: Full fetch complete (6804 records). [2025-10-20T17:39:04.298376] [DEBUG] [VALIDATOR ] BNH: Validating primary data. [2025-10-20T17:39:04.695801] [DEBUG] [VALIDATOR ] BNH: All expected trading days are present. [2025-10-20T17:39:04.695917] [DEBUG] [VALIDATOR ] BNH: Primary data validation passed. [2025-10-20T17:39:04.696226] [INFO ] [MAIN ] BNH: Fetched primary data is valid. [2025-10-20T17:39:04.696308] [DEBUG] [LOCAL ] BNH: Saving primary data. [2025-10-20T17:39:04.697269] [DEBUG] [LOCAL ] BNH: Trimmed 6804 records to 4731 (keeping 200 days of data) [2025-10-20T17:39:04.772725] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T17:39:04.771941] [DEBUG] [LOCAL ] BNH: Primary data saved (4731 records). [2025-10-20T17:39:04.776526] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-10-20T17:39:04.778857] [INFO ] [MAIN ] BNH: Full history saved. [2025-10-20T17:39:04.779227] [INFO ] [MAIN ] Update Progress: 312/2367 symbols completed. BNH took 23.9s. Estimated remaining: 4h 16m 55s. Rate: 8.0 symbols/min [2025-10-20T17:39:04.971337] [INFO ] [MAIN ] BNL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:05.016390] [DEBUG] [LOCAL ] BNL: Loaded 47981 existing records from primary storage. [2025-10-20T17:39:05.016780] [INFO ] [MAIN ] BNL: Loaded 47981 existing records. [2025-10-20T17:39:05.056759] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-08. [2025-10-20T17:39:05.202166] [WARN ] [MAIN ] BNL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:05.202404] [INFO ] [MAIN ] BNL: Wiping existing primary data before full refresh. [2025-10-20T17:39:05.202454] [INFO ] [LOCAL ] BNL: Starting deletion of primary data. [2025-10-20T17:39:05.202697] [DEBUG] [LOCAL ] BNL: Deleted primary data file. [2025-10-20T17:39:05.202755] [INFO ] [MAIN ] BNL: Starting full primary refresh. [2025-10-20T17:39:05.558121] [INFO ] [REMOTE ] BNL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:05.851403] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-08. [2025-10-20T17:39:06.248871] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T17:39:06.651480] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-06. [2025-10-20T17:39:07.082002] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T17:39:07.451418] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-01. [2025-10-20T17:39:07.687016] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T17:39:08.251388] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-09. [2025-10-20T17:39:08.791908] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:39:09.051376] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-09. [2025-10-20T17:39:09.377421] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T17:39:09.851457] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-02. [2025-10-20T17:39:10.651464] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-10. [2025-10-20T17:39:11.222775] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T17:39:11.417141] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T17:39:11.451351] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-03. [2025-10-20T17:39:12.022148] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:39:12.025786] [INFO ] [REMOTE ] BNED: Full fetch complete (42265 records). [2025-10-20T17:39:12.025914] [DEBUG] [VALIDATOR ] BNED: Validating primary data. [2025-10-20T17:39:12.256579] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-10. [2025-10-20T17:39:12.419841] [DEBUG] [VALIDATOR ] BNED: All expected trading days are present. [2025-10-20T17:39:12.419922] [DEBUG] [VALIDATOR ] BNED: Primary data validation passed. [2025-10-20T17:39:12.420454] [INFO ] [MAIN ] BNED: Fetched primary data is valid. [2025-10-20T17:39:12.420495] [DEBUG] [LOCAL ] BNED: Saving primary data. [2025-10-20T17:39:12.421390] [DEBUG] [LOCAL ] BNED: Trimmed 42265 records to 28483 (keeping 200 days of data) [2025-10-20T17:39:12.586547] [DEBUG] [LOCAL ] BNED: Primary data saved (28483 records). [2025-10-20T17:39:12.586740] [INFO ] [MAIN ] BNED: Full history saved. [2025-10-20T17:39:12.586831] [INFO ] [MAIN ] Update Progress: 313/2367 symbols completed. BNED took 37.6s. Estimated remaining: 4h 16m 49s. Rate: 8.0 symbols/min [2025-10-20T17:39:12.759817] [INFO ] [MAIN ] BNS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:12.805446] [DEBUG] [LOCAL ] BNS: Loaded 52255 existing records from primary storage. [2025-10-20T17:39:12.805773] [INFO ] [MAIN ] BNS: Loaded 52255 existing records. [2025-10-20T17:39:13.007138] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:39:13.013691] [WARN ] [MAIN ] BNS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:13.013948] [INFO ] [MAIN ] BNS: Wiping existing primary data before full refresh. [2025-10-20T17:39:13.014020] [INFO ] [LOCAL ] BNS: Starting deletion of primary data. [2025-10-20T17:39:13.017477] [DEBUG] [LOCAL ] BNS: Deleted primary data file. [2025-10-20T17:39:13.023960] [INFO ] [MAIN ] BNS: Starting full primary refresh. [2025-10-20T17:39:13.055504] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-07. [2025-10-20T17:39:13.391724] [INFO ] [REMOTE ] BNS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:13.577815] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T17:39:13.851339] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-01. [2025-10-20T17:39:14.345132] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T17:39:14.353203] [INFO ] [REMOTE ] BN: Full fetch complete (76951 records). [2025-10-20T17:39:14.353359] [DEBUG] [VALIDATOR ] BN: Validating primary data. [2025-10-20T17:39:14.656549] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-08. [2025-10-20T17:39:14.743830] [DEBUG] [VALIDATOR ] BN: All expected trading days are present. [2025-10-20T17:39:14.743911] [DEBUG] [VALIDATOR ] BN: Primary data validation passed. [2025-10-20T17:39:14.744692] [INFO ] [MAIN ] BN: Fetched primary data is valid. [2025-10-20T17:39:14.744756] [DEBUG] [LOCAL ] BN: Saving primary data. [2025-10-20T17:39:14.746228] [DEBUG] [LOCAL ] BN: Trimmed 76951 records to 52986 (keeping 200 days of data) [2025-10-20T17:39:15.065289] [DEBUG] [LOCAL ] BN: Primary data saved (52986 records). [2025-10-20T17:39:15.065412] [INFO ] [MAIN ] BN: Full history saved. [2025-10-20T17:39:15.065760] [INFO ] [MAIN ] Update Progress: 314/2367 symbols completed. BN took 43.5s. Estimated remaining: 4h 16m 9s. Rate: 8.0 symbols/min [2025-10-20T17:39:15.238735] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T17:39:15.264396] [INFO ] [MAIN ] BNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:15.271881] [DEBUG] [LOCAL ] BNT: Loaded 3594 existing records from primary storage. [2025-10-20T17:39:15.272093] [INFO ] [MAIN ] BNT: Loaded 3594 existing records. [2025-10-20T17:39:15.449186] [WARN ] [MAIN ] BNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:15.449439] [INFO ] [MAIN ] BNT: Wiping existing primary data before full refresh. [2025-10-20T17:39:15.449490] [INFO ] [LOCAL ] BNT: Starting deletion of primary data. [2025-10-20T17:39:15.449615] [DEBUG] [LOCAL ] BNT: Deleted primary data file. [2025-10-20T17:39:15.449697] [INFO ] [MAIN ] BNT: Starting full primary refresh. [2025-10-20T17:39:15.461079] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-04. [2025-10-20T17:39:15.834840] [INFO ] [REMOTE ] BNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:15.852341] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:39:16.251391] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-09. [2025-10-20T17:39:16.698946] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-10-20T17:39:16.885524] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T17:39:17.051374] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-02. [2025-10-20T17:39:17.851359] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-01. [2025-10-20T17:39:18.468429] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T17:39:18.645222] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T17:39:18.654590] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-10. [2025-10-20T17:39:19.247381] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T17:39:19.248529] [INFO ] [REMOTE ] BNJ: Full fetch complete (3894 records). [2025-10-20T17:39:19.248608] [DEBUG] [VALIDATOR ] BNJ: Validating primary data. [2025-10-20T17:39:19.456497] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-05. [2025-10-20T17:39:19.646047] [DEBUG] [VALIDATOR ] BNJ: All expected trading days are present. [2025-10-20T17:39:19.646134] [DEBUG] [VALIDATOR ] BNJ: Primary data validation passed. [2025-10-20T17:39:19.646284] [INFO ] [MAIN ] BNJ: Fetched primary data is valid. [2025-10-20T17:39:19.646323] [DEBUG] [LOCAL ] BNJ: Saving primary data. [2025-10-20T17:39:19.646809] [DEBUG] [LOCAL ] BNJ: Trimmed 3894 records to 2531 (keeping 200 days of data) [2025-10-20T17:39:19.666392] [DEBUG] [LOCAL ] BNJ: Primary data saved (2531 records). [2025-10-20T17:39:19.666571] [INFO ] [MAIN ] BNJ: Full history saved. [2025-10-20T17:39:19.666700] [INFO ] [MAIN ] Update Progress: 315/2367 symbols completed. BNJ took 23.4s. Estimated remaining: 4h 15m 43s. Rate: 8.0 symbols/min [2025-10-20T17:39:19.839215] [INFO ] [MAIN ] BNY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:19.847792] [DEBUG] [LOCAL ] BNY: Loaded 7307 existing records from primary storage. [2025-10-20T17:39:19.847981] [INFO ] [MAIN ] BNY: Loaded 7307 existing records. [2025-10-20T17:39:20.029799] [WARN ] [MAIN ] BNY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:20.030078] [INFO ] [MAIN ] BNY: Wiping existing primary data before full refresh. [2025-10-20T17:39:20.030128] [INFO ] [LOCAL ] BNY: Starting deletion of primary data. [2025-10-20T17:39:20.030296] [DEBUG] [LOCAL ] BNY: Deleted primary data file. [2025-10-20T17:39:20.030352] [INFO ] [MAIN ] BNY: Starting full primary refresh. [2025-10-20T17:39:20.259358] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-02. [2025-10-20T17:39:20.400077] [INFO ] [REMOTE ] BNY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:20.794707] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T17:39:20.945249] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T17:39:21.051361] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-03. [2025-10-20T17:39:21.851428] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-01. [2025-10-20T17:39:22.405935] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T17:39:22.651424] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-02. [2025-10-20T17:39:22.797186] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:39:23.090259] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:39:23.451370] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-03. [2025-10-20T17:39:23.952524] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T17:39:24.251361] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-06. [2025-10-20T17:39:25.051380] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-04. [2025-10-20T17:39:25.770103] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:39:25.851334] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-03. [2025-10-20T17:39:26.582831] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T17:39:26.651364] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-04. [2025-10-20T17:39:27.070449] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T17:39:27.165205] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-10-20T17:39:27.451430] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-05. [2025-10-20T17:39:28.154300] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:39:28.251366] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-04. [2025-10-20T17:39:29.000781] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T17:39:29.051382] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-05. [2025-10-20T17:39:29.851406] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-05. [2025-10-20T17:39:30.394709] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T17:39:30.421848] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T17:39:30.651384] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-06. [2025-10-20T17:39:31.040012] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-10-20T17:39:31.451476] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-07. [2025-10-20T17:39:32.015087] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T17:39:32.251426] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-07. [2025-10-20T17:39:33.051390] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-08. [2025-10-20T17:39:33.537072] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T17:39:33.706451] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T17:39:33.851400] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-09. [2025-10-20T17:39:34.469485] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T17:39:34.651393] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-08. [2025-10-20T17:39:35.451411] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-06. [2025-10-20T17:39:36.000459] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:39:36.251367] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-09. [2025-10-20T17:39:37.051451] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-06. [2025-10-20T17:39:37.235256] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:39:37.368499] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T17:39:37.532139] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T17:39:37.851479] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-10. [2025-10-20T17:39:38.524849] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:39:38.526157] [INFO ] [REMOTE ] BNT: Full fetch complete (4796 records). [2025-10-20T17:39:38.526223] [DEBUG] [VALIDATOR ] BNT: Validating primary data. [2025-10-20T17:39:38.663857] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-07. [2025-10-20T17:39:38.889298] [DEBUG] [VALIDATOR ] BNT: All expected trading days are present. [2025-10-20T17:39:38.889379] [DEBUG] [VALIDATOR ] BNT: Primary data validation passed. [2025-10-20T17:39:38.889540] [INFO ] [MAIN ] BNT: Fetched primary data is valid. [2025-10-20T17:39:38.889578] [DEBUG] [LOCAL ] BNT: Saving primary data. [2025-10-20T17:39:38.889958] [DEBUG] [LOCAL ] BNT: Trimmed 4796 records to 3553 (keeping 200 days of data) [2025-10-20T17:39:38.911986] [DEBUG] [LOCAL ] BNT: Primary data saved (3553 records). [2025-10-20T17:39:38.912118] [INFO ] [MAIN ] BNT: Full history saved. [2025-10-20T17:39:38.912198] [INFO ] [MAIN ] Update Progress: 316/2367 symbols completed. BNT took 23.8s. Estimated remaining: 4h 16m 52s. Rate: 8.0 symbols/min [2025-10-20T17:39:39.081062] [INFO ] [MAIN ] BOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:39.099403] [DEBUG] [LOCAL ] BOC: Loaded 18798 existing records from primary storage. [2025-10-20T17:39:39.099691] [INFO ] [MAIN ] BOC: Loaded 18798 existing records. [2025-10-20T17:39:39.297033] [WARN ] [MAIN ] BOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:39.297341] [INFO ] [MAIN ] BOC: Wiping existing primary data before full refresh. [2025-10-20T17:39:39.297395] [INFO ] [LOCAL ] BOC: Starting deletion of primary data. [2025-10-20T17:39:39.297592] [DEBUG] [LOCAL ] BOC: Deleted primary data file. [2025-10-20T17:39:39.297633] [INFO ] [MAIN ] BOC: Starting full primary refresh. [2025-10-20T17:39:39.456536] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-07. [2025-10-20T17:39:39.613302] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:39:39.705376] [INFO ] [REMOTE ] BOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:40.251391] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-10. [2025-10-20T17:39:41.001448] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:39:41.051429] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-08. [2025-10-20T17:39:41.851387] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-01. [2025-10-20T17:39:41.879944] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:39:41.887236] [INFO ] [REMOTE ] BNL: Full fetch complete (67279 records). [2025-10-20T17:39:41.887387] [DEBUG] [VALIDATOR ] BNL: Validating primary data. [2025-10-20T17:39:42.263590] [DEBUG] [VALIDATOR ] BNL: All expected trading days are present. [2025-10-20T17:39:42.263689] [DEBUG] [VALIDATOR ] BNL: Primary data validation passed. [2025-10-20T17:39:42.264315] [INFO ] [MAIN ] BNL: Fetched primary data is valid. [2025-10-20T17:39:42.264356] [DEBUG] [LOCAL ] BNL: Saving primary data. [2025-10-20T17:39:42.265558] [DEBUG] [LOCAL ] BNL: Trimmed 67279 records to 47210 (keeping 200 days of data) [2025-10-20T17:39:42.546544] [DEBUG] [LOCAL ] BNL: Primary data saved (47210 records). [2025-10-20T17:39:42.546955] [INFO ] [MAIN ] BNL: Full history saved. [2025-10-20T17:39:42.547042] [INFO ] [MAIN ] Update Progress: 317/2367 symbols completed. BNL took 37.8s. Estimated remaining: 4h 16m 19s. Rate: 8.0 symbols/min [2025-10-20T17:39:42.588461] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T17:39:42.651414] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-08. [2025-10-20T17:39:42.974607] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:39:43.344915] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:39:43.451472] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-02. [2025-10-20T17:39:44.251426] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-09. [2025-10-20T17:39:44.410162] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T17:39:45.051379] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-03. [2025-10-20T17:39:45.711374] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T17:39:45.799324] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T17:39:45.851462] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-09. [2025-10-20T17:39:46.589074] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T17:39:46.651392] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-10. [2025-10-20T17:39:47.356108] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T17:39:47.358789] [INFO ] [REMOTE ] BNY: Full fetch complete (10426 records). [2025-10-20T17:39:47.358871] [DEBUG] [VALIDATOR ] BNY: Validating primary data. [2025-10-20T17:39:47.458875] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-04. [2025-10-20T17:39:47.746376] [DEBUG] [VALIDATOR ] BNY: All expected trading days are present. [2025-10-20T17:39:47.746482] [DEBUG] [VALIDATOR ] BNY: Primary data validation passed. [2025-10-20T17:39:47.746921] [INFO ] [MAIN ] BNY: Fetched primary data is valid. [2025-10-20T17:39:47.746988] [DEBUG] [LOCAL ] BNY: Saving primary data. [2025-10-20T17:39:47.747566] [DEBUG] [LOCAL ] BNY: Trimmed 10426 records to 7267 (keeping 200 days of data) [2025-10-20T17:39:47.796287] [DEBUG] [LOCAL ] BNY: Primary data saved (7267 records). [2025-10-20T17:39:47.796497] [INFO ] [MAIN ] BNY: Full history saved. [2025-10-20T17:39:47.796625] [INFO ] [MAIN ] Update Progress: 318/2367 symbols completed. BNY took 28.1s. Estimated remaining: 4h 15m 57s. Rate: 8.0 symbols/min [2025-10-20T17:39:48.251458] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-10. [2025-10-20T17:39:48.664569] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:39:49.051429] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-05. [2025-10-20T17:39:49.827346] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:39:49.836275] [INFO ] [REMOTE ] BNS: Full fetch complete (74670 records). [2025-10-20T17:39:49.836535] [DEBUG] [VALIDATOR ] BNS: Validating primary data. [2025-10-20T17:39:50.184637] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:39:50.215884] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-06. [2025-10-20T17:39:50.298498] [DEBUG] [VALIDATOR ] BNS: All expected trading days are present. [2025-10-20T17:39:50.298626] [DEBUG] [VALIDATOR ] BNS: Primary data validation passed. [2025-10-20T17:39:50.299868] [INFO ] [MAIN ] BNS: Fetched primary data is valid. [2025-10-20T17:39:50.299939] [DEBUG] [LOCAL ] BNS: Saving primary data. [2025-10-20T17:39:50.302200] [DEBUG] [LOCAL ] BNS: Trimmed 74670 records to 51449 (keeping 200 days of data) [2025-10-20T17:39:50.638162] [DEBUG] [LOCAL ] BNS: Primary data saved (51449 records). [2025-10-20T17:39:50.638397] [INFO ] [MAIN ] BNS: Full history saved. [2025-10-20T17:39:50.638507] [INFO ] [MAIN ] Update Progress: 319/2367 symbols completed. BNS took 38.1s. Estimated remaining: 4h 15m 20s. Rate: 8.0 symbols/min [2025-10-20T17:39:51.323613] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T17:39:51.328830] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-07. [2025-10-20T17:39:52.307173] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:39:52.311921] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-08. [2025-10-20T17:39:53.445389] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:39:53.452080] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-09. [2025-10-20T17:39:54.390452] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T17:39:54.394999] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-10. [2025-10-20T17:39:55.409028] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T17:39:55.412819] [INFO ] [REMOTE ] BOC: Full fetch complete (25815 records). [2025-10-20T17:39:55.412911] [DEBUG] [VALIDATOR ] BOC: Validating primary data. [2025-10-20T17:39:55.814306] [DEBUG] [VALIDATOR ] BOC: All expected trading days are present. [2025-10-20T17:39:55.814412] [DEBUG] [VALIDATOR ] BOC: Primary data validation passed. [2025-10-20T17:39:55.815130] [INFO ] [MAIN ] BOC: Fetched primary data is valid. [2025-10-20T17:39:55.815194] [DEBUG] [LOCAL ] BOC: Saving primary data. [2025-10-20T17:39:55.816045] [DEBUG] [LOCAL ] BOC: Trimmed 25815 records to 18418 (keeping 200 days of data) [2025-10-20T17:39:55.932776] [DEBUG] [LOCAL ] BOC: Primary data saved (18418 records). [2025-10-20T17:39:55.933061] [INFO ] [MAIN ] BOC: Full history saved. [2025-10-20T17:39:55.933184] [INFO ] [MAIN ] Update Progress: 320/2367 symbols completed. BOC took 17.0s. Estimated remaining: 4h 14m 58s. Rate: 8.0 symbols/min [2025-10-20T17:39:56.599970] [INFO ] [MAIN ] BOE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:56.730820] [INFO ] [MAIN ] BOOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:56.713693] [INFO ] [MAIN ] BOND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:56.732497] [INFO ] [MAIN ] BOH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:56.757316] [DEBUG] [LOCAL ] BOE: Loaded 18435 existing records from primary storage. [2025-10-20T17:39:56.757872] [INFO ] [MAIN ] BOE: Loaded 18435 existing records. [2025-10-20T17:39:56.910045] [DEBUG] [LOCAL ] BOND: Loaded 32938 existing records from primary storage. [2025-10-20T17:39:56.969517] [INFO ] [MAIN ] BOND: Loaded 32938 existing records. [2025-10-20T17:39:56.969259] [DEBUG] [LOCAL ] BOOT: Loaded 40347 existing records from primary storage. [2025-10-20T17:39:57.011564] [WARN ] [MAIN ] BOE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:57.057513] [INFO ] [MAIN ] BOOT: Loaded 40347 existing records. [2025-10-20T17:39:57.052139] [INFO ] [MAIN ] BOE: Wiping existing primary data before full refresh. [2025-10-20T17:39:57.076786] [DEBUG] [LOCAL ] BOH: Loaded 31214 existing records from primary storage. [2025-10-20T17:39:57.092257] [INFO ] [LOCAL ] BOE: Starting deletion of primary data. [2025-10-20T17:39:57.092526] [INFO ] [MAIN ] BOH: Loaded 31214 existing records. [2025-10-20T17:39:57.126027] [DEBUG] [LOCAL ] BOE: Deleted primary data file. [2025-10-20T17:39:57.147998] [INFO ] [MAIN ] BOE: Starting full primary refresh. [2025-10-20T17:39:57.699124] [WARN ] [MAIN ] BOND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:57.695956] [WARN ] [MAIN ] BOOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:57.720695] [INFO ] [MAIN ] BOND: Wiping existing primary data before full refresh. [2025-10-20T17:39:57.746135] [INFO ] [LOCAL ] BOND: Starting deletion of primary data. [2025-10-20T17:39:57.760637] [INFO ] [MAIN ] BOOT: Wiping existing primary data before full refresh. [2025-10-20T17:39:57.794322] [WARN ] [MAIN ] BOH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:57.820071] [DEBUG] [LOCAL ] BOND: Deleted primary data file. [2025-10-20T17:39:57.812917] [INFO ] [LOCAL ] BOOT: Starting deletion of primary data. [2025-10-20T17:39:57.827547] [INFO ] [MAIN ] BOND: Starting full primary refresh. [2025-10-20T17:39:57.838354] [INFO ] [MAIN ] BOH: Wiping existing primary data before full refresh. [2025-10-20T17:39:57.838007] [DEBUG] [LOCAL ] BOOT: Deleted primary data file. [2025-10-20T17:39:57.904008] [INFO ] [MAIN ] BOOT: Starting full primary refresh. [2025-10-20T17:39:57.919797] [INFO ] [LOCAL ] BOH: Starting deletion of primary data. [2025-10-20T17:39:57.950244] [DEBUG] [LOCAL ] BOH: Deleted primary data file. [2025-10-20T17:39:57.955626] [INFO ] [MAIN ] BOH: Starting full primary refresh. [2025-10-20T17:39:58.386637] [INFO ] [REMOTE ] BOE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:58.471025] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-01. [2025-10-20T17:39:59.244836] [INFO ] [REMOTE ] BOH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:59.209533] [INFO ] [REMOTE ] BOND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:59.281976] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-01. [2025-10-20T17:39:59.281481] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-01. [2025-10-20T17:39:59.318322] [INFO ] [REMOTE ] BOOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:59.321148] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-01. [2025-10-20T17:39:59.535754] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T17:39:59.540406] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-02. [2025-10-20T17:40:00.444849] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T17:40:00.452051] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-02. [2025-10-20T17:40:00.525550] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T17:40:00.531379] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-03. [2025-10-20T17:40:00.706383] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:40:00.714991] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-02. [2025-10-20T17:40:00.772523] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:40:00.781994] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-02. [2025-10-20T17:40:01.431323] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:40:01.437597] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-03. [2025-10-20T17:40:01.469088] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T17:40:01.851470] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-04. [2025-10-20T17:40:02.044718] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:40:02.149719] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T17:40:02.456063] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T17:40:02.651453] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-03. [2025-10-20T17:40:02.679051] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T17:40:03.451387] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-04. [2025-10-20T17:40:04.214251] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:40:04.251439] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-04. [2025-10-20T17:40:04.439833] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T17:40:05.051384] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-03. [2025-10-20T17:40:05.468115] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T17:40:05.851382] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-05. [2025-10-20T17:40:06.575412] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:40:06.651447] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-05. [2025-10-20T17:40:07.232884] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T17:40:07.451378] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-04. [2025-10-20T17:40:07.621874] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T17:40:08.251405] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-06. [2025-10-20T17:40:08.690137] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T17:40:09.051367] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-05. [2025-10-20T17:40:09.104936] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:40:09.851377] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-07. [2025-10-20T17:40:10.085121] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T17:40:10.651391] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-06. [2025-10-20T17:40:11.408579] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T17:40:11.451354] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-08. [2025-10-20T17:40:11.737480] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T17:40:12.251413] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-05. [2025-10-20T17:40:12.590143] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T17:40:13.051424] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-09. [2025-10-20T17:40:13.851390] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-07. [2025-10-20T17:40:14.100574] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T17:40:14.475258] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T17:40:14.651369] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-06. [2025-10-20T17:40:14.869723] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T17:40:15.451363] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-08. [2025-10-20T17:40:15.766489] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T17:40:16.251376] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-06. [2025-10-20T17:40:16.346988] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T17:40:17.051424] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-10. [2025-10-20T17:40:17.419638] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T17:40:17.851363] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-07. [2025-10-20T17:40:18.439593] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:40:18.444684] [INFO ] [REMOTE ] BOND: Full fetch complete (46586 records). [2025-10-20T17:40:18.444771] [DEBUG] [VALIDATOR ] BOND: Validating primary data. [2025-10-20T17:40:18.656591] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-09. [2025-10-20T17:40:18.812985] [DEBUG] [VALIDATOR ] BOND: All expected trading days are present. [2025-10-20T17:40:18.813063] [DEBUG] [VALIDATOR ] BOND: Primary data validation passed. [2025-10-20T17:40:18.813598] [INFO ] [MAIN ] BOND: Fetched primary data is valid. [2025-10-20T17:40:18.813638] [DEBUG] [LOCAL ] BOND: Saving primary data. [2025-10-20T17:40:18.814614] [DEBUG] [LOCAL ] BOND: Trimmed 46586 records to 32481 (keeping 200 days of data) [2025-10-20T17:40:19.012079] [DEBUG] [LOCAL ] BOND: Primary data saved (32481 records). [2025-10-20T17:40:19.012232] [INFO ] [MAIN ] BOND: Full history saved. [2025-10-20T17:40:19.012323] [INFO ] [MAIN ] Update Progress: 321/2367 symbols completed. BOND took 23.1s. Estimated remaining: 4h 16m 30s. Rate: 8.0 symbols/min [2025-10-20T17:40:19.179168] [INFO ] [MAIN ] BORR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:19.225546] [DEBUG] [LOCAL ] BORR: Loaded 51855 existing records from primary storage. [2025-10-20T17:40:19.226037] [INFO ] [MAIN ] BORR: Loaded 51855 existing records. [2025-10-20T17:40:19.331540] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:40:19.433103] [WARN ] [MAIN ] BORR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:19.433312] [INFO ] [MAIN ] BORR: Wiping existing primary data before full refresh. [2025-10-20T17:40:19.433361] [INFO ] [LOCAL ] BORR: Starting deletion of primary data. [2025-10-20T17:40:19.433813] [DEBUG] [LOCAL ] BORR: Deleted primary data file. [2025-10-20T17:40:19.433873] [INFO ] [MAIN ] BORR: Starting full primary refresh. [2025-10-20T17:40:19.456447] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-07. [2025-10-20T17:40:19.708988] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T17:40:19.822732] [INFO ] [REMOTE ] BORR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:20.251383] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-01. [2025-10-20T17:40:21.052782] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-08. [2025-10-20T17:40:21.058305] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T17:40:21.851357] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-08. [2025-10-20T17:40:22.072586] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T17:40:22.148133] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T17:40:22.651440] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-09. [2025-10-20T17:40:23.312464] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T17:40:23.451436] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-02. [2025-10-20T17:40:23.802017] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T17:40:24.251397] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-10. [2025-10-20T17:40:25.051487] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-09. [2025-10-20T17:40:25.145605] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:40:25.591392] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T17:40:25.596963] [INFO ] [REMOTE ] BOE: Full fetch complete (26835 records). [2025-10-20T17:40:25.597139] [DEBUG] [VALIDATOR ] BOE: Validating primary data. [2025-10-20T17:40:25.856770] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-03. [2025-10-20T17:40:25.972367] [DEBUG] [VALIDATOR ] BOE: All expected trading days are present. [2025-10-20T17:40:25.972441] [DEBUG] [VALIDATOR ] BOE: Primary data validation passed. [2025-10-20T17:40:25.972843] [INFO ] [MAIN ] BOE: Fetched primary data is valid. [2025-10-20T17:40:25.972904] [DEBUG] [LOCAL ] BOE: Saving primary data. [2025-10-20T17:40:25.973523] [DEBUG] [LOCAL ] BOE: Trimmed 26835 records to 18109 (keeping 200 days of data) [2025-10-20T17:40:26.077731] [DEBUG] [LOCAL ] BOE: Primary data saved (18109 records). [2025-10-20T17:40:26.077864] [INFO ] [MAIN ] BOE: Full history saved. [2025-10-20T17:40:26.077940] [INFO ] [MAIN ] Update Progress: 322/2367 symbols completed. BOE took 30.1s. Estimated remaining: 4h 16m 20s. Rate: 8.0 symbols/min [2025-10-20T17:40:26.247073] [INFO ] [MAIN ] BOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:26.269017] [DEBUG] [LOCAL ] BOW: Loaded 23579 existing records from primary storage. [2025-10-20T17:40:26.269258] [INFO ] [MAIN ] BOW: Loaded 23579 existing records. [2025-10-20T17:40:26.447933] [WARN ] [MAIN ] BOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:26.460387] [INFO ] [MAIN ] BOW: Wiping existing primary data before full refresh. [2025-10-20T17:40:26.472943] [INFO ] [LOCAL ] BOW: Starting deletion of primary data. [2025-10-20T17:40:26.476187] [DEBUG] [LOCAL ] BOW: Deleted primary data file. [2025-10-20T17:40:26.479069] [INFO ] [MAIN ] BOW: Starting full primary refresh. [2025-10-20T17:40:26.479009] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:40:26.655777] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-10. [2025-10-20T17:40:26.847982] [INFO ] [REMOTE ] BOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:27.451361] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-10. [2025-10-20T17:40:27.483284] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T17:40:28.251433] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-04. [2025-10-20T17:40:28.286711] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:40:28.292327] [INFO ] [REMOTE ] BOOT: Full fetch complete (60126 records). [2025-10-20T17:40:28.292450] [DEBUG] [VALIDATOR ] BOOT: Validating primary data. [2025-10-20T17:40:28.685742] [DEBUG] [VALIDATOR ] BOOT: All expected trading days are present. [2025-10-20T17:40:28.685838] [DEBUG] [VALIDATOR ] BOOT: Primary data validation passed. [2025-10-20T17:40:28.686619] [INFO ] [MAIN ] BOOT: Fetched primary data is valid. [2025-10-20T17:40:28.686718] [DEBUG] [LOCAL ] BOOT: Saving primary data. [2025-10-20T17:40:28.687980] [DEBUG] [LOCAL ] BOOT: Trimmed 60126 records to 39698 (keeping 200 days of data) [2025-10-20T17:40:28.924964] [DEBUG] [LOCAL ] BOOT: Primary data saved (39698 records). [2025-10-20T17:40:28.925115] [INFO ] [MAIN ] BOOT: Full history saved. [2025-10-20T17:40:28.926059] [INFO ] [MAIN ] Update Progress: 323/2367 symbols completed. BOOT took 33.0s. Estimated remaining: 4h 15m 43s. Rate: 8.0 symbols/min [2025-10-20T17:40:29.056637] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T17:40:29.076479] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-01. [2025-10-20T17:40:29.097581] [INFO ] [REMOTE ] BOH: Full fetch complete (45178 records). [2025-10-20T17:40:29.125498] [INFO ] [MAIN ] BOX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:29.126017] [DEBUG] [VALIDATOR ] BOH: Validating primary data. [2025-10-20T17:40:29.127496] [FATAL] [VALIDATOR ] BOH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:40:29.127610] [FATAL] [MAIN ] BOH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:40:29.127700] [WARN ] [MAIN ] BOH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:40:29.127772] [INFO ] [LOCAL ] BOH: Starting deletion of primary data. [2025-10-20T17:40:29.127853] [DEBUG] [LOCAL ] BOH: No primary data file found to delete. [2025-10-20T17:40:29.128030] [INFO ] [LOCAL ] BOH: Starting deletion of derivative data. [2025-10-20T17:40:29.128234] [DEBUG] [LOCAL ] BOH: Deleted derivative data file. [2025-10-20T17:40:29.128368] [INFO ] [MAIN ] Update Progress: 324/2367 symbols completed. BOH took 33.2s. Estimated remaining: 4h 14m 49s. Rate: 8.0 symbols/min [2025-10-20T17:40:29.314357] [INFO ] [MAIN ] BP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:29.334271] [DEBUG] [LOCAL ] BOX: Loaded 50897 existing records from primary storage. [2025-10-20T17:40:29.334712] [INFO ] [MAIN ] BOX: Loaded 50897 existing records. [2025-10-20T17:40:29.536209] [WARN ] [MAIN ] BOX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:29.551105] [INFO ] [MAIN ] BOX: Wiping existing primary data before full refresh. [2025-10-20T17:40:29.551187] [INFO ] [LOCAL ] BOX: Starting deletion of primary data. [2025-10-20T17:40:29.552054] [DEBUG] [LOCAL ] BOX: Deleted primary data file. [2025-10-20T17:40:29.552013] [DEBUG] [LOCAL ] BP: Loaded 54488 existing records from primary storage. [2025-10-20T17:40:29.552161] [INFO ] [MAIN ] BOX: Starting full primary refresh. [2025-10-20T17:40:29.583604] [INFO ] [MAIN ] BP: Loaded 54488 existing records. [2025-10-20T17:40:29.963665] [WARN ] [MAIN ] BP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:29.987184] [INFO ] [MAIN ] BP: Wiping existing primary data before full refresh. [2025-10-20T17:40:29.997851] [INFO ] [LOCAL ] BP: Starting deletion of primary data. [2025-10-20T17:40:30.023476] [DEBUG] [LOCAL ] BP: Deleted primary data file. [2025-10-20T17:40:30.044214] [INFO ] [MAIN ] BP: Starting full primary refresh. [2025-10-20T17:40:30.169697] [INFO ] [REMOTE ] BOX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:30.236573] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-01. [2025-10-20T17:40:30.259675] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:40:30.301761] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T17:40:30.535316] [INFO ] [REMOTE ] BP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:30.651405] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-01. [2025-10-20T17:40:31.451421] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-02. [2025-10-20T17:40:31.830605] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T17:40:32.256629] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-05. [2025-10-20T17:40:32.259778] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T17:40:32.360478] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T17:40:33.051428] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-03. [2025-10-20T17:40:33.851373] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-02. [2025-10-20T17:40:34.145316] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T17:40:34.317293] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T17:40:34.651376] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-04. [2025-10-20T17:40:35.080526] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:40:35.451340] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-06. [2025-10-20T17:40:35.862761] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:40:36.251403] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-05. [2025-10-20T17:40:36.995435] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T17:40:37.051414] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-03. [2025-10-20T17:40:37.322989] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:40:37.851373] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-06. [2025-10-20T17:40:38.641571] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:40:38.651349] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-07. [2025-10-20T17:40:38.724477] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:40:39.451401] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-02. [2025-10-20T17:40:39.598874] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:40:40.251368] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-08. [2025-10-20T17:40:41.051383] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-07. [2025-10-20T17:40:41.238387] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T17:40:41.263969] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:40:41.851436] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-09. [2025-10-20T17:40:42.651356] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-03. [2025-10-20T17:40:42.774003] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T17:40:42.792549] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T17:40:43.451407] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-08. [2025-10-20T17:40:44.251451] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-04. [2025-10-20T17:40:44.879900] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-03 (2.23 seconds). [2025-10-20T17:40:44.951361] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T17:40:45.051361] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-09. [2025-10-20T17:40:45.851353] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-10. [2025-10-20T17:40:45.927812] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T17:40:46.514452] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:40:46.651372] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-10. [2025-10-20T17:40:46.748057] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:40:46.752610] [INFO ] [REMOTE ] BOW: Full fetch complete (32083 records). [2025-10-20T17:40:46.752787] [DEBUG] [VALIDATOR ] BOW: Validating primary data. [2025-10-20T17:40:47.133575] [DEBUG] [VALIDATOR ] BOW: All expected trading days are present. [2025-10-20T17:40:47.133680] [DEBUG] [VALIDATOR ] BOW: Primary data validation passed. [2025-10-20T17:40:47.134157] [INFO ] [MAIN ] BOW: Fetched primary data is valid. [2025-10-20T17:40:47.134218] [DEBUG] [LOCAL ] BOW: Saving primary data. [2025-10-20T17:40:47.134951] [DEBUG] [LOCAL ] BOW: Trimmed 32083 records to 22961 (keeping 200 days of data) [2025-10-20T17:40:47.265502] [DEBUG] [LOCAL ] BOW: Primary data saved (22961 records). [2025-10-20T17:40:47.265620] [INFO ] [MAIN ] BOW: Full history saved. [2025-10-20T17:40:47.265725] [INFO ] [MAIN ] Update Progress: 325/2367 symbols completed. BOW took 21.2s. Estimated remaining: 4h 15m 49s. Rate: 8.0 symbols/min [2025-10-20T17:40:47.432948] [INFO ] [MAIN ] BR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:47.452572] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-04. [2025-10-20T17:40:47.467235] [DEBUG] [LOCAL ] BR: Loaded 35340 existing records from primary storage. [2025-10-20T17:40:47.467493] [INFO ] [MAIN ] BR: Loaded 35340 existing records. [2025-10-20T17:40:47.645542] [WARN ] [MAIN ] BR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:47.645781] [INFO ] [MAIN ] BR: Wiping existing primary data before full refresh. [2025-10-20T17:40:47.645830] [INFO ] [LOCAL ] BR: Starting deletion of primary data. [2025-10-20T17:40:47.646190] [DEBUG] [LOCAL ] BR: Deleted primary data file. [2025-10-20T17:40:47.646245] [INFO ] [MAIN ] BR: Starting full primary refresh. [2025-10-20T17:40:47.997569] [INFO ] [REMOTE ] BR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:48.251365] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-01. [2025-10-20T17:40:48.500163] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T17:40:48.507953] [INFO ] [REMOTE ] BORR: Full fetch complete (74559 records). [2025-10-20T17:40:48.508088] [DEBUG] [VALIDATOR ] BORR: Validating primary data. [2025-10-20T17:40:48.922580] [DEBUG] [VALIDATOR ] BORR: All expected trading days are present. [2025-10-20T17:40:48.922687] [DEBUG] [VALIDATOR ] BORR: Primary data validation passed. [2025-10-20T17:40:48.923483] [INFO ] [MAIN ] BORR: Fetched primary data is valid. [2025-10-20T17:40:48.923525] [DEBUG] [LOCAL ] BORR: Saving primary data. [2025-10-20T17:40:48.924885] [DEBUG] [LOCAL ] BORR: Trimmed 74559 records to 51118 (keeping 200 days of data) [2025-10-20T17:40:49.076934] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-05. [2025-10-20T17:40:49.208241] [DEBUG] [LOCAL ] BORR: Primary data saved (51118 records). [2025-10-20T17:40:49.208424] [INFO ] [MAIN ] BORR: Full history saved. [2025-10-20T17:40:49.208507] [INFO ] [MAIN ] Update Progress: 326/2367 symbols completed. BORR took 30.2s. Estimated remaining: 4h 15m 6s. Rate: 8.0 symbols/min [2025-10-20T17:40:49.385584] [INFO ] [MAIN ] BRBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:49.386007] [DEBUG] [LOCAL ] BRBR: No existing data file found in primary storage. [2025-10-20T17:40:49.386078] [WARN ] [MAIN ] BRBR: No existing data found, performing full refresh instead. [2025-10-20T17:40:49.386304] [INFO ] [MAIN ] BRBR: Wiping existing primary data before full refresh. [2025-10-20T17:40:49.386462] [INFO ] [LOCAL ] BRBR: Starting deletion of primary data. [2025-10-20T17:40:49.386540] [DEBUG] [LOCAL ] BRBR: No primary data file found to delete. [2025-10-20T17:40:49.386586] [INFO ] [MAIN ] BRBR: Starting full primary refresh. [2025-10-20T17:40:49.620419] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T17:40:49.798212] [INFO ] [REMOTE ] BRBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:49.831745] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-04 (2.38 seconds). [2025-10-20T17:40:49.851346] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-01. [2025-10-20T17:40:50.651350] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-05. [2025-10-20T17:40:50.868728] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T17:40:51.458876] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-06. [2025-10-20T17:40:51.462045] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:40:52.251442] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-02. [2025-10-20T17:40:52.605095] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T17:40:53.051409] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-02. [2025-10-20T17:40:53.115989] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:40:53.224781] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T17:40:53.851453] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-07. [2025-10-20T17:40:54.600809] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:40:54.651350] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-06. [2025-10-20T17:40:55.451405] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-03. [2025-10-20T17:40:55.491523] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T17:40:56.251441] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-03. [2025-10-20T17:40:56.680813] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T17:40:57.051520] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-08. [2025-10-20T17:40:57.090828] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-06 (2.44 seconds). [2025-10-20T17:40:57.711215] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T17:40:57.851367] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-04. [2025-10-20T17:40:58.397441] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:40:58.651376] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-07. [2025-10-20T17:40:59.451418] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-04. [2025-10-20T17:40:59.480678] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T17:41:00.251382] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-05. [2025-10-20T17:41:00.914189] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T17:41:01.051415] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-08. [2025-10-20T17:41:01.159795] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:41:01.851407] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-09. [2025-10-20T17:41:01.975683] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:41:02.651437] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-05. [2025-10-20T17:41:02.836612] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T17:41:03.451368] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-06. [2025-10-20T17:41:03.551506] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:41:04.069997] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T17:41:04.251390] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-10. [2025-10-20T17:41:05.051414] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-06. [2025-10-20T17:41:05.265002] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T17:41:05.851433] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-09. [2025-10-20T17:41:06.083792] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T17:41:06.091687] [INFO ] [REMOTE ] BOX: Full fetch complete (72341 records). [2025-10-20T17:41:06.091869] [DEBUG] [VALIDATOR ] BOX: Validating primary data. [2025-10-20T17:41:06.516313] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T17:41:06.544361] [DEBUG] [VALIDATOR ] BOX: All expected trading days are present. [2025-10-20T17:41:06.544561] [DEBUG] [VALIDATOR ] BOX: Primary data validation passed. [2025-10-20T17:41:06.545387] [INFO ] [MAIN ] BOX: Fetched primary data is valid. [2025-10-20T17:41:06.545489] [DEBUG] [LOCAL ] BOX: Saving primary data. [2025-10-20T17:41:06.546977] [DEBUG] [LOCAL ] BOX: Trimmed 72341 records to 50187 (keeping 200 days of data) [2025-10-20T17:41:06.658987] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-07. [2025-10-20T17:41:06.847578] [DEBUG] [LOCAL ] BOX: Primary data saved (50187 records). [2025-10-20T17:41:06.847798] [INFO ] [MAIN ] BOX: Full history saved. [2025-10-20T17:41:06.847895] [INFO ] [MAIN ] Update Progress: 327/2367 symbols completed. BOX took 37.9s. Estimated remaining: 4h 16m 2s. Rate: 8.0 symbols/min [2025-10-20T17:41:07.015961] [INFO ] [MAIN ] BRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:07.043021] [DEBUG] [LOCAL ] BRC: Loaded 18893 existing records from primary storage. [2025-10-20T17:41:07.043242] [INFO ] [MAIN ] BRC: Loaded 18893 existing records. [2025-10-20T17:41:07.218164] [WARN ] [MAIN ] BRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:07.218428] [INFO ] [MAIN ] BRC: Wiping existing primary data before full refresh. [2025-10-20T17:41:07.218487] [INFO ] [LOCAL ] BRC: Starting deletion of primary data. [2025-10-20T17:41:07.218712] [DEBUG] [LOCAL ] BRC: Deleted primary data file. [2025-10-20T17:41:07.218769] [INFO ] [MAIN ] BRC: Starting full primary refresh. [2025-10-20T17:41:07.459325] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-07. [2025-10-20T17:41:07.583966] [INFO ] [REMOTE ] BRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:08.123290] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-09 (2.27 seconds). [2025-10-20T17:41:08.159112] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:41:08.251372] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-08. [2025-10-20T17:41:09.051377] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-01. [2025-10-20T17:41:09.342971] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T17:41:09.817860] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T17:41:09.851386] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-10. [2025-10-20T17:41:10.270007] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T17:41:10.651411] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-09. [2025-10-20T17:41:11.451402] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-08. [2025-10-20T17:41:12.011075] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-10 (2.16 seconds). [2025-10-20T17:41:12.019442] [INFO ] [REMOTE ] BP: Full fetch complete (77882 records). [2025-10-20T17:41:12.019634] [DEBUG] [VALIDATOR ] BP: Validating primary data. [2025-10-20T17:41:12.257245] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-02. [2025-10-20T17:41:12.366247] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T17:41:12.462353] [DEBUG] [VALIDATOR ] BP: All expected trading days are present. [2025-10-20T17:41:12.462434] [DEBUG] [VALIDATOR ] BP: Primary data validation passed. [2025-10-20T17:41:12.463287] [INFO ] [MAIN ] BP: Fetched primary data is valid. [2025-10-20T17:41:12.463360] [DEBUG] [LOCAL ] BP: Saving primary data. [2025-10-20T17:41:12.465034] [DEBUG] [LOCAL ] BP: Trimmed 77882 records to 53708 (keeping 200 days of data) [2025-10-20T17:41:12.796103] [DEBUG] [LOCAL ] BP: Primary data saved (53708 records). [2025-10-20T17:41:12.796306] [INFO ] [MAIN ] BP: Full history saved. [2025-10-20T17:41:12.796824] [INFO ] [MAIN ] Update Progress: 328/2367 symbols completed. BP took 43.7s. Estimated remaining: 4h 15m 45s. Rate: 8.0 symbols/min [2025-10-20T17:41:12.979590] [INFO ] [MAIN ] BRCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:13.015294] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T17:41:13.061943] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-09. [2025-10-20T17:41:13.066448] [DEBUG] [LOCAL ] BRCC: Loaded 40784 existing records from primary storage. [2025-10-20T17:41:13.067170] [INFO ] [MAIN ] BRCC: Loaded 40784 existing records. [2025-10-20T17:41:13.160835] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T17:41:13.287035] [WARN ] [MAIN ] BRCC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:13.287305] [INFO ] [MAIN ] BRCC: Wiping existing primary data before full refresh. [2025-10-20T17:41:13.287356] [INFO ] [LOCAL ] BRCC: Starting deletion of primary data. [2025-10-20T17:41:13.287579] [DEBUG] [LOCAL ] BRCC: Deleted primary data file. [2025-10-20T17:41:13.287617] [INFO ] [MAIN ] BRCC: Starting full primary refresh. [2025-10-20T17:41:13.644132] [INFO ] [REMOTE ] BRCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:13.851400] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-10. [2025-10-20T17:41:14.651458] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-03. [2025-10-20T17:41:14.765371] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:41:15.391120] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T17:41:15.398870] [INFO ] [REMOTE ] BR: Full fetch complete (50107 records). [2025-10-20T17:41:15.399150] [DEBUG] [VALIDATOR ] BR: Validating primary data. [2025-10-20T17:41:15.459612] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-01. [2025-10-20T17:41:15.813911] [DEBUG] [VALIDATOR ] BR: All expected trading days are present. [2025-10-20T17:41:15.814044] [DEBUG] [VALIDATOR ] BR: Primary data validation passed. [2025-10-20T17:41:15.814848] [INFO ] [MAIN ] BR: Fetched primary data is valid. [2025-10-20T17:41:15.814932] [DEBUG] [LOCAL ] BR: Saving primary data. [2025-10-20T17:41:15.832702] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T17:41:15.838353] [DEBUG] [LOCAL ] BR: Trimmed 50107 records to 34741 (keeping 200 days of data) [2025-10-20T17:41:16.063941] [DEBUG] [LOCAL ] BR: Primary data saved (34741 records). [2025-10-20T17:41:16.064183] [INFO ] [MAIN ] BR: Full history saved. [2025-10-20T17:41:16.064287] [INFO ] [MAIN ] Update Progress: 329/2367 symbols completed. BR took 28.8s. Estimated remaining: 4h 15m 11s. Rate: 8.0 symbols/min [2025-10-20T17:41:16.246464] [INFO ] [MAIN ] BRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:16.251344] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-04. [2025-10-20T17:41:16.300635] [DEBUG] [LOCAL ] BRO: Loaded 52839 existing records from primary storage. [2025-10-20T17:41:16.300990] [INFO ] [MAIN ] BRO: Loaded 52839 existing records. [2025-10-20T17:41:16.491429] [WARN ] [MAIN ] BRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:16.491685] [INFO ] [MAIN ] BRO: Wiping existing primary data before full refresh. [2025-10-20T17:41:16.491747] [INFO ] [LOCAL ] BRO: Starting deletion of primary data. [2025-10-20T17:41:16.491983] [DEBUG] [LOCAL ] BRO: Deleted primary data file. [2025-10-20T17:41:16.492041] [INFO ] [MAIN ] BRO: Starting full primary refresh. [2025-10-20T17:41:16.899903] [INFO ] [REMOTE ] BRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:17.057414] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-01. [2025-10-20T17:41:17.063721] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:41:17.127830] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T17:41:17.851391] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-02. [2025-10-20T17:41:18.651351] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-05. [2025-10-20T17:41:18.844848] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:41:19.189200] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T17:41:19.374365] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T17:41:19.451408] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-06. [2025-10-20T17:41:20.251390] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-03. [2025-10-20T17:41:20.744221] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:41:21.051400] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-02. [2025-10-20T17:41:21.425468] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T17:41:21.851347] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-10. [2025-10-20T17:41:22.326192] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T17:41:22.651420] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-04. [2025-10-20T17:41:23.451450] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-07. [2025-10-20T17:41:23.772023] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:41:23.784448] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T17:41:23.793772] [INFO ] [REMOTE ] BRBR: Full fetch complete (73114 records). [2025-10-20T17:41:23.794569] [DEBUG] [VALIDATOR ] BRBR: Validating primary data. [2025-10-20T17:41:24.202618] [DEBUG] [VALIDATOR ] BRBR: All expected trading days are present. [2025-10-20T17:41:24.202801] [DEBUG] [VALIDATOR ] BRBR: Primary data validation passed. [2025-10-20T17:41:24.203806] [INFO ] [MAIN ] BRBR: Fetched primary data is valid. [2025-10-20T17:41:24.203879] [DEBUG] [LOCAL ] BRBR: Saving primary data. [2025-10-20T17:41:24.205473] [DEBUG] [LOCAL ] BRBR: Trimmed 73114 records to 51831 (keeping 200 days of data) [2025-10-20T17:41:24.256523] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-03. [2025-10-20T17:41:24.429120] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:41:24.536899] [DEBUG] [LOCAL ] BRBR: Primary data saved (51831 records). [2025-10-20T17:41:24.537042] [INFO ] [MAIN ] BRBR: Full history saved. [2025-10-20T17:41:24.537112] [INFO ] [MAIN ] Update Progress: 330/2367 symbols completed. BRBR took 35.3s. Estimated remaining: 4h 15m 9s. Rate: 8.0 symbols/min [2025-10-20T17:41:24.725370] [INFO ] [MAIN ] BROS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:24.725589] [DEBUG] [LOCAL ] BROS: No existing data file found in primary storage. [2025-10-20T17:41:24.725663] [WARN ] [MAIN ] BROS: No existing data found, performing full refresh instead. [2025-10-20T17:41:24.725782] [INFO ] [MAIN ] BROS: Wiping existing primary data before full refresh. [2025-10-20T17:41:24.725823] [INFO ] [LOCAL ] BROS: Starting deletion of primary data. [2025-10-20T17:41:24.725867] [DEBUG] [LOCAL ] BROS: No primary data file found to delete. [2025-10-20T17:41:24.725921] [INFO ] [MAIN ] BROS: Starting full primary refresh. [2025-10-20T17:41:25.062543] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-08. [2025-10-20T17:41:25.081672] [INFO ] [REMOTE ] BROS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:25.851368] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-01. [2025-10-20T17:41:26.152988] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T17:41:26.190585] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:41:26.651380] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-04. [2025-10-20T17:41:27.451363] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-09. [2025-10-20T17:41:27.641786] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:41:28.223115] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T17:41:28.245524] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T17:41:28.251318] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-10. [2025-10-20T17:41:28.993905] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T17:41:28.996922] [INFO ] [REMOTE ] BRC: Full fetch complete (28472 records). [2025-10-20T17:41:28.996990] [DEBUG] [VALIDATOR ] BRC: Validating primary data. [2025-10-20T17:41:29.057824] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-05. [2025-10-20T17:41:29.367282] [DEBUG] [VALIDATOR ] BRC: All expected trading days are present. [2025-10-20T17:41:29.367380] [DEBUG] [VALIDATOR ] BRC: Primary data validation passed. [2025-10-20T17:41:29.367854] [INFO ] [MAIN ] BRC: Fetched primary data is valid. [2025-10-20T17:41:29.367918] [DEBUG] [LOCAL ] BRC: Saving primary data. [2025-10-20T17:41:29.368785] [DEBUG] [LOCAL ] BRC: Trimmed 28472 records to 18533 (keeping 200 days of data) [2025-10-20T17:41:29.478899] [DEBUG] [LOCAL ] BRC: Primary data saved (18533 records). [2025-10-20T17:41:29.479040] [INFO ] [MAIN ] BRC: Full history saved. [2025-10-20T17:41:29.479119] [INFO ] [MAIN ] Update Progress: 331/2367 symbols completed. BRC took 22.6s. Estimated remaining: 4h 14m 46s. Rate: 8.0 symbols/min [2025-10-20T17:41:29.651800] [INFO ] [MAIN ] BRSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:29.651967] [DEBUG] [LOCAL ] BRSL: No existing data file found in primary storage. [2025-10-20T17:41:29.652004] [WARN ] [MAIN ] BRSL: No existing data found, performing full refresh instead. [2025-10-20T17:41:29.652134] [INFO ] [MAIN ] BRSL: Wiping existing primary data before full refresh. [2025-10-20T17:41:29.652211] [INFO ] [LOCAL ] BRSL: Starting deletion of primary data. [2025-10-20T17:41:29.652272] [DEBUG] [LOCAL ] BRSL: No primary data file found to delete. [2025-10-20T17:41:29.652305] [INFO ] [MAIN ] BRSL: Starting full primary refresh. [2025-10-20T17:41:29.857788] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-02. [2025-10-20T17:41:30.011741] [INFO ] [REMOTE ] BRSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:30.661327] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-01. [2025-10-20T17:41:30.665858] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:41:30.862913] [DEBUG] [REMOTE ] BRSL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:41:30.863220] [WARN ] [REMOTE ] BRSL: No data returned for month 2025-01 [2025-10-20T17:41:30.863311] [FATAL] [REMOTE ] BRSL: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:41:30.863398] [FATAL] [MAIN ] BRSL: Failed to fetch primary data. [2025-10-20T17:41:30.863528] [WARN ] [MAIN ] BRSL: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:41:30.863601] [INFO ] [LOCAL ] BRSL: Starting deletion of primary data. [2025-10-20T17:41:30.863697] [DEBUG] [LOCAL ] BRSL: No primary data file found to delete. [2025-10-20T17:41:30.863751] [INFO ] [LOCAL ] BRSL: Starting deletion of derivative data. [2025-10-20T17:41:30.863794] [DEBUG] [LOCAL ] BRSL: No derivative data file found to delete. [2025-10-20T17:41:30.863840] [INFO ] [MAIN ] Update Progress: 332/2367 symbols completed. BRSL took 1.4s. Estimated remaining: 4h 14m 1s. Rate: 8.0 symbols/min [2025-10-20T17:41:31.037932] [INFO ] [MAIN ] BRSP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:31.038120] [DEBUG] [LOCAL ] BRSP: No existing data file found in primary storage. [2025-10-20T17:41:31.038158] [WARN ] [MAIN ] BRSP: No existing data found, performing full refresh instead. [2025-10-20T17:41:31.038295] [INFO ] [MAIN ] BRSP: Wiping existing primary data before full refresh. [2025-10-20T17:41:31.038351] [INFO ] [LOCAL ] BRSP: Starting deletion of primary data. [2025-10-20T17:41:31.038395] [DEBUG] [LOCAL ] BRSP: No primary data file found to delete. [2025-10-20T17:41:31.038427] [INFO ] [MAIN ] BRSP: Starting full primary refresh. [2025-10-20T17:41:31.397312] [INFO ] [REMOTE ] BRSP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:31.451352] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-01. [2025-10-20T17:41:31.549108] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:41:32.251380] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-06. [2025-10-20T17:41:32.873406] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:41:33.051420] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-05. [2025-10-20T17:41:33.622954] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:41:33.851437] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-03. [2025-10-20T17:41:34.651425] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-02. [2025-10-20T17:41:34.879606] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T17:41:35.415913] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:41:35.451353] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-04. [2025-10-20T17:41:35.865462] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:41:36.251440] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-06. [2025-10-20T17:41:37.051400] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-07. [2025-10-20T17:41:37.092834] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:41:37.851452] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-03. [2025-10-20T17:41:38.149461] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T17:41:38.200721] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T17:41:38.651404] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-08. [2025-10-20T17:41:39.144376] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T17:41:39.451438] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-05. [2025-10-20T17:41:40.241401] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T17:41:40.251704] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-04. [2025-10-20T17:41:41.051383] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-07. [2025-10-20T17:41:41.138319] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T17:41:41.839739] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:41:41.851342] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-05. [2025-10-20T17:41:42.651433] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-06. [2025-10-20T17:41:42.781474] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:41:42.988022] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:41:43.451420] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-08. [2025-10-20T17:41:44.251370] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-06. [2025-10-20T17:41:44.385438] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:41:44.859157] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T17:41:45.051373] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-07. [2025-10-20T17:41:45.097592] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:41:45.851468] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-09. [2025-10-20T17:41:46.651374] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-09. [2025-10-20T17:41:47.068627] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T17:41:47.456479] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-08. [2025-10-20T17:41:47.478761] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T17:41:47.860222] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T17:41:48.251435] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-10. [2025-10-20T17:41:49.051376] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-07. [2025-10-20T17:41:49.211836] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T17:41:49.803525] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T17:41:49.809363] [INFO ] [REMOTE ] BRCC: Full fetch complete (57453 records). [2025-10-20T17:41:49.809503] [DEBUG] [VALIDATOR ] BRCC: Validating primary data. [2025-10-20T17:41:49.863602] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-09. [2025-10-20T17:41:50.219911] [DEBUG] [VALIDATOR ] BRCC: All expected trading days are present. [2025-10-20T17:41:50.220054] [DEBUG] [VALIDATOR ] BRCC: Primary data validation passed. [2025-10-20T17:41:50.221013] [INFO ] [MAIN ] BRCC: Fetched primary data is valid. [2025-10-20T17:41:50.221109] [DEBUG] [LOCAL ] BRCC: Saving primary data. [2025-10-20T17:41:50.246218] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:41:50.255420] [DEBUG] [LOCAL ] BRCC: Trimmed 57453 records to 40137 (keeping 200 days of data) [2025-10-20T17:41:50.495309] [DEBUG] [LOCAL ] BRCC: Primary data saved (40137 records). [2025-10-20T17:41:50.495483] [INFO ] [MAIN ] BRCC: Full history saved. [2025-10-20T17:41:50.495579] [INFO ] [MAIN ] Update Progress: 333/2367 symbols completed. BRCC took 37.7s. Estimated remaining: 4h 15m 7s. Rate: 8.0 symbols/min [2025-10-20T17:41:50.670145] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-08. [2025-10-20T17:41:50.674582] [INFO ] [MAIN ] BRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:50.682718] [DEBUG] [LOCAL ] BRT: Loaded 5316 existing records from primary storage. [2025-10-20T17:41:50.682947] [INFO ] [MAIN ] BRT: Loaded 5316 existing records. [2025-10-20T17:41:50.869844] [WARN ] [MAIN ] BRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:50.870310] [INFO ] [MAIN ] BRT: Wiping existing primary data before full refresh. [2025-10-20T17:41:50.870367] [INFO ] [LOCAL ] BRT: Starting deletion of primary data. [2025-10-20T17:41:50.870777] [DEBUG] [LOCAL ] BRT: Deleted primary data file. [2025-10-20T17:41:50.870846] [INFO ] [MAIN ] BRT: Starting full primary refresh. [2025-10-20T17:41:51.262604] [INFO ] [REMOTE ] BRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:51.451378] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-01. [2025-10-20T17:41:52.121262] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T17:41:52.125072] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T17:41:52.251362] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-09. [2025-10-20T17:41:52.347349] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-09 (2.48 seconds). [2025-10-20T17:41:53.051466] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-10. [2025-10-20T17:41:53.783513] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:41:53.851476] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-10. [2025-10-20T17:41:54.651418] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-02. [2025-10-20T17:41:55.101145] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T17:41:55.144571] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T17:41:55.152802] [INFO ] [REMOTE ] BROS: Full fetch complete (77271 records). [2025-10-20T17:41:55.152940] [DEBUG] [VALIDATOR ] BROS: Validating primary data. [2025-10-20T17:41:55.452729] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-03. [2025-10-20T17:41:55.563354] [DEBUG] [VALIDATOR ] BROS: All expected trading days are present. [2025-10-20T17:41:55.563445] [DEBUG] [VALIDATOR ] BROS: Primary data validation passed. [2025-10-20T17:41:55.564395] [INFO ] [MAIN ] BROS: Fetched primary data is valid. [2025-10-20T17:41:55.564463] [DEBUG] [LOCAL ] BROS: Saving primary data. [2025-10-20T17:41:55.565970] [DEBUG] [LOCAL ] BROS: Trimmed 77271 records to 53300 (keeping 200 days of data) [2025-10-20T17:41:55.801345] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T17:41:55.814710] [INFO ] [REMOTE ] BRO: Full fetch complete (75054 records). [2025-10-20T17:41:55.818196] [DEBUG] [VALIDATOR ] BRO: Validating primary data. [2025-10-20T17:41:56.116139] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T17:41:56.254115] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-10. [2025-10-20T17:41:56.358579] [DEBUG] [VALIDATOR ] BRO: All expected trading days are present. [2025-10-20T17:41:56.360212] [DEBUG] [VALIDATOR ] BRO: Primary data validation passed. [2025-10-20T17:41:56.361077] [INFO ] [MAIN ] BRO: Fetched primary data is valid. [2025-10-20T17:41:56.364568] [DEBUG] [LOCAL ] BRO: Saving primary data. [2025-10-20T17:41:56.377168] [DEBUG] [LOCAL ] BROS: Primary data saved (53300 records). [2025-10-20T17:41:56.377604] [INFO ] [MAIN ] BROS: Full history saved. [2025-10-20T17:41:56.377767] [INFO ] [MAIN ] Update Progress: 334/2367 symbols completed. BROS took 31.8s. Estimated remaining: 4h 14m 50s. Rate: 8.0 symbols/min [2025-10-20T17:41:56.377883] [DEBUG] [LOCAL ] BRO: Trimmed 75054 records to 52094 (keeping 200 days of data) [2025-10-20T17:41:56.680189] [INFO ] [MAIN ] BRW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:56.723877] [DEBUG] [LOCAL ] BRW: Loaded 21211 existing records from primary storage. [2025-10-20T17:41:56.724146] [INFO ] [MAIN ] BRW: Loaded 21211 existing records. [2025-10-20T17:41:57.006629] [WARN ] [MAIN ] BRW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:57.016934] [INFO ] [MAIN ] BRW: Wiping existing primary data before full refresh. [2025-10-20T17:41:57.018501] [INFO ] [LOCAL ] BRW: Starting deletion of primary data. [2025-10-20T17:41:57.028292] [DEBUG] [LOCAL ] BRW: Deleted primary data file. [2025-10-20T17:41:57.028739] [INFO ] [MAIN ] BRW: Starting full primary refresh. [2025-10-20T17:41:57.061471] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-04. [2025-10-20T17:41:57.450994] [DEBUG] [LOCAL ] BRO: Primary data saved (52094 records). [2025-10-20T17:41:57.451254] [INFO ] [MAIN ] BRO: Full history saved. [2025-10-20T17:41:57.452889] [INFO ] [MAIN ] Update Progress: 335/2367 symbols completed. BRO took 41.4s. Estimated remaining: 4h 14m 3s. Rate: 8.0 symbols/min [2025-10-20T17:41:57.637709] [INFO ] [REMOTE ] BRW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:57.715666] [INFO ] [MAIN ] BRX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:57.735874] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T17:41:57.743383] [INFO ] [REMOTE ] BRSP: Full fetch complete (54172 records). [2025-10-20T17:41:57.743583] [DEBUG] [VALIDATOR ] BRSP: Validating primary data. [2025-10-20T17:41:57.857566] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-01. [2025-10-20T17:41:57.875939] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T17:41:58.125214] [DEBUG] [LOCAL ] BRX: Loaded 52572 existing records from primary storage. [2025-10-20T17:41:58.136748] [INFO ] [MAIN ] BRX: Loaded 52572 existing records. [2025-10-20T17:41:58.343014] [DEBUG] [VALIDATOR ] BRSP: All expected trading days are present. [2025-10-20T17:41:58.348305] [DEBUG] [VALIDATOR ] BRSP: Primary data validation passed. [2025-10-20T17:41:58.354212] [INFO ] [MAIN ] BRSP: Fetched primary data is valid. [2025-10-20T17:41:58.359478] [DEBUG] [LOCAL ] BRSP: Saving primary data. [2025-10-20T17:41:58.416511] [DEBUG] [LOCAL ] BRSP: Trimmed 54172 records to 36685 (keeping 200 days of data) [2025-10-20T17:41:58.420949] [WARN ] [MAIN ] BRX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:58.421783] [INFO ] [MAIN ] BRX: Wiping existing primary data before full refresh. [2025-10-20T17:41:58.421861] [INFO ] [LOCAL ] BRX: Starting deletion of primary data. [2025-10-20T17:41:58.422495] [DEBUG] [LOCAL ] BRX: Deleted primary data file. [2025-10-20T17:41:58.422753] [INFO ] [MAIN ] BRX: Starting full primary refresh. [2025-10-20T17:41:58.654870] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-05. [2025-10-20T17:41:59.024069] [INFO ] [REMOTE ] BRX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:59.056425] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:41:59.079680] [DEBUG] [LOCAL ] BRSP: Primary data saved (36685 records). [2025-10-20T17:41:59.079864] [INFO ] [MAIN ] BRSP: Full history saved. [2025-10-20T17:41:59.079953] [INFO ] [MAIN ] Update Progress: 336/2367 symbols completed. BRSP took 28.2s. Estimated remaining: 4h 13m 20s. Rate: 8.0 symbols/min [2025-10-20T17:41:59.267194] [INFO ] [MAIN ] BSAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:59.300541] [DEBUG] [LOCAL ] BSAC: Loaded 33351 existing records from primary storage. [2025-10-20T17:41:59.300942] [INFO ] [MAIN ] BSAC: Loaded 33351 existing records. [2025-10-20T17:41:59.411516] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T17:41:59.458197] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-01. [2025-10-20T17:41:59.510306] [WARN ] [MAIN ] BSAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:59.510600] [INFO ] [MAIN ] BSAC: Wiping existing primary data before full refresh. [2025-10-20T17:41:59.510715] [INFO ] [LOCAL ] BSAC: Starting deletion of primary data. [2025-10-20T17:41:59.511032] [DEBUG] [LOCAL ] BSAC: Deleted primary data file. [2025-10-20T17:41:59.511093] [INFO ] [MAIN ] BSAC: Starting full primary refresh. [2025-10-20T17:41:59.890862] [INFO ] [REMOTE ] BSAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:00.251413] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-01. [2025-10-20T17:42:01.051403] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-06. [2025-10-20T17:42:01.174852] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T17:42:01.492356] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T17:42:01.563777] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T17:42:01.851392] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-02. [2025-10-20T17:42:02.651534] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-07. [2025-10-20T17:42:03.008602] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:42:03.412911] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T17:42:03.451399] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-03. [2025-10-20T17:42:04.251357] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-08. [2025-10-20T17:42:05.051424] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-02. [2025-10-20T17:42:05.410069] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T17:42:05.572328] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T17:42:05.851438] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-02. [2025-10-20T17:42:06.327086] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T17:42:06.651425] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-09. [2025-10-20T17:42:07.228404] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T17:42:07.451450] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-10. [2025-10-20T17:42:08.251425] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-03. [2025-10-20T17:42:08.282498] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T17:42:08.284433] [INFO ] [REMOTE ] BRT: Full fetch complete (7077 records). [2025-10-20T17:42:08.284540] [DEBUG] [VALIDATOR ] BRT: Validating primary data. [2025-10-20T17:42:08.569209] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-02 (2.72 seconds). [2025-10-20T17:42:08.741930] [DEBUG] [VALIDATOR ] BRT: All expected trading days are present. [2025-10-20T17:42:08.742042] [DEBUG] [VALIDATOR ] BRT: Primary data validation passed. [2025-10-20T17:42:08.742303] [INFO ] [MAIN ] BRT: Fetched primary data is valid. [2025-10-20T17:42:08.742346] [DEBUG] [LOCAL ] BRT: Saving primary data. [2025-10-20T17:42:08.743153] [DEBUG] [LOCAL ] BRT: Trimmed 7077 records to 5249 (keeping 200 days of data) [2025-10-20T17:42:08.779733] [DEBUG] [LOCAL ] BRT: Primary data saved (5249 records). [2025-10-20T17:42:08.779917] [INFO ] [MAIN ] BRT: Full history saved. [2025-10-20T17:42:08.780015] [INFO ] [MAIN ] Update Progress: 337/2367 symbols completed. BRT took 18.3s. Estimated remaining: 4h 13m 26s. Rate: 8.0 symbols/min [2025-10-20T17:42:08.971466] [INFO ] [MAIN ] BSBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:09.011518] [DEBUG] [LOCAL ] BSBR: Loaded 36277 existing records from primary storage. [2025-10-20T17:42:09.011988] [INFO ] [MAIN ] BSBR: Loaded 36277 existing records. [2025-10-20T17:42:09.056715] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-03. [2025-10-20T17:42:09.230066] [WARN ] [MAIN ] BSBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:42:09.230603] [INFO ] [MAIN ] BSBR: Wiping existing primary data before full refresh. [2025-10-20T17:42:09.230706] [INFO ] [LOCAL ] BSBR: Starting deletion of primary data. [2025-10-20T17:42:09.231062] [DEBUG] [LOCAL ] BSBR: Deleted primary data file. [2025-10-20T17:42:09.231122] [INFO ] [MAIN ] BSBR: Starting full primary refresh. [2025-10-20T17:42:09.354247] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T17:42:09.629353] [INFO ] [REMOTE ] BSBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:09.851378] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-04. [2025-10-20T17:42:10.613260] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:42:10.651384] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-04. [2025-10-20T17:42:11.117365] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T17:42:11.451403] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-04. [2025-10-20T17:42:12.251384] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-01. [2025-10-20T17:42:12.327497] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T17:42:12.531487] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T17:42:13.051374] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-05. [2025-10-20T17:42:13.458956] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:42:13.851374] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-05. [2025-10-20T17:42:14.116685] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:42:14.651476] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-02. [2025-10-20T17:42:14.736420] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T17:42:15.451453] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-06. [2025-10-20T17:42:15.870469] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:42:16.251451] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-05. [2025-10-20T17:42:16.503484] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T17:42:17.051376] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-06. [2025-10-20T17:42:17.851350] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-03. [2025-10-20T17:42:17.894812] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T17:42:18.283303] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T17:42:18.651335] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-07. [2025-10-20T17:42:18.749574] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T17:42:19.451425] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-07. [2025-10-20T17:42:20.075531] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:42:20.251404] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-04. [2025-10-20T17:42:20.361573] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T17:42:21.051394] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-06. [2025-10-20T17:42:21.832947] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:42:21.851346] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-05. [2025-10-20T17:42:22.610826] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T17:42:22.651415] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-08. [2025-10-20T17:42:23.035365] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T17:42:23.451444] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-06. [2025-10-20T17:42:23.774139] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:42:24.251439] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-08. [2025-10-20T17:42:24.808986] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T17:42:25.051485] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-07. [2025-10-20T17:42:25.111625] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T17:42:25.851413] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-09. [2025-10-20T17:42:26.535213] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:42:26.651412] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-09. [2025-10-20T17:42:27.214928] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T17:42:27.451429] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-07. [2025-10-20T17:42:27.712293] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T17:42:28.251438] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-08. [2025-10-20T17:42:28.828507] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:42:29.051378] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-10. [2025-10-20T17:42:29.550429] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T17:42:29.851378] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-09. [2025-10-20T17:42:30.017048] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T17:42:30.022108] [INFO ] [REMOTE ] BSAC: Full fetch complete (45052 records). [2025-10-20T17:42:30.022225] [DEBUG] [VALIDATOR ] BSAC: Validating primary data. [2025-10-20T17:42:30.389808] [DEBUG] [VALIDATOR ] BSAC: All expected trading days are present. [2025-10-20T17:42:30.389889] [DEBUG] [VALIDATOR ] BSAC: Primary data validation passed. [2025-10-20T17:42:30.390421] [INFO ] [MAIN ] BSAC: Fetched primary data is valid. [2025-10-20T17:42:30.390461] [DEBUG] [LOCAL ] BSAC: Saving primary data. [2025-10-20T17:42:30.391574] [DEBUG] [LOCAL ] BSAC: Trimmed 45052 records to 32970 (keeping 200 days of data) [2025-10-20T17:42:30.580096] [DEBUG] [LOCAL ] BSAC: Primary data saved (32970 records). [2025-10-20T17:42:30.580239] [INFO ] [MAIN ] BSAC: Full history saved. [2025-10-20T17:42:30.580714] [INFO ] [MAIN ] Update Progress: 338/2367 symbols completed. BSAC took 31.5s. Estimated remaining: 4h 14m 45s. Rate: 8.0 symbols/min [2025-10-20T17:42:30.658436] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-08. [2025-10-20T17:42:30.758340] [INFO ] [MAIN ] BSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:30.758507] [DEBUG] [LOCAL ] BSL: No existing data file found in primary storage. [2025-10-20T17:42:30.758545] [WARN ] [MAIN ] BSL: No existing data found, performing full refresh instead. [2025-10-20T17:42:30.758683] [INFO ] [MAIN ] BSL: Wiping existing primary data before full refresh. [2025-10-20T17:42:30.758737] [INFO ] [LOCAL ] BSL: Starting deletion of primary data. [2025-10-20T17:42:30.758781] [DEBUG] [LOCAL ] BSL: No primary data file found to delete. [2025-10-20T17:42:30.758814] [INFO ] [MAIN ] BSL: Starting full primary refresh. [2025-10-20T17:42:31.143255] [INFO ] [REMOTE ] BSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:31.158595] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T17:42:31.451374] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-01. [2025-10-20T17:42:32.154598] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:42:32.191866] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:42:32.251449] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-10. [2025-10-20T17:42:33.051412] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-09. [2025-10-20T17:42:33.293440] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T17:42:33.297357] [INFO ] [REMOTE ] BRW: Full fetch complete (30137 records). [2025-10-20T17:42:33.297439] [DEBUG] [VALIDATOR ] BRW: Validating primary data. [2025-10-20T17:42:33.699697] [DEBUG] [VALIDATOR ] BRW: All expected trading days are present. [2025-10-20T17:42:33.699791] [DEBUG] [VALIDATOR ] BRW: Primary data validation passed. [2025-10-20T17:42:33.700390] [INFO ] [MAIN ] BRW: Fetched primary data is valid. [2025-10-20T17:42:33.700465] [DEBUG] [LOCAL ] BRW: Saving primary data. [2025-10-20T17:42:33.701551] [DEBUG] [LOCAL ] BRW: Trimmed 30137 records to 20943 (keeping 200 days of data) [2025-10-20T17:42:33.834753] [DEBUG] [LOCAL ] BRW: Primary data saved (20943 records). [2025-10-20T17:42:33.834886] [INFO ] [MAIN ] BRW: Full history saved. [2025-10-20T17:42:33.835029] [INFO ] [MAIN ] Update Progress: 339/2367 symbols completed. BRW took 37.5s. Estimated remaining: 4h 14m 11s. Rate: 8.0 symbols/min [2025-10-20T17:42:33.859915] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-02. [2025-10-20T17:42:34.043521] [INFO ] [MAIN ] BSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:34.075234] [DEBUG] [LOCAL ] BSM: Loaded 32822 existing records from primary storage. [2025-10-20T17:42:34.075546] [INFO ] [MAIN ] BSM: Loaded 32822 existing records. [2025-10-20T17:42:34.283119] [WARN ] [MAIN ] BSM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:42:34.283338] [INFO ] [MAIN ] BSM: Wiping existing primary data before full refresh. [2025-10-20T17:42:34.283385] [INFO ] [LOCAL ] BSM: Starting deletion of primary data. [2025-10-20T17:42:34.283622] [DEBUG] [LOCAL ] BSM: Deleted primary data file. [2025-10-20T17:42:34.283705] [INFO ] [MAIN ] BSM: Starting full primary refresh. [2025-10-20T17:42:34.677439] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-10. [2025-10-20T17:42:34.677342] [INFO ] [REMOTE ] BSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:34.679944] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T17:42:34.772136] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T17:42:35.451370] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-03. [2025-10-20T17:42:35.899473] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T17:42:35.906583] [INFO ] [REMOTE ] BSBR: Full fetch complete (51147 records). [2025-10-20T17:42:35.906784] [DEBUG] [VALIDATOR ] BSBR: Validating primary data. [2025-10-20T17:42:36.035117] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T17:42:36.256558] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-04. [2025-10-20T17:42:36.324956] [DEBUG] [VALIDATOR ] BSBR: All expected trading days are present. [2025-10-20T17:42:36.325044] [DEBUG] [VALIDATOR ] BSBR: Primary data validation passed. [2025-10-20T17:42:36.325692] [INFO ] [MAIN ] BSBR: Fetched primary data is valid. [2025-10-20T17:42:36.325759] [DEBUG] [LOCAL ] BSBR: Saving primary data. [2025-10-20T17:42:36.327007] [DEBUG] [LOCAL ] BSBR: Trimmed 51147 records to 35912 (keeping 200 days of data) [2025-10-20T17:42:36.533531] [DEBUG] [LOCAL ] BSBR: Primary data saved (35912 records). [2025-10-20T17:42:36.533725] [INFO ] [MAIN ] BSBR: Full history saved. [2025-10-20T17:42:36.533826] [INFO ] [MAIN ] Update Progress: 340/2367 symbols completed. BSBR took 27.8s. Estimated remaining: 4h 13m 35s. Rate: 8.0 symbols/min [2025-10-20T17:42:36.715903] [INFO ] [MAIN ] BST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:36.716130] [DEBUG] [LOCAL ] BST: No existing data file found in primary storage. [2025-10-20T17:42:36.716194] [WARN ] [MAIN ] BST: No existing data found, performing full refresh instead. [2025-10-20T17:42:36.716346] [INFO ] [MAIN ] BST: Wiping existing primary data before full refresh. [2025-10-20T17:42:36.716417] [INFO ] [LOCAL ] BST: Starting deletion of primary data. [2025-10-20T17:42:36.716489] [DEBUG] [LOCAL ] BST: No primary data file found to delete. [2025-10-20T17:42:36.716549] [INFO ] [MAIN ] BST: Starting full primary refresh. [2025-10-20T17:42:37.062916] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-10. [2025-10-20T17:42:37.106255] [INFO ] [REMOTE ] BST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:37.116288] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:42:37.851409] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-01. [2025-10-20T17:42:38.651468] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-01. [2025-10-20T17:42:38.866017] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T17:42:38.874787] [INFO ] [REMOTE ] BRX: Full fetch complete (75144 records). [2025-10-20T17:42:38.874994] [DEBUG] [VALIDATOR ] BRX: Validating primary data. [2025-10-20T17:42:39.214923] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T17:42:39.322769] [DEBUG] [VALIDATOR ] BRX: All expected trading days are present. [2025-10-20T17:42:39.322899] [DEBUG] [VALIDATOR ] BRX: Primary data validation passed. [2025-10-20T17:42:39.323920] [INFO ] [MAIN ] BRX: Fetched primary data is valid. [2025-10-20T17:42:39.323989] [DEBUG] [LOCAL ] BRX: Saving primary data. [2025-10-20T17:42:39.326045] [DEBUG] [LOCAL ] BRX: Trimmed 75144 records to 51836 (keeping 200 days of data) [2025-10-20T17:42:39.457961] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-05. [2025-10-20T17:42:39.669532] [DEBUG] [LOCAL ] BRX: Primary data saved (51836 records). [2025-10-20T17:42:39.669773] [INFO ] [MAIN ] BRX: Full history saved. [2025-10-20T17:42:39.669885] [INFO ] [MAIN ] Update Progress: 341/2367 symbols completed. BRX took 42.2s. Estimated remaining: 4h 13m 2s. Rate: 8.0 symbols/min [2025-10-20T17:42:39.865373] [INFO ] [MAIN ] BSTZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:39.898617] [DEBUG] [LOCAL ] BSTZ: Loaded 25243 existing records from primary storage. [2025-10-20T17:42:39.899029] [INFO ] [MAIN ] BSTZ: Loaded 25243 existing records. [2025-10-20T17:42:40.049226] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T17:42:40.142758] [WARN ] [MAIN ] BSTZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:42:40.143042] [INFO ] [MAIN ] BSTZ: Wiping existing primary data before full refresh. [2025-10-20T17:42:40.143093] [INFO ] [LOCAL ] BSTZ: Starting deletion of primary data. [2025-10-20T17:42:40.143348] [DEBUG] [LOCAL ] BSTZ: Deleted primary data file. [2025-10-20T17:42:40.143406] [INFO ] [MAIN ] BSTZ: Starting full primary refresh. [2025-10-20T17:42:40.254083] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-02. [2025-10-20T17:42:40.486940] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T17:42:40.551689] [INFO ] [REMOTE ] BSTZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:41.051339] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-06. [2025-10-20T17:42:41.293528] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T17:42:41.730528] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T17:42:41.851400] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-01. [2025-10-20T17:42:42.651396] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-03. [2025-10-20T17:42:43.188440] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:42:43.451357] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-07. [2025-10-20T17:42:43.851777] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T17:42:44.251423] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-04. [2025-10-20T17:42:44.375238] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T17:42:45.051427] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-08. [2025-10-20T17:42:45.350134] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T17:42:45.696991] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T17:42:45.851380] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-09. [2025-10-20T17:42:46.612952] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T17:42:46.651369] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-02. [2025-10-20T17:42:47.451432] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-02. [2025-10-20T17:42:47.700472] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T17:42:48.251432] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-10. [2025-10-20T17:42:48.801211] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T17:42:49.051374] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-03. [2025-10-20T17:42:49.060883] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T17:42:49.063758] [INFO ] [REMOTE ] BSL: Full fetch complete (13050 records). [2025-10-20T17:42:49.063862] [DEBUG] [VALIDATOR ] BSL: Validating primary data. [2025-10-20T17:42:49.460220] [DEBUG] [VALIDATOR ] BSL: All expected trading days are present. [2025-10-20T17:42:49.460330] [DEBUG] [VALIDATOR ] BSL: Primary data validation passed. [2025-10-20T17:42:49.460899] [INFO ] [MAIN ] BSL: Fetched primary data is valid. [2025-10-20T17:42:49.460965] [DEBUG] [LOCAL ] BSL: Saving primary data. [2025-10-20T17:42:49.461536] [DEBUG] [LOCAL ] BSL: Trimmed 13050 records to 8235 (keeping 200 days of data) [2025-10-20T17:42:49.520451] [DEBUG] [LOCAL ] BSL: Primary data saved (8235 records). [2025-10-20T17:42:49.520745] [INFO ] [MAIN ] BSL: Full history saved. [2025-10-20T17:42:49.520940] [INFO ] [MAIN ] Update Progress: 342/2367 symbols completed. BSL took 18.9s. Estimated remaining: 4h 13m 8s. Rate: 8.0 symbols/min [2025-10-20T17:42:49.721246] [INFO ] [MAIN ] BSX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:49.786124] [DEBUG] [LOCAL ] BSX: Loaded 54468 existing records from primary storage. [2025-10-20T17:42:49.786733] [INFO ] [MAIN ] BSX: Loaded 54468 existing records. [2025-10-20T17:42:49.860433] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-03. [2025-10-20T17:42:49.993880] [WARN ] [MAIN ] BSX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:42:49.994187] [INFO ] [MAIN ] BSX: Wiping existing primary data before full refresh. [2025-10-20T17:42:49.994241] [INFO ] [LOCAL ] BSX: Starting deletion of primary data. [2025-10-20T17:42:49.994520] [DEBUG] [LOCAL ] BSX: Deleted primary data file. [2025-10-20T17:42:49.994579] [INFO ] [MAIN ] BSX: Starting full primary refresh. [2025-10-20T17:42:50.409125] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T17:42:50.416414] [INFO ] [REMOTE ] BSX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:50.651417] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-04. [2025-10-20T17:42:50.978033] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T17:42:51.451376] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-04. [2025-10-20T17:42:51.978620] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:42:52.251384] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-05. [2025-10-20T17:42:52.600064] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T17:42:53.051409] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-05. [2025-10-20T17:42:53.310878] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T17:42:53.851418] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-05. [2025-10-20T17:42:54.317500] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T17:42:54.651395] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-06. [2025-10-20T17:42:54.734503] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T17:42:55.451375] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-06. [2025-10-20T17:42:55.657069] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:42:56.251379] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-01. [2025-10-20T17:42:56.750680] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T17:42:57.051417] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-06. [2025-10-20T17:42:57.851371] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-07. [2025-10-20T17:42:58.168604] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T17:42:58.211674] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T17:42:58.651413] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-07. [2025-10-20T17:42:58.881248] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T17:42:59.451421] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-07. [2025-10-20T17:42:59.878506] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T17:43:00.251371] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-08. [2025-10-20T17:43:00.929855] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:43:01.051355] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-08. [2025-10-20T17:43:01.356192] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T17:43:01.851447] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-09. [2025-10-20T17:43:02.651410] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-08. [2025-10-20T17:43:02.702967] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:43:02.924552] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T17:43:03.451435] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-02. [2025-10-20T17:43:03.808069] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T17:43:04.251471] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-09. [2025-10-20T17:43:04.921159] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T17:43:05.051331] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-03. [2025-10-20T17:43:05.146632] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T17:43:05.851390] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-10. [2025-10-20T17:43:06.651401] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-09. [2025-10-20T17:43:06.994280] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T17:43:07.285972] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T17:43:07.290103] [INFO ] [REMOTE ] BSTZ: Full fetch complete (36549 records). [2025-10-20T17:43:07.290185] [DEBUG] [VALIDATOR ] BSTZ: Validating primary data. [2025-10-20T17:43:07.456486] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-10. [2025-10-20T17:43:07.724381] [DEBUG] [VALIDATOR ] BSTZ: All expected trading days are present. [2025-10-20T17:43:07.724591] [DEBUG] [VALIDATOR ] BSTZ: Primary data validation passed. [2025-10-20T17:43:07.725264] [INFO ] [MAIN ] BSTZ: Fetched primary data is valid. [2025-10-20T17:43:07.725361] [DEBUG] [LOCAL ] BSTZ: Saving primary data. [2025-10-20T17:43:07.745809] [DEBUG] [LOCAL ] BSTZ: Trimmed 36549 records to 24687 (keeping 200 days of data) [2025-10-20T17:43:07.745692] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T17:43:07.898144] [DEBUG] [LOCAL ] BSTZ: Primary data saved (24687 records). [2025-10-20T17:43:07.898300] [INFO ] [MAIN ] BSTZ: Full history saved. [2025-10-20T17:43:07.898398] [INFO ] [MAIN ] Update Progress: 343/2367 symbols completed. BSTZ took 28.2s. Estimated remaining: 4h 14m 5s. Rate: 8.0 symbols/min [2025-10-20T17:43:08.065295] [INFO ] [MAIN ] BTA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:08.073053] [DEBUG] [LOCAL ] BTA: Loaded 5810 existing records from primary storage. [2025-10-20T17:43:08.073245] [INFO ] [MAIN ] BTA: Loaded 5810 existing records. [2025-10-20T17:43:08.254307] [WARN ] [MAIN ] BTA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:08.254388] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-04. [2025-10-20T17:43:08.255469] [INFO ] [MAIN ] BTA: Wiping existing primary data before full refresh. [2025-10-20T17:43:08.255527] [INFO ] [LOCAL ] BTA: Starting deletion of primary data. [2025-10-20T17:43:08.255741] [DEBUG] [LOCAL ] BTA: Deleted primary data file. [2025-10-20T17:43:08.255797] [INFO ] [MAIN ] BTA: Starting full primary refresh. [2025-10-20T17:43:08.356603] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:43:08.370520] [INFO ] [REMOTE ] BST: Full fetch complete (28416 records). [2025-10-20T17:43:08.381114] [DEBUG] [VALIDATOR ] BST: Validating primary data. [2025-10-20T17:43:08.964532] [INFO ] [REMOTE ] BTA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:09.041815] [DEBUG] [VALIDATOR ] BST: All expected trading days are present. [2025-10-20T17:43:09.041888] [DEBUG] [VALIDATOR ] BST: Primary data validation passed. [2025-10-20T17:43:09.042295] [INFO ] [MAIN ] BST: Fetched primary data is valid. [2025-10-20T17:43:09.042334] [DEBUG] [LOCAL ] BST: Saving primary data. [2025-10-20T17:43:09.043003] [DEBUG] [LOCAL ] BST: Trimmed 28416 records to 18723 (keeping 200 days of data) [2025-10-20T17:43:09.090434] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-10. [2025-10-20T17:43:09.155239] [DEBUG] [LOCAL ] BST: Primary data saved (18723 records). [2025-10-20T17:43:09.155359] [INFO ] [MAIN ] BST: Full history saved. [2025-10-20T17:43:09.155427] [INFO ] [MAIN ] Update Progress: 344/2367 symbols completed. BST took 32.6s. Estimated remaining: 4h 13m 20s. Rate: 8.0 symbols/min [2025-10-20T17:43:09.326960] [INFO ] [MAIN ] BTE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:09.327181] [DEBUG] [LOCAL ] BTE: No existing data file found in primary storage. [2025-10-20T17:43:09.327232] [WARN ] [MAIN ] BTE: No existing data found, performing full refresh instead. [2025-10-20T17:43:09.327422] [INFO ] [MAIN ] BTE: Wiping existing primary data before full refresh. [2025-10-20T17:43:09.327472] [INFO ] [LOCAL ] BTE: Starting deletion of primary data. [2025-10-20T17:43:09.327552] [DEBUG] [LOCAL ] BTE: No primary data file found to delete. [2025-10-20T17:43:09.327627] [INFO ] [MAIN ] BTE: Starting full primary refresh. [2025-10-20T17:43:09.680609] [INFO ] [REMOTE ] BTE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:09.851375] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-01. [2025-10-20T17:43:10.446897] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T17:43:10.469483] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T17:43:10.474054] [INFO ] [REMOTE ] BSM: Full fetch complete (46955 records). [2025-10-20T17:43:10.474134] [DEBUG] [VALIDATOR ] BSM: Validating primary data. [2025-10-20T17:43:10.662456] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-05. [2025-10-20T17:43:10.890400] [DEBUG] [VALIDATOR ] BSM: All expected trading days are present. [2025-10-20T17:43:10.890553] [DEBUG] [VALIDATOR ] BSM: Primary data validation passed. [2025-10-20T17:43:10.891553] [INFO ] [MAIN ] BSM: Fetched primary data is valid. [2025-10-20T17:43:10.891622] [DEBUG] [LOCAL ] BSM: Saving primary data. [2025-10-20T17:43:10.893242] [DEBUG] [LOCAL ] BSM: Trimmed 46955 records to 32482 (keeping 200 days of data) [2025-10-20T17:43:11.105251] [DEBUG] [LOCAL ] BSM: Primary data saved (32482 records). [2025-10-20T17:43:11.105510] [INFO ] [MAIN ] BSM: Full history saved. [2025-10-20T17:43:11.105622] [INFO ] [MAIN ] Update Progress: 345/2367 symbols completed. BSM took 37.3s. Estimated remaining: 4h 12m 40s. Rate: 8.0 symbols/min [2025-10-20T17:43:11.291175] [INFO ] [MAIN ] BTI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:11.346674] [DEBUG] [LOCAL ] BTI: Loaded 54375 existing records from primary storage. [2025-10-20T17:43:11.347245] [INFO ] [MAIN ] BTI: Loaded 54375 existing records. [2025-10-20T17:43:11.459206] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-01. [2025-10-20T17:43:11.557451] [WARN ] [MAIN ] BTI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:11.558540] [INFO ] [MAIN ] BTI: Wiping existing primary data before full refresh. [2025-10-20T17:43:11.559076] [INFO ] [LOCAL ] BTI: Starting deletion of primary data. [2025-10-20T17:43:11.559616] [DEBUG] [LOCAL ] BTI: Deleted primary data file. [2025-10-20T17:43:11.559703] [INFO ] [MAIN ] BTI: Starting full primary refresh. [2025-10-20T17:43:11.984566] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-01 (2.13 seconds). [2025-10-20T17:43:12.061910] [INFO ] [REMOTE ] BTI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:12.251395] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-02. [2025-10-20T17:43:12.476417] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T17:43:12.848908] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-05 (2.19 seconds). [2025-10-20T17:43:13.051444] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-06. [2025-10-20T17:43:13.851425] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-01. [2025-10-20T17:43:13.906930] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:43:14.651398] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-03. [2025-10-20T17:43:14.740731] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:43:15.451420] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-02. [2025-10-20T17:43:15.950194] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:43:15.980706] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T17:43:16.251416] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-03. [2025-10-20T17:43:16.337943] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T17:43:16.742228] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T17:43:17.051450] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-04. [2025-10-20T17:43:17.492572] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-10-20T17:43:17.851379] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-04. [2025-10-20T17:43:18.651376] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-07. [2025-10-20T17:43:19.451400] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-05. [2025-10-20T17:43:19.653959] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T17:43:19.825792] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-10-20T17:43:20.251394] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-02. [2025-10-20T17:43:20.672508] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T17:43:21.051405] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-05. [2025-10-20T17:43:21.851425] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-06. [2025-10-20T17:43:22.232057] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T17:43:22.645343] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:43:22.648020] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T17:43:22.654412] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-03. [2025-10-20T17:43:23.451418] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-08. [2025-10-20T17:43:24.270947] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:43:24.256774] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-06. [2025-10-20T17:43:25.051411] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-04. [2025-10-20T17:43:25.277776] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T17:43:25.851371] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-09. [2025-10-20T17:43:25.913594] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:43:26.651385] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-07. [2025-10-20T17:43:26.742288] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:43:27.451370] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-07. [2025-10-20T17:43:27.731550] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T17:43:28.047937] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T17:43:28.124867] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T17:43:28.251422] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-08. [2025-10-20T17:43:28.610506] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T17:43:29.051384] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-08. [2025-10-20T17:43:29.851414] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-09. [2025-10-20T17:43:30.354985] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T17:43:30.465271] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T17:43:30.651411] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-10. [2025-10-20T17:43:31.203530] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T17:43:31.205794] [INFO ] [REMOTE ] BTA: Full fetch complete (9469 records). [2025-10-20T17:43:31.205897] [DEBUG] [VALIDATOR ] BTA: Validating primary data. [2025-10-20T17:43:31.456716] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-09. [2025-10-20T17:43:31.564271] [DEBUG] [VALIDATOR ] BTA: All expected trading days are present. [2025-10-20T17:43:31.564347] [DEBUG] [VALIDATOR ] BTA: Primary data validation passed. [2025-10-20T17:43:31.564550] [INFO ] [MAIN ] BTA: Fetched primary data is valid. [2025-10-20T17:43:31.564592] [DEBUG] [LOCAL ] BTA: Saving primary data. [2025-10-20T17:43:31.565010] [DEBUG] [LOCAL ] BTA: Trimmed 9469 records to 5756 (keeping 200 days of data) [2025-10-20T17:43:31.597524] [DEBUG] [LOCAL ] BTA: Primary data saved (5756 records). [2025-10-20T17:43:31.597589] [INFO ] [MAIN ] BTA: Full history saved. [2025-10-20T17:43:31.597683] [INFO ] [MAIN ] Update Progress: 346/2367 symbols completed. BTA took 23.7s. Estimated remaining: 4h 13m 49s. Rate: 8.0 symbols/min [2025-10-20T17:43:31.763741] [INFO ] [MAIN ] BTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:31.774288] [DEBUG] [LOCAL ] BTO: Loaded 9046 existing records from primary storage. [2025-10-20T17:43:31.774487] [INFO ] [MAIN ] BTO: Loaded 9046 existing records. [2025-10-20T17:43:31.945133] [WARN ] [MAIN ] BTO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:31.945283] [INFO ] [MAIN ] BTO: Wiping existing primary data before full refresh. [2025-10-20T17:43:31.945326] [INFO ] [LOCAL ] BTO: Starting deletion of primary data. [2025-10-20T17:43:31.945474] [DEBUG] [LOCAL ] BTO: Deleted primary data file. [2025-10-20T17:43:31.945510] [INFO ] [MAIN ] BTO: Starting full primary refresh. [2025-10-20T17:43:32.254691] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-10. [2025-10-20T17:43:32.288626] [INFO ] [REMOTE ] BTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:33.051427] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-01. [2025-10-20T17:43:33.141837] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T17:43:33.172941] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T17:43:33.180973] [INFO ] [REMOTE ] BSX: Full fetch complete (77861 records). [2025-10-20T17:43:33.181142] [DEBUG] [VALIDATOR ] BSX: Validating primary data. [2025-10-20T17:43:33.563608] [DEBUG] [VALIDATOR ] BSX: All expected trading days are present. [2025-10-20T17:43:33.563723] [DEBUG] [VALIDATOR ] BSX: Primary data validation passed. [2025-10-20T17:43:33.564514] [INFO ] [MAIN ] BSX: Fetched primary data is valid. [2025-10-20T17:43:33.564559] [DEBUG] [LOCAL ] BSX: Saving primary data. [2025-10-20T17:43:33.566001] [DEBUG] [LOCAL ] BSX: Trimmed 77861 records to 53688 (keeping 200 days of data) [2025-10-20T17:43:33.867394] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-05. [2025-10-20T17:43:33.896105] [DEBUG] [LOCAL ] BSX: Primary data saved (53688 records). [2025-10-20T17:43:33.896254] [INFO ] [MAIN ] BSX: Full history saved. [2025-10-20T17:43:33.896544] [INFO ] [MAIN ] Update Progress: 347/2367 symbols completed. BSX took 44.4s. Estimated remaining: 4h 13m 11s. Rate: 8.0 symbols/min [2025-10-20T17:43:34.015174] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T17:43:34.085150] [INFO ] [MAIN ] BTT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:34.085325] [DEBUG] [LOCAL ] BTT: No existing data file found in primary storage. [2025-10-20T17:43:34.085362] [WARN ] [MAIN ] BTT: No existing data found, performing full refresh instead. [2025-10-20T17:43:34.085471] [INFO ] [MAIN ] BTT: Wiping existing primary data before full refresh. [2025-10-20T17:43:34.085511] [INFO ] [LOCAL ] BTT: Starting deletion of primary data. [2025-10-20T17:43:34.085574] [DEBUG] [LOCAL ] BTT: No primary data file found to delete. [2025-10-20T17:43:34.085606] [INFO ] [MAIN ] BTT: Starting full primary refresh. [2025-10-20T17:43:34.443964] [INFO ] [REMOTE ] BTT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:34.651369] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-01. [2025-10-20T17:43:35.451422] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-02. [2025-10-20T17:43:35.691565] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:43:35.808299] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T17:43:36.201481] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T17:43:36.251378] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-03. [2025-10-20T17:43:37.051425] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-06. [2025-10-20T17:43:37.079423] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T17:43:37.851418] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-02. [2025-10-20T17:43:38.505467] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:43:38.651413] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-10. [2025-10-20T17:43:39.082811] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T17:43:39.451456] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-03. [2025-10-20T17:43:40.251457] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-07. [2025-10-20T17:43:40.453292] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T17:43:40.471601] [INFO ] [REMOTE ] BTE: Full fetch complete (76364 records). [2025-10-20T17:43:40.472556] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T17:43:40.472696] [DEBUG] [VALIDATOR ] BTE: Validating primary data. [2025-10-20T17:43:40.871325] [DEBUG] [VALIDATOR ] BTE: All expected trading days are present. [2025-10-20T17:43:40.871422] [DEBUG] [VALIDATOR ] BTE: Primary data validation passed. [2025-10-20T17:43:40.872241] [INFO ] [MAIN ] BTE: Fetched primary data is valid. [2025-10-20T17:43:40.872314] [DEBUG] [LOCAL ] BTE: Saving primary data. [2025-10-20T17:43:40.873895] [DEBUG] [LOCAL ] BTE: Trimmed 76364 records to 52728 (keeping 200 days of data) [2025-10-20T17:43:41.052330] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-04. [2025-10-20T17:43:41.219213] [DEBUG] [LOCAL ] BTE: Primary data saved (52728 records). [2025-10-20T17:43:41.219405] [INFO ] [MAIN ] BTE: Full history saved. [2025-10-20T17:43:41.219503] [INFO ] [MAIN ] Update Progress: 348/2367 symbols completed. BTE took 32.1s. Estimated remaining: 4h 13m 2s. Rate: 8.0 symbols/min [2025-10-20T17:43:41.401134] [INFO ] [MAIN ] BTU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:41.451778] [DEBUG] [LOCAL ] BTU: Loaded 54104 existing records from primary storage. [2025-10-20T17:43:41.452159] [INFO ] [MAIN ] BTU: Loaded 54104 existing records. [2025-10-20T17:43:41.626812] [WARN ] [MAIN ] BTU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:41.627035] [INFO ] [MAIN ] BTU: Wiping existing primary data before full refresh. [2025-10-20T17:43:41.627082] [INFO ] [LOCAL ] BTU: Starting deletion of primary data. [2025-10-20T17:43:41.627540] [DEBUG] [LOCAL ] BTU: Deleted primary data file. [2025-10-20T17:43:41.627578] [INFO ] [MAIN ] BTU: Starting full primary refresh. [2025-10-20T17:43:41.862840] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-04. [2025-10-20T17:43:42.062748] [INFO ] [REMOTE ] BTU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:42.133926] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T17:43:42.270312] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T17:43:42.651381] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-08. [2025-10-20T17:43:42.793980] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T17:43:43.451389] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-05. [2025-10-20T17:43:44.254716] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-05. [2025-10-20T17:43:44.260112] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T17:43:44.763478] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T17:43:44.850597] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T17:43:45.051420] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-06. [2025-10-20T17:43:45.808493] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T17:43:45.851430] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-09. [2025-10-20T17:43:46.651439] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-01. [2025-10-20T17:43:47.451364] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-07. [2025-10-20T17:43:47.812881] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T17:43:47.940868] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T17:43:48.192081] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-10-20T17:43:48.251368] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-10. [2025-10-20T17:43:49.051392] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-08. [2025-10-20T17:43:49.851734] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-06. [2025-10-20T17:43:49.851968] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T17:43:50.166717] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:43:50.175169] [INFO ] [REMOTE ] BTI: Full fetch complete (77645 records). [2025-10-20T17:43:50.175339] [DEBUG] [VALIDATOR ] BTI: Validating primary data. [2025-10-20T17:43:50.574122] [DEBUG] [VALIDATOR ] BTI: All expected trading days are present. [2025-10-20T17:43:50.574228] [DEBUG] [VALIDATOR ] BTI: Primary data validation passed. [2025-10-20T17:43:50.575045] [INFO ] [MAIN ] BTI: Fetched primary data is valid. [2025-10-20T17:43:50.575116] [DEBUG] [LOCAL ] BTI: Saving primary data. [2025-10-20T17:43:50.576556] [DEBUG] [LOCAL ] BTI: Trimmed 77645 records to 53594 (keeping 200 days of data) [2025-10-20T17:43:50.665821] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-02. [2025-10-20T17:43:50.891556] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:43:50.919822] [DEBUG] [LOCAL ] BTI: Primary data saved (53594 records). [2025-10-20T17:43:50.919972] [INFO ] [MAIN ] BTI: Full history saved. [2025-10-20T17:43:50.920055] [INFO ] [MAIN ] Update Progress: 349/2367 symbols completed. BTI took 39.8s. Estimated remaining: 4h 13m 7s. Rate: 8.0 symbols/min [2025-10-20T17:43:51.087528] [INFO ] [MAIN ] BTX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:51.087717] [DEBUG] [LOCAL ] BTX: No existing data file found in primary storage. [2025-10-20T17:43:51.087768] [WARN ] [MAIN ] BTX: No existing data found, performing full refresh instead. [2025-10-20T17:43:51.087931] [INFO ] [MAIN ] BTX: Wiping existing primary data before full refresh. [2025-10-20T17:43:51.087976] [INFO ] [LOCAL ] BTX: Starting deletion of primary data. [2025-10-20T17:43:51.088036] [DEBUG] [LOCAL ] BTX: No primary data file found to delete. [2025-10-20T17:43:51.088071] [INFO ] [MAIN ] BTX: Starting full primary refresh. [2025-10-20T17:43:51.448107] [INFO ] [REMOTE ] BTX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:51.451333] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-01. [2025-10-20T17:43:51.653471] [DEBUG] [REMOTE ] BTX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:43:51.653839] [WARN ] [REMOTE ] BTX: No data returned for month 2025-01 [2025-10-20T17:43:51.653925] [FATAL] [REMOTE ] BTX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:43:51.653971] [FATAL] [MAIN ] BTX: Failed to fetch primary data. [2025-10-20T17:43:51.654028] [WARN ] [MAIN ] BTX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:43:51.654097] [INFO ] [LOCAL ] BTX: Starting deletion of primary data. [2025-10-20T17:43:51.654160] [DEBUG] [LOCAL ] BTX: No primary data file found to delete. [2025-10-20T17:43:51.654194] [INFO ] [LOCAL ] BTX: Starting deletion of derivative data. [2025-10-20T17:43:51.654232] [DEBUG] [LOCAL ] BTX: No derivative data file found to delete. [2025-10-20T17:43:51.654277] [INFO ] [MAIN ] Update Progress: 350/2367 symbols completed. BTX took 0.7s. Estimated remaining: 4h 12m 20s. Rate: 8.0 symbols/min [2025-10-20T17:43:51.822507] [INFO ] [MAIN ] BTZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:51.845451] [DEBUG] [LOCAL ] BTZ: Loaded 22828 existing records from primary storage. [2025-10-20T17:43:51.845756] [INFO ] [MAIN ] BTZ: Loaded 22828 existing records. [2025-10-20T17:43:52.018943] [WARN ] [MAIN ] BTZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:52.019158] [INFO ] [MAIN ] BTZ: Wiping existing primary data before full refresh. [2025-10-20T17:43:52.019204] [INFO ] [LOCAL ] BTZ: Starting deletion of primary data. [2025-10-20T17:43:52.019450] [DEBUG] [LOCAL ] BTZ: Deleted primary data file. [2025-10-20T17:43:52.019506] [INFO ] [MAIN ] BTZ: Starting full primary refresh. [2025-10-20T17:43:52.261469] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-07. [2025-10-20T17:43:52.373577] [INFO ] [REMOTE ] BTZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:52.405287] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T17:43:53.051419] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-01. [2025-10-20T17:43:53.341927] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T17:43:53.851361] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-08. [2025-10-20T17:43:54.173908] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:43:54.651458] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-09. [2025-10-20T17:43:54.901255] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T17:43:55.127957] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T17:43:55.451407] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-09. [2025-10-20T17:43:56.251442] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-02. [2025-10-20T17:43:56.426734] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T17:43:57.061778] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-10. [2025-10-20T17:43:57.064732] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T17:43:57.851359] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-03. [2025-10-20T17:43:57.906350] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T17:43:57.908790] [INFO ] [REMOTE ] BTO: Full fetch complete (13315 records). [2025-10-20T17:43:57.908912] [DEBUG] [VALIDATOR ] BTO: Validating primary data. [2025-10-20T17:43:58.284044] [DEBUG] [VALIDATOR ] BTO: All expected trading days are present. [2025-10-20T17:43:58.284149] [DEBUG] [VALIDATOR ] BTO: Primary data validation passed. [2025-10-20T17:43:58.284634] [INFO ] [MAIN ] BTO: Fetched primary data is valid. [2025-10-20T17:43:58.284718] [DEBUG] [LOCAL ] BTO: Saving primary data. [2025-10-20T17:43:58.285327] [DEBUG] [LOCAL ] BTO: Trimmed 13315 records to 8905 (keeping 200 days of data) [2025-10-20T17:43:58.344142] [DEBUG] [LOCAL ] BTO: Primary data saved (8905 records). [2025-10-20T17:43:58.344345] [INFO ] [MAIN ] BTO: Full history saved. [2025-10-20T17:43:58.344451] [INFO ] [MAIN ] Update Progress: 351/2367 symbols completed. BTO took 26.7s. Estimated remaining: 4h 12m 8s. Rate: 8.0 symbols/min [2025-10-20T17:43:58.532224] [INFO ] [MAIN ] BUD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:58.589515] [DEBUG] [LOCAL ] BUD: Loaded 53369 existing records from primary storage. [2025-10-20T17:43:58.590002] [INFO ] [MAIN ] BUD: Loaded 53369 existing records. [2025-10-20T17:43:58.660558] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-03. [2025-10-20T17:43:58.804108] [WARN ] [MAIN ] BUD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:58.805456] [INFO ] [MAIN ] BUD: Wiping existing primary data before full refresh. [2025-10-20T17:43:58.805522] [INFO ] [LOCAL ] BUD: Starting deletion of primary data. [2025-10-20T17:43:58.806355] [DEBUG] [LOCAL ] BUD: Deleted primary data file. [2025-10-20T17:43:58.806414] [INFO ] [MAIN ] BUD: Starting full primary refresh. [2025-10-20T17:43:59.168249] [INFO ] [REMOTE ] BUD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:59.451385] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-01. [2025-10-20T17:43:59.794480] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T17:43:59.852369] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T17:44:00.251377] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-04. [2025-10-20T17:44:01.051402] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-04. [2025-10-20T17:44:01.130665] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T17:44:01.851351] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-02. [2025-10-20T17:44:02.031131] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:44:02.426793] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:44:02.651380] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-10. [2025-10-20T17:44:03.438181] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T17:44:03.440422] [INFO ] [REMOTE ] BTT: Full fetch complete (22470 records). [2025-10-20T17:44:03.440493] [DEBUG] [VALIDATOR ] BTT: Validating primary data. [2025-10-20T17:44:03.461998] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-05. [2025-10-20T17:44:03.831431] [DEBUG] [VALIDATOR ] BTT: All expected trading days are present. [2025-10-20T17:44:03.831669] [DEBUG] [VALIDATOR ] BTT: Primary data validation passed. [2025-10-20T17:44:03.832135] [INFO ] [MAIN ] BTT: Fetched primary data is valid. [2025-10-20T17:44:03.832206] [DEBUG] [LOCAL ] BTT: Saving primary data. [2025-10-20T17:44:03.843350] [DEBUG] [LOCAL ] BTT: Trimmed 22470 records to 15488 (keeping 200 days of data) [2025-10-20T17:44:03.866934] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-10-20T17:44:03.969370] [DEBUG] [LOCAL ] BTT: Primary data saved (15488 records). [2025-10-20T17:44:03.969560] [INFO ] [MAIN ] BTT: Full history saved. [2025-10-20T17:44:03.969639] [INFO ] [MAIN ] Update Progress: 352/2367 symbols completed. BTT took 30.1s. Estimated remaining: 4h 11m 50s. Rate: 8.0 symbols/min [2025-10-20T17:44:04.143677] [INFO ] [MAIN ] BUI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:04.156786] [DEBUG] [LOCAL ] BUI: Loaded 12102 existing records from primary storage. [2025-10-20T17:44:04.157002] [INFO ] [MAIN ] BUI: Loaded 12102 existing records. [2025-10-20T17:44:04.257445] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-03. [2025-10-20T17:44:04.340014] [WARN ] [MAIN ] BUI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:04.340261] [INFO ] [MAIN ] BUI: Wiping existing primary data before full refresh. [2025-10-20T17:44:04.340309] [INFO ] [LOCAL ] BUI: Starting deletion of primary data. [2025-10-20T17:44:04.340516] [DEBUG] [LOCAL ] BUI: Deleted primary data file. [2025-10-20T17:44:04.340573] [INFO ] [MAIN ] BUI: Starting full primary refresh. [2025-10-20T17:44:04.710459] [INFO ] [REMOTE ] BUI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:04.724898] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T17:44:05.051370] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-01. [2025-10-20T17:44:05.851356] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-05. [2025-10-20T17:44:05.912396] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T17:44:06.002421] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:44:06.651379] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-02. [2025-10-20T17:44:07.400520] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T17:44:07.451366] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-04. [2025-10-20T17:44:07.768125] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T17:44:08.251412] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-06. [2025-10-20T17:44:09.051375] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-03. [2025-10-20T17:44:09.326893] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T17:44:09.401575] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T17:44:09.528323] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T17:44:09.851420] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-07. [2025-10-20T17:44:10.651420] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-06. [2025-10-20T17:44:10.817247] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T17:44:11.451391] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-08. [2025-10-20T17:44:12.251491] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-05. [2025-10-20T17:44:12.584190] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:44:12.638490] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T17:44:13.051390] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-09. [2025-10-20T17:44:13.851442] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-04. [2025-10-20T17:44:13.999968] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T17:44:14.038261] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T17:44:14.651420] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-10. [2025-10-20T17:44:14.728955] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T17:44:15.451434] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-07. [2025-10-20T17:44:15.549437] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:44:15.553837] [INFO ] [REMOTE ] BTZ: Full fetch complete (33268 records). [2025-10-20T17:44:15.554009] [DEBUG] [VALIDATOR ] BTZ: Validating primary data. [2025-10-20T17:44:15.946250] [DEBUG] [VALIDATOR ] BTZ: All expected trading days are present. [2025-10-20T17:44:15.946363] [DEBUG] [VALIDATOR ] BTZ: Primary data validation passed. [2025-10-20T17:44:15.946872] [INFO ] [MAIN ] BTZ: Fetched primary data is valid. [2025-10-20T17:44:15.946934] [DEBUG] [LOCAL ] BTZ: Saving primary data. [2025-10-20T17:44:15.947789] [DEBUG] [LOCAL ] BTZ: Trimmed 33268 records to 22556 (keeping 200 days of data) [2025-10-20T17:44:16.082670] [DEBUG] [LOCAL ] BTZ: Primary data saved (22556 records). [2025-10-20T17:44:16.082808] [INFO ] [MAIN ] BTZ: Full history saved. [2025-10-20T17:44:16.082893] [INFO ] [MAIN ] Update Progress: 353/2367 symbols completed. BTZ took 24.4s. Estimated remaining: 4h 12m 9s. Rate: 8.0 symbols/min [2025-10-20T17:44:16.253540] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-06. [2025-10-20T17:44:16.255186] [INFO ] [MAIN ] BUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:16.299697] [DEBUG] [LOCAL ] BUR: Loaded 46071 existing records from primary storage. [2025-10-20T17:44:16.300018] [INFO ] [MAIN ] BUR: Loaded 46071 existing records. [2025-10-20T17:44:16.479970] [WARN ] [MAIN ] BUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:16.480177] [INFO ] [MAIN ] BUR: Wiping existing primary data before full refresh. [2025-10-20T17:44:16.480224] [INFO ] [LOCAL ] BUR: Starting deletion of primary data. [2025-10-20T17:44:16.480415] [DEBUG] [LOCAL ] BUR: Deleted primary data file. [2025-10-20T17:44:16.480454] [INFO ] [MAIN ] BUR: Starting full primary refresh. [2025-10-20T17:44:16.851244] [INFO ] [REMOTE ] BUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:17.051524] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-05. [2025-10-20T17:44:17.542031] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-10-20T17:44:17.693932] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T17:44:17.851445] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-07. [2025-10-20T17:44:17.891266] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T17:44:18.651459] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-06. [2025-10-20T17:44:19.451407] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-01. [2025-10-20T17:44:19.666303] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:44:19.929883] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-10-20T17:44:20.251371] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-08. [2025-10-20T17:44:21.049529] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:44:21.058456] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-07. [2025-10-20T17:44:21.851435] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-08. [2025-10-20T17:44:21.909882] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T17:44:22.116147] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T17:44:22.651429] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-02. [2025-10-20T17:44:23.268958] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T17:44:23.451350] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-09. [2025-10-20T17:44:23.950890] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T17:44:24.251467] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-08. [2025-10-20T17:44:25.051797] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-03. [2025-10-20T17:44:25.051620] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T17:44:25.446335] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T17:44:25.851415] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-09. [2025-10-20T17:44:26.616421] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:44:26.651415] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-04. [2025-10-20T17:44:27.451395] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-10. [2025-10-20T17:44:27.834320] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T17:44:28.251455] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-10. [2025-10-20T17:44:28.347448] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:44:29.051346] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-05. [2025-10-20T17:44:29.518244] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-10-20T17:44:29.526637] [INFO ] [REMOTE ] BUD: Full fetch complete (76492 records). [2025-10-20T17:44:29.526882] [DEBUG] [VALIDATOR ] BUD: Validating primary data. [2025-10-20T17:44:29.864691] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-09. [2025-10-20T17:44:29.983972] [DEBUG] [VALIDATOR ] BUD: All expected trading days are present. [2025-10-20T17:44:29.984138] [DEBUG] [VALIDATOR ] BUD: Primary data validation passed. [2025-10-20T17:44:29.985162] [INFO ] [MAIN ] BUD: Fetched primary data is valid. [2025-10-20T17:44:29.985233] [DEBUG] [LOCAL ] BUD: Saving primary data. [2025-10-20T17:44:29.987381] [DEBUG] [LOCAL ] BUD: Trimmed 76492 records to 52605 (keeping 200 days of data) [2025-10-20T17:44:30.369939] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T17:44:30.384932] [INFO ] [REMOTE ] BTU: Full fetch complete (77280 records). [2025-10-20T17:44:30.386198] [DEBUG] [VALIDATOR ] BTU: Validating primary data. [2025-10-20T17:44:30.481900] [DEBUG] [LOCAL ] BUD: Primary data saved (52605 records). [2025-10-20T17:44:30.487634] [INFO ] [MAIN ] BUD: Full history saved. [2025-10-20T17:44:30.498774] [INFO ] [MAIN ] Update Progress: 354/2367 symbols completed. BUD took 32.2s. Estimated remaining: 4h 12m 41s. Rate: 8.0 symbols/min [2025-10-20T17:44:30.861514] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T17:44:30.886455] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T17:44:30.939856] [INFO ] [MAIN ] BURL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:30.954716] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-10. [2025-10-20T17:44:31.033146] [DEBUG] [VALIDATOR ] BTU: All expected trading days are present. [2025-10-20T17:44:31.033236] [DEBUG] [VALIDATOR ] BTU: Primary data validation passed. [2025-10-20T17:44:31.034093] [INFO ] [MAIN ] BTU: Fetched primary data is valid. [2025-10-20T17:44:31.034212] [DEBUG] [LOCAL ] BTU: Saving primary data. [2025-10-20T17:44:31.035912] [DEBUG] [LOCAL ] BTU: Trimmed 77280 records to 53336 (keeping 200 days of data) [2025-10-20T17:44:31.178915] [DEBUG] [LOCAL ] BURL: Loaded 46304 existing records from primary storage. [2025-10-20T17:44:31.181331] [INFO ] [MAIN ] BURL: Loaded 46304 existing records. [2025-10-20T17:44:31.461682] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-06. [2025-10-20T17:44:31.467189] [WARN ] [MAIN ] BURL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:31.469114] [INFO ] [MAIN ] BURL: Wiping existing primary data before full refresh. [2025-10-20T17:44:31.469184] [INFO ] [LOCAL ] BURL: Starting deletion of primary data. [2025-10-20T17:44:31.470442] [DEBUG] [LOCAL ] BURL: Deleted primary data file. [2025-10-20T17:44:31.472159] [INFO ] [MAIN ] BURL: Starting full primary refresh. [2025-10-20T17:44:32.015826] [INFO ] [REMOTE ] BURL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:32.020249] [DEBUG] [LOCAL ] BTU: Primary data saved (53336 records). [2025-10-20T17:44:32.027504] [INFO ] [MAIN ] BTU: Full history saved. [2025-10-20T17:44:32.028575] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T17:44:32.028311] [INFO ] [MAIN ] Update Progress: 355/2367 symbols completed. BTU took 50.8s. Estimated remaining: 4h 11m 59s. Rate: 8.0 symbols/min [2025-10-20T17:44:32.037038] [INFO ] [REMOTE ] BUI: Full fetch complete (16655 records). [2025-10-20T17:44:32.075548] [DEBUG] [VALIDATOR ] BUI: Validating primary data. [2025-10-20T17:44:32.256677] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-01. [2025-10-20T17:44:32.354113] [INFO ] [MAIN ] BUXX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:32.565957] [DEBUG] [LOCAL ] BUXX: Loaded 15594 existing records from primary storage. [2025-10-20T17:44:32.598942] [INFO ] [MAIN ] BUXX: Loaded 15594 existing records. [2025-10-20T17:44:32.683220] [DEBUG] [VALIDATOR ] BUI: All expected trading days are present. [2025-10-20T17:44:32.690820] [DEBUG] [VALIDATOR ] BUI: Primary data validation passed. [2025-10-20T17:44:32.691254] [INFO ] [MAIN ] BUI: Fetched primary data is valid. [2025-10-20T17:44:32.700460] [DEBUG] [LOCAL ] BUI: Saving primary data. [2025-10-20T17:44:32.728303] [DEBUG] [LOCAL ] BUI: Trimmed 16655 records to 11949 (keeping 200 days of data) [2025-10-20T17:44:32.917316] [WARN ] [MAIN ] BUXX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:32.921505] [INFO ] [MAIN ] BUXX: Wiping existing primary data before full refresh. [2025-10-20T17:44:32.925708] [INFO ] [LOCAL ] BUXX: Starting deletion of primary data. [2025-10-20T17:44:32.925571] [DEBUG] [LOCAL ] BUI: Primary data saved (11949 records). [2025-10-20T17:44:32.925984] [INFO ] [MAIN ] BUI: Full history saved. [2025-10-20T17:44:32.926100] [INFO ] [MAIN ] Update Progress: 356/2367 symbols completed. BUI took 29.0s. Estimated remaining: 4h 11m 14s. Rate: 8.0 symbols/min [2025-10-20T17:44:32.926130] [DEBUG] [LOCAL ] BUXX: Deleted primary data file. [2025-10-20T17:44:32.926314] [INFO ] [MAIN ] BUXX: Starting full primary refresh. [2025-10-20T17:44:33.323045] [INFO ] [MAIN ] BV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:33.479199] [INFO ] [REMOTE ] BUXX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:33.479775] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-01. [2025-10-20T17:44:33.531578] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-06 (2.07 seconds). [2025-10-20T17:44:33.557671] [DEBUG] [LOCAL ] BV: Loaded 39100 existing records from primary storage. [2025-10-20T17:44:33.558040] [INFO ] [MAIN ] BV: Loaded 39100 existing records. [2025-10-20T17:44:33.771705] [WARN ] [MAIN ] BV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:33.772028] [INFO ] [MAIN ] BV: Wiping existing primary data before full refresh. [2025-10-20T17:44:33.772132] [INFO ] [LOCAL ] BV: Starting deletion of primary data. [2025-10-20T17:44:33.772367] [DEBUG] [LOCAL ] BV: Deleted primary data file. [2025-10-20T17:44:33.772424] [INFO ] [MAIN ] BV: Starting full primary refresh. [2025-10-20T17:44:33.856629] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-07. [2025-10-20T17:44:34.160320] [INFO ] [REMOTE ] BV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:34.180241] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:44:34.179910] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T17:44:34.651421] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-02. [2025-10-20T17:44:35.444796] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T17:44:35.454586] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-01. [2025-10-20T17:44:35.464628] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T17:44:36.251385] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-02. [2025-10-20T17:44:36.466289] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T17:44:37.051495] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-02. [2025-10-20T17:44:37.782889] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T17:44:37.851412] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-03. [2025-10-20T17:44:37.992874] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T17:44:38.651371] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-08. [2025-10-20T17:44:39.451431] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-03. [2025-10-20T17:44:39.594633] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:44:40.258457] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-03. [2025-10-20T17:44:40.286303] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T17:44:40.576891] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T17:44:41.051410] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-04. [2025-10-20T17:44:41.069986] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:44:41.851445] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-04. [2025-10-20T17:44:42.417967] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T17:44:42.651489] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-05. [2025-10-20T17:44:43.249596] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:44:43.451439] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-05. [2025-10-20T17:44:44.251343] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-04. [2025-10-20T17:44:44.501878] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T17:44:44.937426] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T17:44:44.979396] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T17:44:45.051386] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-06. [2025-10-20T17:44:45.851400] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-09. [2025-10-20T17:44:45.986047] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T17:44:46.651377] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-07. [2025-10-20T17:44:47.288217] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T17:44:47.451349] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-10. [2025-10-20T17:44:48.207040] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T17:44:48.251371] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-08. [2025-10-20T17:44:49.051415] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-06. [2025-10-20T17:44:49.172180] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T17:44:49.179418] [INFO ] [REMOTE ] BUR: Full fetch complete (64342 records). [2025-10-20T17:44:49.179596] [DEBUG] [VALIDATOR ] BUR: Validating primary data. [2025-10-20T17:44:49.576528] [DEBUG] [VALIDATOR ] BUR: All expected trading days are present. [2025-10-20T17:44:49.576632] [DEBUG] [VALIDATOR ] BUR: Primary data validation passed. [2025-10-20T17:44:49.577463] [INFO ] [MAIN ] BUR: Fetched primary data is valid. [2025-10-20T17:44:49.577522] [DEBUG] [LOCAL ] BUR: Saving primary data. [2025-10-20T17:44:49.578867] [DEBUG] [LOCAL ] BUR: Trimmed 64342 records to 45494 (keeping 200 days of data) [2025-10-20T17:44:49.802920] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T17:44:49.852583] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-09. [2025-10-20T17:44:49.872838] [DEBUG] [LOCAL ] BUR: Primary data saved (45494 records). [2025-10-20T17:44:49.872923] [INFO ] [MAIN ] BUR: Full history saved. [2025-10-20T17:44:49.872997] [INFO ] [MAIN ] Update Progress: 357/2367 symbols completed. BUR took 33.8s. Estimated remaining: 4h 11m 60s. Rate: 8.0 symbols/min [2025-10-20T17:44:50.353128] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T17:44:50.651380] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-07. [2025-10-20T17:44:51.054694] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T17:44:51.451471] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-10. [2025-10-20T17:44:52.251383] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-05. [2025-10-20T17:44:52.296561] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:44:52.609806] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:44:52.618802] [INFO ] [REMOTE ] BV: Full fetch complete (54895 records). [2025-10-20T17:44:52.618970] [DEBUG] [VALIDATOR ] BV: Validating primary data. [2025-10-20T17:44:52.999720] [DEBUG] [VALIDATOR ] BV: All expected trading days are present. [2025-10-20T17:44:52.999805] [DEBUG] [VALIDATOR ] BV: Primary data validation passed. [2025-10-20T17:44:53.000414] [INFO ] [MAIN ] BV: Fetched primary data is valid. [2025-10-20T17:44:53.000456] [DEBUG] [LOCAL ] BV: Saving primary data. [2025-10-20T17:44:53.001533] [DEBUG] [LOCAL ] BV: Trimmed 54895 records to 38556 (keeping 200 days of data) [2025-10-20T17:44:53.056386] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-08. [2025-10-20T17:44:53.249772] [DEBUG] [LOCAL ] BV: Primary data saved (38556 records). [2025-10-20T17:44:53.250248] [INFO ] [MAIN ] BV: Full history saved. [2025-10-20T17:44:53.250388] [INFO ] [MAIN ] Update Progress: 358/2367 symbols completed. BV took 20.3s. Estimated remaining: 4h 11m 29s. Rate: 8.0 symbols/min [2025-10-20T17:44:53.251512] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T17:44:53.851484] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-06. [2025-10-20T17:44:54.530924] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T17:44:54.661172] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-07. [2025-10-20T17:44:54.667610] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T17:44:55.451356] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-09. [2025-10-20T17:44:55.680214] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T17:44:56.251428] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-08. [2025-10-20T17:44:56.842545] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T17:44:57.051414] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-10. [2025-10-20T17:44:57.077592] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T17:44:57.851384] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-09. [2025-10-20T17:44:58.429461] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T17:44:58.436099] [INFO ] [REMOTE ] BURL: Full fetch complete (65872 records). [2025-10-20T17:44:58.436229] [DEBUG] [VALIDATOR ] BURL: Validating primary data. [2025-10-20T17:44:58.630374] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T17:44:58.693929] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-10. [2025-10-20T17:44:58.886018] [DEBUG] [VALIDATOR ] BURL: All expected trading days are present. [2025-10-20T17:44:58.886107] [DEBUG] [VALIDATOR ] BURL: Primary data validation passed. [2025-10-20T17:44:58.886812] [INFO ] [MAIN ] BURL: Fetched primary data is valid. [2025-10-20T17:44:58.886874] [DEBUG] [LOCAL ] BURL: Saving primary data. [2025-10-20T17:44:58.888246] [DEBUG] [LOCAL ] BURL: Trimmed 65872 records to 45583 (keeping 200 days of data) [2025-10-20T17:44:59.170316] [DEBUG] [LOCAL ] BURL: Primary data saved (45583 records). [2025-10-20T17:44:59.170458] [INFO ] [MAIN ] BURL: Full history saved. [2025-10-20T17:44:59.170537] [INFO ] [MAIN ] Update Progress: 359/2367 symbols completed. BURL took 28.7s. Estimated remaining: 4h 11m 13s. Rate: 8.0 symbols/min [2025-10-20T17:44:59.689799] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:44:59.692553] [INFO ] [REMOTE ] BUXX: Full fetch complete (20711 records). [2025-10-20T17:44:59.692686] [DEBUG] [VALIDATOR ] BUXX: Validating primary data. [2025-10-20T17:45:00.134137] [DEBUG] [VALIDATOR ] BUXX: All expected trading days are present. [2025-10-20T17:45:00.134243] [DEBUG] [VALIDATOR ] BUXX: Primary data validation passed. [2025-10-20T17:45:00.134720] [INFO ] [MAIN ] BUXX: Fetched primary data is valid. [2025-10-20T17:45:00.134783] [DEBUG] [LOCAL ] BUXX: Saving primary data. [2025-10-20T17:45:00.135685] [DEBUG] [LOCAL ] BUXX: Trimmed 20711 records to 15466 (keeping 200 days of data) [2025-10-20T17:45:00.232675] [DEBUG] [LOCAL ] BUXX: Primary data saved (15466 records). [2025-10-20T17:45:00.232833] [INFO ] [MAIN ] BUXX: Full history saved. [2025-10-20T17:45:00.232923] [INFO ] [MAIN ] Update Progress: 360/2367 symbols completed. BUXX took 28.2s. Estimated remaining: 4h 10m 29s. Rate: 8.0 symbols/min [2025-10-20T17:45:00.749090] [INFO ] [MAIN ] BWA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:00.867636] [INFO ] [MAIN ] BWG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:00.882103] [DEBUG] [LOCAL ] BWG: No existing data file found in primary storage. [2025-10-20T17:45:00.942590] [INFO ] [MAIN ] BVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:00.929385] [WARN ] [MAIN ] BWG: No existing data found, performing full refresh instead. [2025-10-20T17:45:00.966044] [INFO ] [MAIN ] BWG: Wiping existing primary data before full refresh. [2025-10-20T17:45:00.965820] [INFO ] [MAIN ] BW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:00.966721] [INFO ] [LOCAL ] BWG: Starting deletion of primary data. [2025-10-20T17:45:00.966820] [DEBUG] [LOCAL ] BWG: No primary data file found to delete. [2025-10-20T17:45:00.967184] [INFO ] [MAIN ] BWG: Starting full primary refresh. [2025-10-20T17:45:01.305967] [DEBUG] [LOCAL ] BWA: Loaded 52901 existing records from primary storage. [2025-10-20T17:45:01.322011] [INFO ] [MAIN ] BWA: Loaded 52901 existing records. [2025-10-20T17:45:01.345112] [DEBUG] [LOCAL ] BVN: Loaded 45065 existing records from primary storage. [2025-10-20T17:45:01.391691] [DEBUG] [LOCAL ] BW: Loaded 41213 existing records from primary storage. [2025-10-20T17:45:01.404468] [INFO ] [MAIN ] BVN: Loaded 45065 existing records. [2025-10-20T17:45:01.415516] [INFO ] [MAIN ] BW: Loaded 41213 existing records. [2025-10-20T17:45:01.725368] [INFO ] [REMOTE ] BWG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:01.756949] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-01. [2025-10-20T17:45:01.991542] [WARN ] [MAIN ] BWA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:02.077039] [INFO ] [MAIN ] BWA: Wiping existing primary data before full refresh. [2025-10-20T17:45:02.076952] [WARN ] [MAIN ] BVN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:02.120030] [INFO ] [MAIN ] BVN: Wiping existing primary data before full refresh. [2025-10-20T17:45:02.119867] [WARN ] [MAIN ] BW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:02.113508] [INFO ] [LOCAL ] BWA: Starting deletion of primary data. [2025-10-20T17:45:02.120211] [INFO ] [MAIN ] BW: Wiping existing primary data before full refresh. [2025-10-20T17:45:02.120309] [INFO ] [LOCAL ] BVN: Starting deletion of primary data. [2025-10-20T17:45:02.120442] [INFO ] [LOCAL ] BW: Starting deletion of primary data. [2025-10-20T17:45:02.120730] [DEBUG] [LOCAL ] BWA: Deleted primary data file. [2025-10-20T17:45:02.120790] [INFO ] [MAIN ] BWA: Starting full primary refresh. [2025-10-20T17:45:02.131788] [DEBUG] [LOCAL ] BW: Deleted primary data file. [2025-10-20T17:45:02.131914] [DEBUG] [LOCAL ] BVN: Deleted primary data file. [2025-10-20T17:45:02.132088] [INFO ] [MAIN ] BVN: Starting full primary refresh. [2025-10-20T17:45:02.131984] [INFO ] [MAIN ] BW: Starting full primary refresh. [2025-10-20T17:45:03.182224] [INFO ] [REMOTE ] BW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:03.183128] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-01. [2025-10-20T17:45:03.294144] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:45:03.346688] [INFO ] [REMOTE ] BVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:03.361594] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-02. [2025-10-20T17:45:03.361901] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-01. [2025-10-20T17:45:03.386341] [INFO ] [REMOTE ] BWA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:03.387497] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-01. [2025-10-20T17:45:04.241773] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T17:45:04.245771] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-03. [2025-10-20T17:45:04.784766] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:45:04.782524] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T17:45:04.794034] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-02. [2025-10-20T17:45:05.051378] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-04. [2025-10-20T17:45:05.098377] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T17:45:05.487792] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:45:05.749617] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T17:45:05.851381] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-02. [2025-10-20T17:45:06.393608] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:45:06.651449] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-03. [2025-10-20T17:45:07.269869] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:45:07.451456] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-02. [2025-10-20T17:45:08.189123] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:45:08.251405] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-04. [2025-10-20T17:45:08.656830] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:45:09.051370] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-03. [2025-10-20T17:45:09.851367] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-03. [2025-10-20T17:45:10.077815] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T17:45:10.651348] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-05. [2025-10-20T17:45:10.826288] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:45:11.435514] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T17:45:11.451324] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-04. [2025-10-20T17:45:11.971595] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T17:45:12.251427] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-05. [2025-10-20T17:45:13.051363] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-04. [2025-10-20T17:45:13.143053] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:45:13.173500] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T17:45:13.851441] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-06. [2025-10-20T17:45:14.657623] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-05. [2025-10-20T17:45:14.662955] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:45:15.047576] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T17:45:15.451378] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-06. [2025-10-20T17:45:15.886683] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T17:45:16.253036] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-07. [2025-10-20T17:45:16.255126] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T17:45:17.051441] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-07. [2025-10-20T17:45:17.637348] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:45:17.830040] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T17:45:17.851345] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-08. [2025-10-20T17:45:18.651393] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-05. [2025-10-20T17:45:18.754099] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T17:45:19.451437] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-09. [2025-10-20T17:45:20.256505] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-06. [2025-10-20T17:45:20.262817] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T17:45:20.390689] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:45:21.051420] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-10. [2025-10-20T17:45:21.744883] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T17:45:21.747506] [INFO ] [REMOTE ] BWG: Full fetch complete (17726 records). [2025-10-20T17:45:21.747809] [DEBUG] [VALIDATOR ] BWG: Validating primary data. [2025-10-20T17:45:21.853554] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-08. [2025-10-20T17:45:21.976971] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T17:45:22.187936] [DEBUG] [VALIDATOR ] BWG: All expected trading days are present. [2025-10-20T17:45:22.188021] [DEBUG] [VALIDATOR ] BWG: Primary data validation passed. [2025-10-20T17:45:22.188385] [INFO ] [MAIN ] BWG: Fetched primary data is valid. [2025-10-20T17:45:22.188445] [DEBUG] [LOCAL ] BWG: Saving primary data. [2025-10-20T17:45:22.188999] [DEBUG] [LOCAL ] BWG: Trimmed 17726 records to 12624 (keeping 200 days of data) [2025-10-20T17:45:22.261724] [DEBUG] [LOCAL ] BWG: Primary data saved (12624 records). [2025-10-20T17:45:22.261865] [INFO ] [MAIN ] BWG: Full history saved. [2025-10-20T17:45:22.261936] [INFO ] [MAIN ] Update Progress: 361/2367 symbols completed. BWG took 22.0s. Estimated remaining: 4h 11m 43s. Rate: 8.0 symbols/min [2025-10-20T17:45:22.428786] [INFO ] [MAIN ] BWLP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:22.459740] [DEBUG] [LOCAL ] BWLP: Loaded 32641 existing records from primary storage. [2025-10-20T17:45:22.460011] [INFO ] [MAIN ] BWLP: Loaded 32641 existing records. [2025-10-20T17:45:22.641065] [WARN ] [MAIN ] BWLP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:22.641318] [INFO ] [MAIN ] BWLP: Wiping existing primary data before full refresh. [2025-10-20T17:45:22.641366] [INFO ] [LOCAL ] BWLP: Starting deletion of primary data. [2025-10-20T17:45:22.641584] [DEBUG] [LOCAL ] BWLP: Deleted primary data file. [2025-10-20T17:45:22.641673] [INFO ] [MAIN ] BWLP: Starting full primary refresh. [2025-10-20T17:45:22.661759] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-06. [2025-10-20T17:45:22.995823] [INFO ] [REMOTE ] BWLP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:23.058858] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T17:45:23.451397] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-07. [2025-10-20T17:45:23.873477] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:45:24.251433] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-01. [2025-10-20T17:45:25.043225] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T17:45:25.051935] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-07. [2025-10-20T17:45:25.483904] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T17:45:25.851429] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-08. [2025-10-20T17:45:26.355998] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T17:45:26.651395] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-08. [2025-10-20T17:45:27.451348] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-02. [2025-10-20T17:45:27.557207] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:45:27.925557] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T17:45:28.251383] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-09. [2025-10-20T17:45:28.432537] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:45:29.051424] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-09. [2025-10-20T17:45:29.712818] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:45:29.851352] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-10. [2025-10-20T17:45:30.619347] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T17:45:30.651328] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-10. [2025-10-20T17:45:31.451393] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-03. [2025-10-20T17:45:31.608267] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:45:31.616453] [INFO ] [REMOTE ] BWA: Full fetch complete (75908 records). [2025-10-20T17:45:31.616625] [DEBUG] [VALIDATOR ] BWA: Validating primary data. [2025-10-20T17:45:32.024393] [DEBUG] [VALIDATOR ] BWA: All expected trading days are present. [2025-10-20T17:45:32.024483] [DEBUG] [VALIDATOR ] BWA: Primary data validation passed. [2025-10-20T17:45:32.025414] [INFO ] [MAIN ] BWA: Fetched primary data is valid. [2025-10-20T17:45:32.025489] [DEBUG] [LOCAL ] BWA: Saving primary data. [2025-10-20T17:45:32.027139] [DEBUG] [LOCAL ] BWA: Trimmed 75908 records to 52108 (keeping 200 days of data) [2025-10-20T17:45:32.252164] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-09. [2025-10-20T17:45:32.363764] [DEBUG] [LOCAL ] BWA: Primary data saved (52108 records). [2025-10-20T17:45:32.363908] [INFO ] [MAIN ] BWA: Full history saved. [2025-10-20T17:45:32.364089] [INFO ] [MAIN ] Update Progress: 362/2367 symbols completed. BWA took 32.1s. Estimated remaining: 4h 11m 49s. Rate: 8.0 symbols/min [2025-10-20T17:45:32.534624] [INFO ] [MAIN ] BWMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:32.535025] [DEBUG] [LOCAL ] BWMX: No existing data file found in primary storage. [2025-10-20T17:45:32.535135] [WARN ] [MAIN ] BWMX: No existing data found, performing full refresh instead. [2025-10-20T17:45:32.535286] [INFO ] [MAIN ] BWMX: Wiping existing primary data before full refresh. [2025-10-20T17:45:32.535410] [INFO ] [LOCAL ] BWMX: Starting deletion of primary data. [2025-10-20T17:45:32.535527] [DEBUG] [LOCAL ] BWMX: No primary data file found to delete. [2025-10-20T17:45:32.535600] [INFO ] [MAIN ] BWMX: Starting full primary refresh. [2025-10-20T17:45:32.671532] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T17:45:32.753109] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:45:32.865221] [INFO ] [REMOTE ] BW: Full fetch complete (59407 records). [2025-10-20T17:45:32.876300] [DEBUG] [VALIDATOR ] BW: Validating primary data. [2025-10-20T17:45:33.069801] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-04. [2025-10-20T17:45:33.094708] [INFO ] [REMOTE ] BWMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:33.387424] [DEBUG] [VALIDATOR ] BW: All expected trading days are present. [2025-10-20T17:45:33.387513] [DEBUG] [VALIDATOR ] BW: Primary data validation passed. [2025-10-20T17:45:33.388197] [INFO ] [MAIN ] BW: Fetched primary data is valid. [2025-10-20T17:45:33.388260] [DEBUG] [LOCAL ] BW: Saving primary data. [2025-10-20T17:45:33.389536] [DEBUG] [LOCAL ] BW: Trimmed 59407 records to 40452 (keeping 200 days of data) [2025-10-20T17:45:33.614800] [DEBUG] [LOCAL ] BW: Primary data saved (40452 records). [2025-10-20T17:45:33.614928] [INFO ] [MAIN ] BW: Full history saved. [2025-10-20T17:45:33.615003] [INFO ] [MAIN ] Update Progress: 363/2367 symbols completed. BW took 33.4s. Estimated remaining: 4h 11m 7s. Rate: 8.0 symbols/min [2025-10-20T17:45:33.814489] [INFO ] [MAIN ] BWNB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:33.820945] [DEBUG] [LOCAL ] BWNB: Loaded 4225 existing records from primary storage. [2025-10-20T17:45:33.821181] [INFO ] [MAIN ] BWNB: Loaded 4225 existing records. [2025-10-20T17:45:33.857677] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-01. [2025-10-20T17:45:34.005034] [WARN ] [MAIN ] BWNB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:34.005320] [INFO ] [MAIN ] BWNB: Wiping existing primary data before full refresh. [2025-10-20T17:45:34.005417] [INFO ] [LOCAL ] BWNB: Starting deletion of primary data. [2025-10-20T17:45:34.014952] [DEBUG] [LOCAL ] BWNB: Deleted primary data file. [2025-10-20T17:45:34.028970] [INFO ] [MAIN ] BWNB: Starting full primary refresh. [2025-10-20T17:45:34.078008] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T17:45:34.418271] [INFO ] [REMOTE ] BWNB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:34.535978] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:45:34.624306] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T17:45:34.651375] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-02. [2025-10-20T17:45:35.227721] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T17:45:35.451417] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-10. [2025-10-20T17:45:36.251375] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-03. [2025-10-20T17:45:36.601256] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T17:45:37.051436] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-01. [2025-10-20T17:45:37.188835] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:45:37.196407] [INFO ] [REMOTE ] BVN: Full fetch complete (63402 records). [2025-10-20T17:45:37.196588] [DEBUG] [VALIDATOR ] BVN: Validating primary data. [2025-10-20T17:45:37.574794] [DEBUG] [VALIDATOR ] BVN: All expected trading days are present. [2025-10-20T17:45:37.574871] [DEBUG] [VALIDATOR ] BVN: Primary data validation passed. [2025-10-20T17:45:37.575555] [INFO ] [MAIN ] BVN: Fetched primary data is valid. [2025-10-20T17:45:37.575594] [DEBUG] [LOCAL ] BVN: Saving primary data. [2025-10-20T17:45:37.576816] [DEBUG] [LOCAL ] BVN: Trimmed 63402 records to 44477 (keeping 200 days of data) [2025-10-20T17:45:37.670956] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T17:45:37.838924] [DEBUG] [LOCAL ] BVN: Primary data saved (44477 records). [2025-10-20T17:45:37.839063] [INFO ] [MAIN ] BVN: Full history saved. [2025-10-20T17:45:37.839143] [INFO ] [MAIN ] Update Progress: 364/2367 symbols completed. BVN took 37.6s. Estimated remaining: 4h 10m 41s. Rate: 8.0 symbols/min [2025-10-20T17:45:37.859460] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-04. [2025-10-20T17:45:38.014583] [INFO ] [MAIN ] BWSN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:38.021051] [DEBUG] [LOCAL ] BWSN: Loaded 5235 existing records from primary storage. [2025-10-20T17:45:38.021241] [INFO ] [MAIN ] BWSN: Loaded 5235 existing records. [2025-10-20T17:45:38.201532] [WARN ] [MAIN ] BWSN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:38.201800] [INFO ] [MAIN ] BWSN: Wiping existing primary data before full refresh. [2025-10-20T17:45:38.201850] [INFO ] [LOCAL ] BWSN: Starting deletion of primary data. [2025-10-20T17:45:38.201993] [DEBUG] [LOCAL ] BWSN: Deleted primary data file. [2025-10-20T17:45:38.202034] [INFO ] [MAIN ] BWSN: Starting full primary refresh. [2025-10-20T17:45:38.567778] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T17:45:38.585542] [INFO ] [REMOTE ] BWSN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:38.651379] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-05. [2025-10-20T17:45:39.393107] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T17:45:39.451357] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-06. [2025-10-20T17:45:40.188818] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T17:45:40.251398] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-07. [2025-10-20T17:45:41.051356] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-05. [2025-10-20T17:45:41.150595] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T17:45:41.851416] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-01. [2025-10-20T17:45:42.270132] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T17:45:42.617391] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:45:42.651367] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-06. [2025-10-20T17:45:43.451371] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-02. [2025-10-20T17:45:43.923956] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T17:45:44.024689] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:45:44.251387] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-03. [2025-10-20T17:45:44.896553] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T17:45:45.051360] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-04. [2025-10-20T17:45:45.608204] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T17:45:45.851401] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-08. [2025-10-20T17:45:46.651399] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-07. [2025-10-20T17:45:46.898081] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T17:45:47.451392] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-09. [2025-10-20T17:45:47.694800] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T17:45:48.047235] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T17:45:48.251422] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-08. [2025-10-20T17:45:49.051411] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-10. [2025-10-20T17:45:49.600367] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:45:49.822432] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T17:45:49.824217] [INFO ] [REMOTE ] BWMX: Full fetch complete (9952 records). [2025-10-20T17:45:49.824295] [DEBUG] [VALIDATOR ] BWMX: Validating primary data. [2025-10-20T17:45:49.856546] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-05. [2025-10-20T17:45:50.208223] [DEBUG] [VALIDATOR ] BWMX: All expected trading days are present. [2025-10-20T17:45:50.208402] [DEBUG] [VALIDATOR ] BWMX: Primary data validation passed. [2025-10-20T17:45:50.208806] [INFO ] [MAIN ] BWMX: Fetched primary data is valid. [2025-10-20T17:45:50.208871] [DEBUG] [LOCAL ] BWMX: Saving primary data. [2025-10-20T17:45:50.209569] [DEBUG] [LOCAL ] BWMX: Trimmed 9952 records to 7887 (keeping 200 days of data) [2025-10-20T17:45:50.260113] [DEBUG] [LOCAL ] BWMX: Primary data saved (7887 records). [2025-10-20T17:45:50.260316] [INFO ] [MAIN ] BWMX: Full history saved. [2025-10-20T17:45:50.260410] [INFO ] [MAIN ] Update Progress: 365/2367 symbols completed. BWMX took 17.9s. Estimated remaining: 4h 11m 1s. Rate: 8.0 symbols/min [2025-10-20T17:45:50.453261] [INFO ] [MAIN ] BWXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:50.508958] [DEBUG] [LOCAL ] BWXT: Loaded 47875 existing records from primary storage. [2025-10-20T17:45:50.509455] [INFO ] [MAIN ] BWXT: Loaded 47875 existing records. [2025-10-20T17:45:50.658886] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-02. [2025-10-20T17:45:50.705458] [WARN ] [MAIN ] BWXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:50.707042] [INFO ] [MAIN ] BWXT: Wiping existing primary data before full refresh. [2025-10-20T17:45:50.707193] [INFO ] [LOCAL ] BWXT: Starting deletion of primary data. [2025-10-20T17:45:50.707553] [DEBUG] [LOCAL ] BWXT: Deleted primary data file. [2025-10-20T17:45:50.707616] [INFO ] [MAIN ] BWXT: Starting full primary refresh. [2025-10-20T17:45:50.801561] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T17:45:51.093777] [INFO ] [REMOTE ] BWXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:51.223241] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T17:45:51.451386] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-03. [2025-10-20T17:45:51.954753] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T17:45:52.251417] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-01. [2025-10-20T17:45:53.051394] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-09. [2025-10-20T17:45:53.851371] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-04. [2025-10-20T17:45:54.095590] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T17:45:54.413923] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T17:45:54.541604] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T17:45:54.651396] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-10. [2025-10-20T17:45:55.451379] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-02. [2025-10-20T17:45:56.028347] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T17:45:56.033727] [INFO ] [REMOTE ] BWLP: Full fetch complete (47924 records). [2025-10-20T17:45:56.033847] [DEBUG] [VALIDATOR ] BWLP: Validating primary data. [2025-10-20T17:45:56.256556] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-05. [2025-10-20T17:45:56.429407] [DEBUG] [VALIDATOR ] BWLP: All expected trading days are present. [2025-10-20T17:45:56.429488] [DEBUG] [VALIDATOR ] BWLP: Primary data validation passed. [2025-10-20T17:45:56.430066] [INFO ] [MAIN ] BWLP: Fetched primary data is valid. [2025-10-20T17:45:56.430163] [DEBUG] [LOCAL ] BWLP: Saving primary data. [2025-10-20T17:45:56.431296] [DEBUG] [LOCAL ] BWLP: Trimmed 47924 records to 32227 (keeping 200 days of data) [2025-10-20T17:45:56.621477] [DEBUG] [LOCAL ] BWLP: Primary data saved (32227 records). [2025-10-20T17:45:56.621741] [INFO ] [MAIN ] BWLP: Full history saved. [2025-10-20T17:45:56.621845] [INFO ] [MAIN ] Update Progress: 366/2367 symbols completed. BWLP took 34.4s. Estimated remaining: 4h 10m 47s. Rate: 8.0 symbols/min [2025-10-20T17:45:56.797854] [INFO ] [MAIN ] BX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:56.846878] [DEBUG] [LOCAL ] BX: Loaded 54263 existing records from primary storage. [2025-10-20T17:45:56.847224] [INFO ] [MAIN ] BX: Loaded 54263 existing records. [2025-10-20T17:45:57.052055] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:45:57.055605] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-06. [2025-10-20T17:45:57.055263] [WARN ] [MAIN ] BX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:57.072288] [INFO ] [MAIN ] BX: Wiping existing primary data before full refresh. [2025-10-20T17:45:57.073098] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T17:45:57.072873] [INFO ] [LOCAL ] BX: Starting deletion of primary data. [2025-10-20T17:45:57.076638] [DEBUG] [LOCAL ] BX: Deleted primary data file. [2025-10-20T17:45:57.076728] [INFO ] [MAIN ] BX: Starting full primary refresh. [2025-10-20T17:45:57.435221] [INFO ] [REMOTE ] BX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:57.587250] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T17:45:57.851379] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-01. [2025-10-20T17:45:58.651374] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-07. [2025-10-20T17:45:59.075263] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-10-20T17:45:59.451361] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-03. [2025-10-20T17:45:59.582768] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T17:46:00.251375] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-02. [2025-10-20T17:46:01.051386] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-06. [2025-10-20T17:46:01.170686] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T17:46:01.700387] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T17:46:01.857996] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-08. [2025-10-20T17:46:01.869310] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T17:46:02.602375] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T17:46:02.651419] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-09. [2025-10-20T17:46:03.219129] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T17:46:03.451425] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-03. [2025-10-20T17:46:04.251417] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-07. [2025-10-20T17:46:04.863352] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T17:46:05.051377] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-08. [2025-10-20T17:46:05.644568] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-10-20T17:46:05.851345] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-04. [2025-10-20T17:46:05.914054] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T17:46:06.651388] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-09. [2025-10-20T17:46:07.166425] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-10-20T17:46:07.451370] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-10. [2025-10-20T17:46:07.511337] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T17:46:07.956543] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T17:46:07.957488] [INFO ] [REMOTE ] BWNB: Full fetch complete (5579 records). [2025-10-20T17:46:07.957577] [DEBUG] [VALIDATOR ] BWNB: Validating primary data. [2025-10-20T17:46:08.259762] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-04. [2025-10-20T17:46:08.349393] [DEBUG] [VALIDATOR ] BWNB: All expected trading days are present. [2025-10-20T17:46:08.349467] [DEBUG] [VALIDATOR ] BWNB: Primary data validation passed. [2025-10-20T17:46:08.349631] [INFO ] [MAIN ] BWNB: Fetched primary data is valid. [2025-10-20T17:46:08.349707] [DEBUG] [LOCAL ] BWNB: Saving primary data. [2025-10-20T17:46:08.350092] [DEBUG] [LOCAL ] BWNB: Trimmed 5579 records to 4101 (keeping 200 days of data) [2025-10-20T17:46:08.374742] [DEBUG] [LOCAL ] BWNB: Primary data saved (4101 records). [2025-10-20T17:46:08.374840] [INFO ] [MAIN ] BWNB: Full history saved. [2025-10-20T17:46:08.374923] [INFO ] [MAIN ] Update Progress: 367/2367 symbols completed. BWNB took 34.8s. Estimated remaining: 4h 11m 3s. Rate: 8.0 symbols/min [2025-10-20T17:46:08.564341] [INFO ] [MAIN ] BXC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:08.577371] [DEBUG] [LOCAL ] BXC: Loaded 13776 existing records from primary storage. [2025-10-20T17:46:08.577596] [INFO ] [MAIN ] BXC: Loaded 13776 existing records. [2025-10-20T17:46:08.762681] [WARN ] [MAIN ] BXC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:08.762923] [INFO ] [MAIN ] BXC: Wiping existing primary data before full refresh. [2025-10-20T17:46:08.762970] [INFO ] [LOCAL ] BXC: Starting deletion of primary data. [2025-10-20T17:46:08.763191] [DEBUG] [LOCAL ] BXC: Deleted primary data file. [2025-10-20T17:46:08.763248] [INFO ] [MAIN ] BXC: Starting full primary refresh. [2025-10-20T17:46:09.059887] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-10. [2025-10-20T17:46:09.155758] [INFO ] [REMOTE ] BXC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:09.821119] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T17:46:09.822451] [INFO ] [REMOTE ] BWSN: Full fetch complete (6353 records). [2025-10-20T17:46:09.822518] [DEBUG] [VALIDATOR ] BWSN: Validating primary data. [2025-10-20T17:46:09.856439] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-01. [2025-10-20T17:46:10.197788] [DEBUG] [VALIDATOR ] BWSN: All expected trading days are present. [2025-10-20T17:46:10.197905] [DEBUG] [VALIDATOR ] BWSN: Primary data validation passed. [2025-10-20T17:46:10.198187] [INFO ] [MAIN ] BWSN: Fetched primary data is valid. [2025-10-20T17:46:10.198340] [DEBUG] [LOCAL ] BWSN: Saving primary data. [2025-10-20T17:46:10.198901] [DEBUG] [LOCAL ] BWSN: Trimmed 6353 records to 5120 (keeping 200 days of data) [2025-10-20T17:46:10.233875] [DEBUG] [LOCAL ] BWSN: Primary data saved (5120 records). [2025-10-20T17:46:10.234006] [INFO ] [MAIN ] BWSN: Full history saved. [2025-10-20T17:46:10.234084] [INFO ] [MAIN ] Update Progress: 368/2367 symbols completed. BWSN took 32.4s. Estimated remaining: 4h 10m 24s. Rate: 8.0 symbols/min [2025-10-20T17:46:10.403265] [INFO ] [MAIN ] BXMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:10.436305] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T17:46:10.483696] [DEBUG] [LOCAL ] BXMT: Loaded 48048 existing records from primary storage. [2025-10-20T17:46:10.484017] [INFO ] [MAIN ] BXMT: Loaded 48048 existing records. [2025-10-20T17:46:10.661908] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-05. [2025-10-20T17:46:10.668066] [WARN ] [MAIN ] BXMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:10.669444] [INFO ] [MAIN ] BXMT: Wiping existing primary data before full refresh. [2025-10-20T17:46:10.669538] [INFO ] [LOCAL ] BXMT: Starting deletion of primary data. [2025-10-20T17:46:10.674142] [DEBUG] [LOCAL ] BXMT: Deleted primary data file. [2025-10-20T17:46:10.675443] [INFO ] [MAIN ] BXMT: Starting full primary refresh. [2025-10-20T17:46:10.676291] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T17:46:11.050710] [INFO ] [REMOTE ] BXMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:11.451517] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-01. [2025-10-20T17:46:12.251440] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-02. [2025-10-20T17:46:12.434836] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:46:12.779511] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T17:46:12.905331] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:46:13.051429] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-02. [2025-10-20T17:46:13.851428] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-05. [2025-10-20T17:46:14.651401] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-03. [2025-10-20T17:46:14.702381] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:46:15.451444] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-06. [2025-10-20T17:46:15.463008] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:46:15.752568] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T17:46:16.251434] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-03. [2025-10-20T17:46:17.051471] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-04. [2025-10-20T17:46:17.177817] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:46:17.851364] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-07. [2025-10-20T17:46:18.017088] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T17:46:18.023186] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:46:18.651402] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-05. [2025-10-20T17:46:19.451422] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-04. [2025-10-20T17:46:19.546344] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T17:46:19.796876] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T17:46:20.251360] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-06. [2025-10-20T17:46:21.051402] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-06. [2025-10-20T17:46:21.210554] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T17:46:21.259819] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:46:21.851379] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-07. [2025-10-20T17:46:22.651347] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-05. [2025-10-20T17:46:22.789724] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:46:22.981257] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T17:46:23.451439] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-08. [2025-10-20T17:46:24.263504] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-07. [2025-10-20T17:46:24.284232] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T17:46:24.759913] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:46:25.051361] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-09. [2025-10-20T17:46:25.851351] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-06. [2025-10-20T17:46:26.110840] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T17:46:26.651431] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-08. [2025-10-20T17:46:26.969568] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:46:27.016011] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T17:46:27.451426] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-08. [2025-10-20T17:46:27.466203] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T17:46:28.251382] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-07. [2025-10-20T17:46:28.899259] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T17:46:29.051429] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-10. [2025-10-20T17:46:29.831328] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T17:46:29.851341] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-08. [2025-10-20T17:46:30.651421] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-09. [2025-10-20T17:46:30.955996] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T17:46:30.963910] [INFO ] [REMOTE ] BWXT: Full fetch complete (68036 records). [2025-10-20T17:46:30.964094] [DEBUG] [VALIDATOR ] BWXT: Validating primary data. [2025-10-20T17:46:31.372491] [DEBUG] [VALIDATOR ] BWXT: All expected trading days are present. [2025-10-20T17:46:31.372616] [DEBUG] [VALIDATOR ] BWXT: Primary data validation passed. [2025-10-20T17:46:31.373630] [INFO ] [MAIN ] BWXT: Fetched primary data is valid. [2025-10-20T17:46:31.373951] [DEBUG] [LOCAL ] BWXT: Saving primary data. [2025-10-20T17:46:31.383948] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-10-20T17:46:31.388167] [DEBUG] [LOCAL ] BWXT: Trimmed 68036 records to 47285 (keeping 200 days of data) [2025-10-20T17:46:31.453189] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-09. [2025-10-20T17:46:31.529350] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:46:31.718280] [DEBUG] [LOCAL ] BWXT: Primary data saved (47285 records). [2025-10-20T17:46:31.718453] [INFO ] [MAIN ] BWXT: Full history saved. [2025-10-20T17:46:31.719206] [INFO ] [MAIN ] Update Progress: 369/2367 symbols completed. BWXT took 41.5s. Estimated remaining: 4h 11m 32s. Rate: 7.9 symbols/min [2025-10-20T17:46:31.904216] [INFO ] [MAIN ] BXMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:31.927063] [DEBUG] [LOCAL ] BXMX: Loaded 19717 existing records from primary storage. [2025-10-20T17:46:31.927433] [INFO ] [MAIN ] BXMX: Loaded 19717 existing records. [2025-10-20T17:46:32.142303] [WARN ] [MAIN ] BXMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:32.142566] [INFO ] [MAIN ] BXMX: Wiping existing primary data before full refresh. [2025-10-20T17:46:32.142614] [INFO ] [LOCAL ] BXMX: Starting deletion of primary data. [2025-10-20T17:46:32.142844] [DEBUG] [LOCAL ] BXMX: Deleted primary data file. [2025-10-20T17:46:32.142898] [INFO ] [MAIN ] BXMX: Starting full primary refresh. [2025-10-20T17:46:32.256696] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-09. [2025-10-20T17:46:32.521635] [INFO ] [REMOTE ] BXMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:33.051414] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-01. [2025-10-20T17:46:33.679394] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-10-20T17:46:33.851379] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-10. [2025-10-20T17:46:34.091520] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T17:46:34.134983] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T17:46:34.651415] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-02. [2025-10-20T17:46:34.750938] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:46:34.754129] [INFO ] [REMOTE ] BXC: Full fetch complete (18287 records). [2025-10-20T17:46:34.754231] [DEBUG] [VALIDATOR ] BXC: Validating primary data. [2025-10-20T17:46:35.135305] [DEBUG] [VALIDATOR ] BXC: All expected trading days are present. [2025-10-20T17:46:35.135405] [DEBUG] [VALIDATOR ] BXC: Primary data validation passed. [2025-10-20T17:46:35.135773] [INFO ] [MAIN ] BXC: Fetched primary data is valid. [2025-10-20T17:46:35.135833] [DEBUG] [LOCAL ] BXC: Saving primary data. [2025-10-20T17:46:35.136391] [DEBUG] [LOCAL ] BXC: Trimmed 18287 records to 13571 (keeping 200 days of data) [2025-10-20T17:46:35.213040] [DEBUG] [LOCAL ] BXC: Primary data saved (13571 records). [2025-10-20T17:46:35.213177] [INFO ] [MAIN ] BXC: Full history saved. [2025-10-20T17:46:35.213251] [INFO ] [MAIN ] Update Progress: 370/2367 symbols completed. BXC took 26.8s. Estimated remaining: 4h 11m 3s. Rate: 8.0 symbols/min [2025-10-20T17:46:35.381948] [INFO ] [MAIN ] BXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:35.428237] [DEBUG] [LOCAL ] BXP: Loaded 50594 existing records from primary storage. [2025-10-20T17:46:35.428598] [INFO ] [MAIN ] BXP: Loaded 50594 existing records. [2025-10-20T17:46:35.459570] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-10. [2025-10-20T17:46:35.632474] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:46:35.636592] [WARN ] [MAIN ] BXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:35.641673] [INFO ] [MAIN ] BXP: Wiping existing primary data before full refresh. [2025-10-20T17:46:35.641752] [INFO ] [LOCAL ] BXP: Starting deletion of primary data. [2025-10-20T17:46:35.641952] [DEBUG] [LOCAL ] BXP: Deleted primary data file. [2025-10-20T17:46:35.641992] [INFO ] [MAIN ] BXP: Starting full primary refresh. [2025-10-20T17:46:36.021250] [INFO ] [REMOTE ] BXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:36.251399] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-01. [2025-10-20T17:46:37.051436] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-03. [2025-10-20T17:46:37.317093] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T17:46:37.324820] [INFO ] [REMOTE ] BXMT: Full fetch complete (69895 records). [2025-10-20T17:46:37.324988] [DEBUG] [VALIDATOR ] BXMT: Validating primary data. [2025-10-20T17:46:37.719488] [DEBUG] [VALIDATOR ] BXMT: All expected trading days are present. [2025-10-20T17:46:37.719632] [DEBUG] [VALIDATOR ] BXMT: Primary data validation passed. [2025-10-20T17:46:37.720581] [INFO ] [MAIN ] BXMT: Fetched primary data is valid. [2025-10-20T17:46:37.720690] [DEBUG] [LOCAL ] BXMT: Saving primary data. [2025-10-20T17:46:37.722159] [DEBUG] [LOCAL ] BXMT: Trimmed 69895 records to 47292 (keeping 200 days of data) [2025-10-20T17:46:37.855048] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-10. [2025-10-20T17:46:37.862953] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:46:38.065752] [DEBUG] [LOCAL ] BXMT: Primary data saved (47292 records). [2025-10-20T17:46:38.067326] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T17:46:38.066154] [INFO ] [MAIN ] BXMT: Full history saved. [2025-10-20T17:46:38.067501] [INFO ] [MAIN ] Update Progress: 371/2367 symbols completed. BXMT took 27.8s. Estimated remaining: 4h 10m 30s. Rate: 8.0 symbols/min [2025-10-20T17:46:38.248102] [INFO ] [MAIN ] BXSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:38.289873] [DEBUG] [LOCAL ] BXSL: Loaded 48894 existing records from primary storage. [2025-10-20T17:46:38.290161] [INFO ] [MAIN ] BXSL: Loaded 48894 existing records. [2025-10-20T17:46:38.465248] [WARN ] [MAIN ] BXSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:38.465479] [INFO ] [MAIN ] BXSL: Wiping existing primary data before full refresh. [2025-10-20T17:46:38.465557] [INFO ] [LOCAL ] BXSL: Starting deletion of primary data. [2025-10-20T17:46:38.465810] [DEBUG] [LOCAL ] BXSL: Deleted primary data file. [2025-10-20T17:46:38.465880] [INFO ] [MAIN ] BXSL: Starting full primary refresh. [2025-10-20T17:46:38.656525] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-04. [2025-10-20T17:46:38.829092] [INFO ] [REMOTE ] BXSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:39.451495] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-02. [2025-10-20T17:46:39.937314] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-10-20T17:46:39.977410] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T17:46:39.985885] [INFO ] [REMOTE ] BX: Full fetch complete (77591 records). [2025-10-20T17:46:39.986050] [DEBUG] [VALIDATOR ] BX: Validating primary data. [2025-10-20T17:46:40.256941] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-05. [2025-10-20T17:46:40.388813] [DEBUG] [VALIDATOR ] BX: All expected trading days are present. [2025-10-20T17:46:40.388897] [DEBUG] [VALIDATOR ] BX: Primary data validation passed. [2025-10-20T17:46:40.389685] [INFO ] [MAIN ] BX: Fetched primary data is valid. [2025-10-20T17:46:40.389746] [DEBUG] [LOCAL ] BX: Saving primary data. [2025-10-20T17:46:40.391181] [DEBUG] [LOCAL ] BX: Trimmed 77591 records to 53483 (keeping 200 days of data) [2025-10-20T17:46:40.719312] [DEBUG] [LOCAL ] BX: Primary data saved (53483 records). [2025-10-20T17:46:40.719452] [INFO ] [MAIN ] BX: Full history saved. [2025-10-20T17:46:40.719530] [INFO ] [MAIN ] Update Progress: 372/2367 symbols completed. BX took 44.1s. Estimated remaining: 4h 9m 56s. Rate: 8.0 symbols/min [2025-10-20T17:46:40.896406] [INFO ] [MAIN ] BY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:40.942282] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:46:40.940764] [DEBUG] [LOCAL ] BY: Loaded 22175 existing records from primary storage. [2025-10-20T17:46:40.952735] [INFO ] [MAIN ] BY: Loaded 22175 existing records. [2025-10-20T17:46:41.056503] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-01. [2025-10-20T17:46:41.138132] [WARN ] [MAIN ] BY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:41.138480] [INFO ] [MAIN ] BY: Wiping existing primary data before full refresh. [2025-10-20T17:46:41.138529] [INFO ] [LOCAL ] BY: Starting deletion of primary data. [2025-10-20T17:46:41.138846] [DEBUG] [LOCAL ] BY: Deleted primary data file. [2025-10-20T17:46:41.138902] [INFO ] [MAIN ] BY: Starting full primary refresh. [2025-10-20T17:46:41.521589] [INFO ] [REMOTE ] BY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:41.523020] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T17:46:41.851424] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-03. [2025-10-20T17:46:42.551095] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T17:46:42.651486] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-01. [2025-10-20T17:46:43.451431] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-06. [2025-10-20T17:46:43.520135] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:46:43.559991] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:46:44.251361] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-04. [2025-10-20T17:46:44.330498] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T17:46:45.051432] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-02. [2025-10-20T17:46:45.851456] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-07. [2025-10-20T17:46:45.949274] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T17:46:46.037804] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:46:46.495593] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T17:46:46.651441] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-02. [2025-10-20T17:46:47.451469] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-03. [2025-10-20T17:46:48.245710] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T17:46:48.254975] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-05. [2025-10-20T17:46:48.423407] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T17:46:49.051371] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-08. [2025-10-20T17:46:49.851394] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-03. [2025-10-20T17:46:49.908591] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T17:46:49.976960] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:46:50.651372] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-06. [2025-10-20T17:46:51.218874] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:46:51.451387] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-04. [2025-10-20T17:46:52.251344] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-09. [2025-10-20T17:46:52.366217] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T17:46:52.840422] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:46:53.051377] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-07. [2025-10-20T17:46:53.259604] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T17:46:53.851425] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-05. [2025-10-20T17:46:54.425464] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T17:46:54.651366] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-08. [2025-10-20T17:46:54.923359] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:46:55.451374] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-06. [2025-10-20T17:46:56.251370] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-10. [2025-10-20T17:46:56.315185] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T17:46:56.582905] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T17:46:57.051345] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-07. [2025-10-20T17:46:57.117898] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T17:46:57.122619] [INFO ] [REMOTE ] BXMX: Full fetch complete (28676 records). [2025-10-20T17:46:57.122770] [DEBUG] [VALIDATOR ] BXMX: Validating primary data. [2025-10-20T17:46:57.557607] [DEBUG] [VALIDATOR ] BXMX: All expected trading days are present. [2025-10-20T17:46:57.557768] [DEBUG] [VALIDATOR ] BXMX: Primary data validation passed. [2025-10-20T17:46:57.558427] [INFO ] [MAIN ] BXMX: Fetched primary data is valid. [2025-10-20T17:46:57.558496] [DEBUG] [LOCAL ] BXMX: Saving primary data. [2025-10-20T17:46:57.559751] [DEBUG] [LOCAL ] BXMX: Trimmed 28676 records to 19355 (keeping 200 days of data) [2025-10-20T17:46:57.681978] [DEBUG] [LOCAL ] BXMX: Primary data saved (19355 records). [2025-10-20T17:46:57.682185] [INFO ] [MAIN ] BXMX: Full history saved. [2025-10-20T17:46:57.682287] [INFO ] [MAIN ] Update Progress: 373/2367 symbols completed. BXMX took 26.0s. Estimated remaining: 4h 10m 39s. Rate: 8.0 symbols/min [2025-10-20T17:46:57.862715] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-09. [2025-10-20T17:46:57.866451] [INFO ] [MAIN ] BYD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:57.916980] [DEBUG] [LOCAL ] BYD: Loaded 45593 existing records from primary storage. [2025-10-20T17:46:57.917486] [INFO ] [MAIN ] BYD: Loaded 45593 existing records. [2025-10-20T17:46:58.131716] [WARN ] [MAIN ] BYD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:58.146842] [INFO ] [MAIN ] BYD: Wiping existing primary data before full refresh. [2025-10-20T17:46:58.147044] [INFO ] [LOCAL ] BYD: Starting deletion of primary data. [2025-10-20T17:46:58.147689] [DEBUG] [LOCAL ] BYD: Deleted primary data file. [2025-10-20T17:46:58.148207] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T17:46:58.148461] [INFO ] [MAIN ] BYD: Starting full primary refresh. [2025-10-20T17:46:58.538495] [INFO ] [REMOTE ] BYD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:58.651448] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-01. [2025-10-20T17:46:59.444236] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T17:46:59.458127] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-08. [2025-10-20T17:47:00.064813] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T17:47:00.251357] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-02. [2025-10-20T17:47:00.338967] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T17:47:01.051370] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-10. [2025-10-20T17:47:01.851436] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-04. [2025-10-20T17:47:01.912626] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:47:02.651387] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-03. [2025-10-20T17:47:02.893805] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T17:47:02.901371] [INFO ] [REMOTE ] BXP: Full fetch complete (71886 records). [2025-10-20T17:47:02.901515] [DEBUG] [VALIDATOR ] BXP: Validating primary data. [2025-10-20T17:47:02.902361] [FATAL] [VALIDATOR ] BXP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:47:02.902452] [FATAL] [MAIN ] BXP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:47:02.902496] [WARN ] [MAIN ] BXP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:47:02.902537] [INFO ] [LOCAL ] BXP: Starting deletion of primary data. [2025-10-20T17:47:02.902598] [DEBUG] [LOCAL ] BXP: No primary data file found to delete. [2025-10-20T17:47:02.902635] [INFO ] [LOCAL ] BXP: Starting deletion of derivative data. [2025-10-20T17:47:02.902879] [DEBUG] [LOCAL ] BXP: Deleted derivative data file. [2025-10-20T17:47:02.903206] [INFO ] [MAIN ] Update Progress: 374/2367 symbols completed. BXP took 27.7s. Estimated remaining: 4h 10m 19s. Rate: 8.0 symbols/min [2025-10-20T17:47:03.089181] [INFO ] [MAIN ] BYM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:03.103663] [DEBUG] [LOCAL ] BYM: Loaded 11842 existing records from primary storage. [2025-10-20T17:47:03.103904] [INFO ] [MAIN ] BYM: Loaded 11842 existing records. [2025-10-20T17:47:03.277215] [WARN ] [MAIN ] BYM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:03.277604] [INFO ] [MAIN ] BYM: Wiping existing primary data before full refresh. [2025-10-20T17:47:03.277680] [INFO ] [LOCAL ] BYM: Starting deletion of primary data. [2025-10-20T17:47:03.277861] [DEBUG] [LOCAL ] BYM: Deleted primary data file. [2025-10-20T17:47:03.277899] [INFO ] [MAIN ] BYM: Starting full primary refresh. [2025-10-20T17:47:03.462175] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-09. [2025-10-20T17:47:03.634851] [INFO ] [REMOTE ] BYM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:03.733388] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T17:47:04.256685] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-05. [2025-10-20T17:47:04.258118] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T17:47:04.305700] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T17:47:05.051362] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-04. [2025-10-20T17:47:05.720709] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T17:47:05.851393] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-06. [2025-10-20T17:47:06.530396] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:47:06.651374] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-05. [2025-10-20T17:47:07.385229] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T17:47:07.451351] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-10. [2025-10-20T17:47:08.215918] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:47:08.251332] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-06. [2025-10-20T17:47:08.448762] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:47:08.452152] [INFO ] [REMOTE ] BY: Full fetch complete (29694 records). [2025-10-20T17:47:08.452237] [DEBUG] [VALIDATOR ] BY: Validating primary data. [2025-10-20T17:47:08.851464] [DEBUG] [VALIDATOR ] BY: All expected trading days are present. [2025-10-20T17:47:08.851580] [DEBUG] [VALIDATOR ] BY: Primary data validation passed. [2025-10-20T17:47:08.852064] [INFO ] [MAIN ] BY: Fetched primary data is valid. [2025-10-20T17:47:08.852125] [DEBUG] [LOCAL ] BY: Saving primary data. [2025-10-20T17:47:08.853099] [DEBUG] [LOCAL ] BY: Trimmed 29694 records to 21868 (keeping 200 days of data) [2025-10-20T17:47:08.982605] [DEBUG] [LOCAL ] BY: Primary data saved (21868 records). [2025-10-20T17:47:08.982763] [INFO ] [MAIN ] BY: Full history saved. [2025-10-20T17:47:08.982847] [INFO ] [MAIN ] Update Progress: 375/2367 symbols completed. BY took 28.3s. Estimated remaining: 4h 10m 4s. Rate: 8.0 symbols/min [2025-10-20T17:47:09.060009] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-07. [2025-10-20T17:47:09.159350] [INFO ] [MAIN ] BZH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:09.189917] [DEBUG] [LOCAL ] BZH: Loaded 32798 existing records from primary storage. [2025-10-20T17:47:09.190209] [INFO ] [MAIN ] BZH: Loaded 32798 existing records. [2025-10-20T17:47:09.379786] [WARN ] [MAIN ] BZH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:09.380011] [INFO ] [MAIN ] BZH: Wiping existing primary data before full refresh. [2025-10-20T17:47:09.380059] [INFO ] [LOCAL ] BZH: Starting deletion of primary data. [2025-10-20T17:47:09.380272] [DEBUG] [LOCAL ] BZH: Deleted primary data file. [2025-10-20T17:47:09.380332] [INFO ] [MAIN ] BZH: Starting full primary refresh. [2025-10-20T17:47:09.757222] [INFO ] [REMOTE ] BZH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:09.791780] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T17:47:09.851440] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-01. [2025-10-20T17:47:10.627578] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T17:47:10.651350] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-02. [2025-10-20T17:47:10.730157] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T17:47:11.304352] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:47:11.451441] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-07. [2025-10-20T17:47:12.251461] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-08. [2025-10-20T17:47:13.054485] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-03. [2025-10-20T17:47:13.062451] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T17:47:13.851389] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-08. [2025-10-20T17:47:13.996955] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T17:47:14.046975] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:47:14.651438] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-01. [2025-10-20T17:47:15.244100] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:47:15.451441] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-09. [2025-10-20T17:47:15.918743] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:47:16.251382] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-02. [2025-10-20T17:47:16.855605] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T17:47:17.051460] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-10. [2025-10-20T17:47:17.742611] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:47:17.851416] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-03. [2025-10-20T17:47:18.302030] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T17:47:18.309193] [INFO ] [REMOTE ] BYD: Full fetch complete (65028 records). [2025-10-20T17:47:18.309411] [DEBUG] [VALIDATOR ] BYD: Validating primary data. [2025-10-20T17:47:18.310741] [FATAL] [VALIDATOR ] BYD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:47:18.310863] [FATAL] [MAIN ] BYD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:47:18.310932] [WARN ] [MAIN ] BYD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:47:18.311035] [INFO ] [LOCAL ] BYD: Starting deletion of primary data. [2025-10-20T17:47:18.311125] [DEBUG] [LOCAL ] BYD: No primary data file found to delete. [2025-10-20T17:47:18.311166] [INFO ] [LOCAL ] BYD: Starting deletion of derivative data. [2025-10-20T17:47:18.311618] [DEBUG] [LOCAL ] BYD: Deleted derivative data file. [2025-10-20T17:47:18.312082] [INFO ] [MAIN ] Update Progress: 376/2367 symbols completed. BYD took 20.6s. Estimated remaining: 4h 10m 6s. Rate: 8.0 symbols/min [2025-10-20T17:47:18.513076] [INFO ] [MAIN ] C: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:18.567807] [DEBUG] [LOCAL ] C: Loaded 54520 existing records from primary storage. [2025-10-20T17:47:18.568219] [INFO ] [MAIN ] C: Loaded 54520 existing records. [2025-10-20T17:47:18.656563] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-04. [2025-10-20T17:47:18.782907] [WARN ] [MAIN ] C: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:18.783122] [INFO ] [MAIN ] C: Wiping existing primary data before full refresh. [2025-10-20T17:47:18.783169] [INFO ] [LOCAL ] C: Starting deletion of primary data. [2025-10-20T17:47:18.783670] [DEBUG] [LOCAL ] C: Deleted primary data file. [2025-10-20T17:47:18.783729] [INFO ] [MAIN ] C: Starting full primary refresh. [2025-10-20T17:47:19.160808] [INFO ] [REMOTE ] C: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:19.258182] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T17:47:19.451421] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-09. [2025-10-20T17:47:19.637212] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T17:47:20.251378] [INFO ] [REMOTE ] C: Fetching primary data for 2025-01. [2025-10-20T17:47:21.051404] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-04. [2025-10-20T17:47:21.292612] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T17:47:21.851382] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-10. [2025-10-20T17:47:22.356322] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:47:22.474040] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T17:47:22.651404] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-05. [2025-10-20T17:47:23.451365] [INFO ] [REMOTE ] C: Fetching primary data for 2025-02. [2025-10-20T17:47:23.808410] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T17:47:23.816287] [INFO ] [REMOTE ] BXSL: Full fetch complete (70017 records). [2025-10-20T17:47:23.816439] [DEBUG] [VALIDATOR ] BXSL: Validating primary data. [2025-10-20T17:47:24.129802] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T17:47:24.230281] [DEBUG] [VALIDATOR ] BXSL: All expected trading days are present. [2025-10-20T17:47:24.230378] [DEBUG] [VALIDATOR ] BXSL: Primary data validation passed. [2025-10-20T17:47:24.231147] [INFO ] [MAIN ] BXSL: Fetched primary data is valid. [2025-10-20T17:47:24.231213] [DEBUG] [LOCAL ] BXSL: Saving primary data. [2025-10-20T17:47:24.232596] [DEBUG] [LOCAL ] BXSL: Trimmed 70017 records to 48208 (keeping 200 days of data) [2025-10-20T17:47:24.284922] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-05. [2025-10-20T17:47:24.538113] [DEBUG] [LOCAL ] BXSL: Primary data saved (48208 records). [2025-10-20T17:47:24.538278] [INFO ] [MAIN ] BXSL: Full history saved. [2025-10-20T17:47:24.538423] [INFO ] [MAIN ] Update Progress: 377/2367 symbols completed. BXSL took 46.5s. Estimated remaining: 4h 9m 52s. Rate: 8.0 symbols/min [2025-10-20T17:47:24.712572] [INFO ] [MAIN ] CAAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:24.733067] [DEBUG] [LOCAL ] CAAP: Loaded 20139 existing records from primary storage. [2025-10-20T17:47:24.733308] [INFO ] [MAIN ] CAAP: Loaded 20139 existing records. [2025-10-20T17:47:24.905463] [WARN ] [MAIN ] CAAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:24.905632] [INFO ] [MAIN ] CAAP: Wiping existing primary data before full refresh. [2025-10-20T17:47:24.905710] [INFO ] [LOCAL ] CAAP: Starting deletion of primary data. [2025-10-20T17:47:24.905915] [DEBUG] [LOCAL ] CAAP: Deleted primary data file. [2025-10-20T17:47:24.905971] [INFO ] [MAIN ] CAAP: Starting full primary refresh. [2025-10-20T17:47:25.062767] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-06. [2025-10-20T17:47:25.196216] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T17:47:25.279955] [INFO ] [REMOTE ] CAAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:25.318628] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T17:47:25.851432] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-06. [2025-10-20T17:47:26.389924] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:47:26.651350] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-01. [2025-10-20T17:47:26.699783] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:47:27.451353] [INFO ] [REMOTE ] C: Fetching primary data for 2025-03. [2025-10-20T17:47:27.840997] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:47:28.251538] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-07. [2025-10-20T17:47:29.056228] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-07. [2025-10-20T17:47:29.058165] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T17:47:29.296020] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T17:47:29.851388] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-08. [2025-10-20T17:47:30.417253] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:47:30.431024] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T17:47:30.651355] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-08. [2025-10-20T17:47:31.451383] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-09. [2025-10-20T17:47:31.772264] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:47:32.251357] [INFO ] [REMOTE ] C: Fetching primary data for 2025-04. [2025-10-20T17:47:32.310390] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T17:47:33.051394] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-09. [2025-10-20T17:47:33.851424] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-02. [2025-10-20T17:47:34.385572] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T17:47:34.433816] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T17:47:34.651439] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-03. [2025-10-20T17:47:34.726803] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-04 (2.48 seconds). [2025-10-20T17:47:35.451431] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-10. [2025-10-20T17:47:35.648315] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T17:47:36.251427] [INFO ] [REMOTE ] C: Fetching primary data for 2025-05. [2025-10-20T17:47:36.439972] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T17:47:36.442272] [INFO ] [REMOTE ] BYM: Full fetch complete (17337 records). [2025-10-20T17:47:36.442399] [DEBUG] [VALIDATOR ] BYM: Validating primary data. [2025-10-20T17:47:36.858673] [DEBUG] [VALIDATOR ] BYM: All expected trading days are present. [2025-10-20T17:47:36.858768] [DEBUG] [VALIDATOR ] BYM: Primary data validation passed. [2025-10-20T17:47:36.859099] [INFO ] [MAIN ] BYM: Fetched primary data is valid. [2025-10-20T17:47:36.859140] [DEBUG] [LOCAL ] BYM: Saving primary data. [2025-10-20T17:47:36.859781] [DEBUG] [LOCAL ] BYM: Trimmed 17337 records to 11543 (keeping 200 days of data) [2025-10-20T17:47:36.927313] [DEBUG] [LOCAL ] BYM: Primary data saved (11543 records). [2025-10-20T17:47:36.927448] [INFO ] [MAIN ] BYM: Full history saved. [2025-10-20T17:47:36.927527] [INFO ] [MAIN ] Update Progress: 378/2367 symbols completed. BYM took 34.0s. Estimated remaining: 4h 10m 10s. Rate: 8.0 symbols/min [2025-10-20T17:47:37.056559] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-04. [2025-10-20T17:47:37.102535] [INFO ] [MAIN ] CABO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:37.119220] [DEBUG] [LOCAL ] CABO: Loaded 16919 existing records from primary storage. [2025-10-20T17:47:37.119468] [INFO ] [MAIN ] CABO: Loaded 16919 existing records. [2025-10-20T17:47:37.299941] [WARN ] [MAIN ] CABO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:37.300167] [INFO ] [MAIN ] CABO: Wiping existing primary data before full refresh. [2025-10-20T17:47:37.300217] [INFO ] [LOCAL ] CABO: Starting deletion of primary data. [2025-10-20T17:47:37.300379] [DEBUG] [LOCAL ] CABO: Deleted primary data file. [2025-10-20T17:47:37.300420] [INFO ] [MAIN ] CABO: Starting full primary refresh. [2025-10-20T17:47:37.661724] [INFO ] [REMOTE ] CABO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:37.851418] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-10. [2025-10-20T17:47:37.989522] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:47:38.013677] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-10-20T17:47:38.651476] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-01. [2025-10-20T17:47:39.185537] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:47:39.190591] [INFO ] [REMOTE ] BZH: Full fetch complete (47167 records). [2025-10-20T17:47:39.190785] [DEBUG] [VALIDATOR ] BZH: Validating primary data. [2025-10-20T17:47:39.456514] [INFO ] [REMOTE ] C: Fetching primary data for 2025-06. [2025-10-20T17:47:39.607382] [DEBUG] [VALIDATOR ] BZH: All expected trading days are present. [2025-10-20T17:47:39.616758] [DEBUG] [VALIDATOR ] BZH: Primary data validation passed. [2025-10-20T17:47:39.617956] [INFO ] [MAIN ] BZH: Fetched primary data is valid. [2025-10-20T17:47:39.618377] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T17:47:39.621878] [DEBUG] [LOCAL ] BZH: Saving primary data. [2025-10-20T17:47:39.623575] [DEBUG] [LOCAL ] BZH: Trimmed 47167 records to 32291 (keeping 200 days of data) [2025-10-20T17:47:39.814925] [DEBUG] [LOCAL ] BZH: Primary data saved (32291 records). [2025-10-20T17:47:39.815074] [INFO ] [MAIN ] BZH: Full history saved. [2025-10-20T17:47:39.815153] [INFO ] [MAIN ] Update Progress: 379/2367 symbols completed. BZH took 30.8s. Estimated remaining: 4h 9m 38s. Rate: 8.0 symbols/min [2025-10-20T17:47:39.982410] [INFO ] [MAIN ] CACI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:40.004163] [DEBUG] [LOCAL ] CACI: Loaded 23405 existing records from primary storage. [2025-10-20T17:47:40.004450] [INFO ] [MAIN ] CACI: Loaded 23405 existing records. [2025-10-20T17:47:40.181063] [WARN ] [MAIN ] CACI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:40.181289] [INFO ] [MAIN ] CACI: Wiping existing primary data before full refresh. [2025-10-20T17:47:40.181336] [INFO ] [LOCAL ] CACI: Starting deletion of primary data. [2025-10-20T17:47:40.181526] [DEBUG] [LOCAL ] CACI: Deleted primary data file. [2025-10-20T17:47:40.181564] [INFO ] [MAIN ] CACI: Starting full primary refresh. [2025-10-20T17:47:40.256590] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-02. [2025-10-20T17:47:40.544039] [INFO ] [REMOTE ] CACI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:41.031260] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T17:47:41.051364] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-01. [2025-10-20T17:47:41.275925] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T17:47:41.851412] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-05. [2025-10-20T17:47:42.385003] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T17:47:42.651396] [INFO ] [REMOTE ] C: Fetching primary data for 2025-07. [2025-10-20T17:47:42.699519] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T17:47:43.451441] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-03. [2025-10-20T17:47:44.211512] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T17:47:44.251392] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-06. [2025-10-20T17:47:44.836471] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T17:47:44.893779] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:47:45.051388] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-02. [2025-10-20T17:47:45.851363] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-04. [2025-10-20T17:47:46.488519] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T17:47:46.651409] [INFO ] [REMOTE ] C: Fetching primary data for 2025-08. [2025-10-20T17:47:46.723395] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T17:47:47.451435] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-05. [2025-10-20T17:47:48.251419] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-07. [2025-10-20T17:47:48.582690] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:47:48.645672] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T17:47:49.051386] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-06. [2025-10-20T17:47:49.097821] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T17:47:49.851357] [INFO ] [REMOTE ] C: Fetching primary data for 2025-09. [2025-10-20T17:47:50.187213] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:47:50.651374] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-07. [2025-10-20T17:47:51.451343] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-08. [2025-10-20T17:47:51.550132] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T17:47:51.697059] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:47:52.251378] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-08. [2025-10-20T17:47:52.465430] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:47:53.051350] [INFO ] [REMOTE ] C: Fetching primary data for 2025-10. [2025-10-20T17:47:53.207057] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:47:53.851377] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-09. [2025-10-20T17:47:54.651356] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-09. [2025-10-20T17:47:55.018062] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T17:47:55.461368] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-10. [2025-10-20T17:47:55.466427] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-10 (2.42 seconds). [2025-10-20T17:47:55.483688] [INFO ] [REMOTE ] C: Full fetch complete (77528 records). [2025-10-20T17:47:55.484777] [DEBUG] [VALIDATOR ] C: Validating primary data. [2025-10-20T17:47:55.484693] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T17:47:55.489352] [FATAL] [VALIDATOR ] C: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:47:55.489504] [FATAL] [MAIN ] C: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:47:55.489581] [WARN ] [MAIN ] C: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:47:55.489695] [INFO ] [LOCAL ] C: Starting deletion of primary data. [2025-10-20T17:47:55.489783] [DEBUG] [LOCAL ] C: No primary data file found to delete. [2025-10-20T17:47:55.489834] [INFO ] [LOCAL ] C: Starting deletion of derivative data. [2025-10-20T17:47:55.490005] [DEBUG] [LOCAL ] C: Deleted derivative data file. [2025-10-20T17:47:55.490109] [INFO ] [MAIN ] Update Progress: 380/2367 symbols completed. C took 37.2s. Estimated remaining: 4h 10m 13s. Rate: 7.9 symbols/min [2025-10-20T17:47:55.660075] [INFO ] [MAIN ] CADE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:55.706399] [DEBUG] [LOCAL ] CADE: Loaded 50257 existing records from primary storage. [2025-10-20T17:47:55.706767] [INFO ] [MAIN ] CADE: Loaded 50257 existing records. [2025-10-20T17:47:55.883069] [WARN ] [MAIN ] CADE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:55.883285] [INFO ] [MAIN ] CADE: Wiping existing primary data before full refresh. [2025-10-20T17:47:55.883332] [INFO ] [LOCAL ] CADE: Starting deletion of primary data. [2025-10-20T17:47:55.883554] [DEBUG] [LOCAL ] CADE: Deleted primary data file. [2025-10-20T17:47:55.883609] [INFO ] [MAIN ] CADE: Starting full primary refresh. [2025-10-20T17:47:56.238770] [INFO ] [REMOTE ] CADE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:56.251344] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-01. [2025-10-20T17:47:56.591272] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T17:47:56.594990] [INFO ] [REMOTE ] CAAP: Full fetch complete (27978 records). [2025-10-20T17:47:56.595068] [DEBUG] [VALIDATOR ] CAAP: Validating primary data. [2025-10-20T17:47:56.960632] [DEBUG] [VALIDATOR ] CAAP: All expected trading days are present. [2025-10-20T17:47:56.960954] [DEBUG] [VALIDATOR ] CAAP: Primary data validation passed. [2025-10-20T17:47:56.961380] [INFO ] [MAIN ] CAAP: Fetched primary data is valid. [2025-10-20T17:47:56.961421] [DEBUG] [LOCAL ] CAAP: Saving primary data. [2025-10-20T17:47:56.962084] [DEBUG] [LOCAL ] CAAP: Trimmed 27978 records to 19913 (keeping 200 days of data) [2025-10-20T17:47:57.053549] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-03. [2025-10-20T17:47:57.076048] [DEBUG] [LOCAL ] CAAP: Primary data saved (19913 records). [2025-10-20T17:47:57.076151] [INFO ] [MAIN ] CAAP: Full history saved. [2025-10-20T17:47:57.076226] [INFO ] [MAIN ] Update Progress: 381/2367 symbols completed. CAAP took 32.5s. Estimated remaining: 4h 9m 34s. Rate: 8.0 symbols/min [2025-10-20T17:47:57.246808] [INFO ] [MAIN ] CAE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:57.280828] [DEBUG] [LOCAL ] CAE: Loaded 39813 existing records from primary storage. [2025-10-20T17:47:57.281137] [INFO ] [MAIN ] CAE: Loaded 39813 existing records. [2025-10-20T17:47:57.460170] [WARN ] [MAIN ] CAE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:57.460408] [INFO ] [MAIN ] CAE: Wiping existing primary data before full refresh. [2025-10-20T17:47:57.460456] [INFO ] [LOCAL ] CAE: Starting deletion of primary data. [2025-10-20T17:47:57.460943] [DEBUG] [LOCAL ] CAE: Deleted primary data file. [2025-10-20T17:47:57.461002] [INFO ] [MAIN ] CAE: Starting full primary refresh. [2025-10-20T17:47:57.852062] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-10. [2025-10-20T17:47:57.852002] [INFO ] [REMOTE ] CAE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:57.878442] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:47:58.651354] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-01. [2025-10-20T17:47:58.728246] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:47:58.838993] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T17:47:58.841418] [INFO ] [REMOTE ] CABO: Full fetch complete (23148 records). [2025-10-20T17:47:58.841552] [DEBUG] [VALIDATOR ] CABO: Validating primary data. [2025-10-20T17:47:59.216630] [DEBUG] [VALIDATOR ] CABO: All expected trading days are present. [2025-10-20T17:47:59.216735] [DEBUG] [VALIDATOR ] CABO: Primary data validation passed. [2025-10-20T17:47:59.217147] [INFO ] [MAIN ] CABO: Fetched primary data is valid. [2025-10-20T17:47:59.217187] [DEBUG] [LOCAL ] CABO: Saving primary data. [2025-10-20T17:47:59.217818] [DEBUG] [LOCAL ] CABO: Trimmed 23148 records to 16723 (keeping 200 days of data) [2025-10-20T17:47:59.315241] [DEBUG] [LOCAL ] CABO: Primary data saved (16723 records). [2025-10-20T17:47:59.315376] [INFO ] [MAIN ] CABO: Full history saved. [2025-10-20T17:47:59.315458] [INFO ] [MAIN ] Update Progress: 382/2367 symbols completed. CABO took 22.4s. Estimated remaining: 4h 8m 59s. Rate: 8.0 symbols/min [2025-10-20T17:47:59.460279] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-02. [2025-10-20T17:47:59.493747] [INFO ] [MAIN ] CAF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:59.499927] [DEBUG] [LOCAL ] CAF: Loaded 3545 existing records from primary storage. [2025-10-20T17:47:59.500136] [INFO ] [MAIN ] CAF: Loaded 3545 existing records. [2025-10-20T17:47:59.686512] [WARN ] [MAIN ] CAF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:59.686797] [INFO ] [MAIN ] CAF: Wiping existing primary data before full refresh. [2025-10-20T17:47:59.686869] [INFO ] [LOCAL ] CAF: Starting deletion of primary data. [2025-10-20T17:47:59.687094] [DEBUG] [LOCAL ] CAF: Deleted primary data file. [2025-10-20T17:47:59.687162] [INFO ] [MAIN ] CAF: Starting full primary refresh. [2025-10-20T17:48:00.084752] [INFO ] [REMOTE ] CAF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:00.187062] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:48:00.251446] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-02. [2025-10-20T17:48:00.777971] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T17:48:01.051381] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-04. [2025-10-20T17:48:01.457710] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:48:01.851381] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-01. [2025-10-20T17:48:02.555366] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:48:02.658084] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-03. [2025-10-20T17:48:02.685601] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T17:48:03.451431] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-03. [2025-10-20T17:48:03.895024] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T17:48:04.251474] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-04. [2025-10-20T17:48:05.051483] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-02. [2025-10-20T17:48:05.203343] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:48:05.320895] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T17:48:05.757050] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T17:48:05.851475] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-03. [2025-10-20T17:48:06.239410] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-10-20T17:48:06.651408] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-05. [2025-10-20T17:48:07.451444] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-04. [2025-10-20T17:48:07.998317] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T17:48:08.251435] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-04. [2025-10-20T17:48:08.883110] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T17:48:09.051435] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-05. [2025-10-20T17:48:09.198863] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T17:48:09.531166] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T17:48:09.851452] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-06. [2025-10-20T17:48:10.332835] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T17:48:10.651444] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-05. [2025-10-20T17:48:11.451454] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-06. [2025-10-20T17:48:12.251448] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-07. [2025-10-20T17:48:13.051422] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-05. [2025-10-20T17:48:13.921788] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-06 (2.47 seconds). [2025-10-20T17:48:13.926793] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-07. [2025-10-20T17:48:14.031178] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T17:48:14.132825] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-05 (3.48 seconds). [2025-10-20T17:48:14.651516] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-06. [2025-10-20T17:48:15.086919] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:48:15.456485] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-08. [2025-10-20T17:48:15.471818] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-05 (2.42 seconds). [2025-10-20T17:48:16.251419] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-08. [2025-10-20T17:48:16.407500] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T17:48:16.513314] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-10-20T17:48:16.741036] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T17:48:17.051389] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-07. [2025-10-20T17:48:17.851467] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-06. [2025-10-20T17:48:18.651347] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-09. [2025-10-20T17:48:19.006121] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T17:48:19.144865] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:48:19.451364] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-08. [2025-10-20T17:48:19.623749] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T17:48:20.251408] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-10. [2025-10-20T17:48:21.051358] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-07. [2025-10-20T17:48:21.117841] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T17:48:21.364550] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T17:48:21.367241] [INFO ] [REMOTE ] CACI: Full fetch complete (38294 records). [2025-10-20T17:48:21.367313] [DEBUG] [VALIDATOR ] CACI: Validating primary data. [2025-10-20T17:48:21.754375] [DEBUG] [VALIDATOR ] CACI: All expected trading days are present. [2025-10-20T17:48:21.754453] [DEBUG] [VALIDATOR ] CACI: Primary data validation passed. [2025-10-20T17:48:21.754965] [INFO ] [MAIN ] CACI: Fetched primary data is valid. [2025-10-20T17:48:21.755026] [DEBUG] [LOCAL ] CACI: Saving primary data. [2025-10-20T17:48:21.755918] [DEBUG] [LOCAL ] CACI: Trimmed 38294 records to 22704 (keeping 200 days of data) [2025-10-20T17:48:21.854175] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-09. [2025-10-20T17:48:21.898853] [DEBUG] [LOCAL ] CACI: Primary data saved (22704 records). [2025-10-20T17:48:21.898977] [INFO ] [MAIN ] CACI: Full history saved. [2025-10-20T17:48:21.899055] [INFO ] [MAIN ] Update Progress: 383/2367 symbols completed. CACI took 42.1s. Estimated remaining: 4h 10m 9s. Rate: 7.9 symbols/min [2025-10-20T17:48:22.081216] [INFO ] [MAIN ] CAG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:22.081469] [DEBUG] [LOCAL ] CAG: No existing data file found in primary storage. [2025-10-20T17:48:22.081523] [WARN ] [MAIN ] CAG: No existing data found, performing full refresh instead. [2025-10-20T17:48:22.081699] [INFO ] [MAIN ] CAG: Wiping existing primary data before full refresh. [2025-10-20T17:48:22.081770] [INFO ] [LOCAL ] CAG: Starting deletion of primary data. [2025-10-20T17:48:22.081842] [DEBUG] [LOCAL ] CAG: No primary data file found to delete. [2025-10-20T17:48:22.081877] [INFO ] [MAIN ] CAG: Starting full primary refresh. [2025-10-20T17:48:22.435701] [INFO ] [REMOTE ] CAG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:22.448633] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T17:48:22.540156] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T17:48:22.651353] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-01. [2025-10-20T17:48:23.451391] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-10. [2025-10-20T17:48:24.034378] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T17:48:24.035492] [INFO ] [REMOTE ] CAF: Full fetch complete (5509 records). [2025-10-20T17:48:24.035559] [DEBUG] [VALIDATOR ] CAF: Validating primary data. [2025-10-20T17:48:24.035928] [FATAL] [VALIDATOR ] CAF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:48:24.036008] [FATAL] [MAIN ] CAF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:48:24.036049] [WARN ] [MAIN ] CAF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:48:24.036085] [INFO ] [LOCAL ] CAF: Starting deletion of primary data. [2025-10-20T17:48:24.036137] [DEBUG] [LOCAL ] CAF: No primary data file found to delete. [2025-10-20T17:48:24.036170] [INFO ] [LOCAL ] CAF: Starting deletion of derivative data. [2025-10-20T17:48:24.036347] [DEBUG] [LOCAL ] CAF: Deleted derivative data file. [2025-10-20T17:48:24.036430] [INFO ] [MAIN ] Update Progress: 384/2367 symbols completed. CAF took 24.7s. Estimated remaining: 4h 9m 34s. Rate: 7.9 symbols/min [2025-10-20T17:48:24.211888] [INFO ] [MAIN ] CAH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:24.244852] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T17:48:24.255176] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-08. [2025-10-20T17:48:24.299853] [DEBUG] [LOCAL ] CAH: Loaded 52214 existing records from primary storage. [2025-10-20T17:48:24.300166] [INFO ] [MAIN ] CAH: Loaded 52214 existing records. [2025-10-20T17:48:24.478278] [WARN ] [MAIN ] CAH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:24.478514] [INFO ] [MAIN ] CAH: Wiping existing primary data before full refresh. [2025-10-20T17:48:24.478561] [INFO ] [LOCAL ] CAH: Starting deletion of primary data. [2025-10-20T17:48:24.478792] [DEBUG] [LOCAL ] CAH: Deleted primary data file. [2025-10-20T17:48:24.478847] [INFO ] [MAIN ] CAH: Starting full primary refresh. [2025-10-20T17:48:24.846458] [INFO ] [REMOTE ] CAH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:25.051346] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-02. [2025-10-20T17:48:25.778143] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:48:25.851336] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-09. [2025-10-20T17:48:26.651435] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-09. [2025-10-20T17:48:27.451353] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-01. [2025-10-20T17:48:27.563614] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-02 (2.51 seconds). [2025-10-20T17:48:27.908448] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T17:48:28.251389] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-03. [2025-10-20T17:48:29.051383] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-10. [2025-10-20T17:48:29.153173] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-09 (2.50 seconds). [2025-10-20T17:48:29.828668] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-01 (2.38 seconds). [2025-10-20T17:48:29.851354] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-10. [2025-10-20T17:48:29.964996] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:48:30.609045] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T17:48:30.615293] [INFO ] [REMOTE ] CAE: Full fetch complete (56551 records). [2025-10-20T17:48:30.615410] [DEBUG] [VALIDATOR ] CAE: Validating primary data. [2025-10-20T17:48:30.659612] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-02. [2025-10-20T17:48:31.015243] [DEBUG] [VALIDATOR ] CAE: All expected trading days are present. [2025-10-20T17:48:31.015350] [DEBUG] [VALIDATOR ] CAE: Primary data validation passed. [2025-10-20T17:48:31.016056] [INFO ] [MAIN ] CAE: Fetched primary data is valid. [2025-10-20T17:48:31.016120] [DEBUG] [LOCAL ] CAE: Saving primary data. [2025-10-20T17:48:31.017301] [DEBUG] [LOCAL ] CAE: Trimmed 56551 records to 39223 (keeping 200 days of data) [2025-10-20T17:48:31.242281] [DEBUG] [LOCAL ] CAE: Primary data saved (39223 records). [2025-10-20T17:48:31.242413] [INFO ] [MAIN ] CAE: Full history saved. [2025-10-20T17:48:31.242490] [INFO ] [MAIN ] Update Progress: 385/2367 symbols completed. CAE took 34.2s. Estimated remaining: 4h 9m 25s. Rate: 7.9 symbols/min [2025-10-20T17:48:31.416090] [INFO ] [MAIN ] CAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:31.437289] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T17:48:31.445387] [INFO ] [REMOTE ] CADE: Full fetch complete (70658 records). [2025-10-20T17:48:31.445594] [DEBUG] [VALIDATOR ] CADE: Validating primary data. [2025-10-20T17:48:31.457674] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-04. [2025-10-20T17:48:31.676691] [DEBUG] [LOCAL ] CAL: Loaded 43547 existing records from primary storage. [2025-10-20T17:48:31.737636] [INFO ] [MAIN ] CAL: Loaded 43547 existing records. [2025-10-20T17:48:32.071537] [DEBUG] [VALIDATOR ] CADE: All expected trading days are present. [2025-10-20T17:48:32.071313] [WARN ] [MAIN ] CAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:32.071728] [DEBUG] [VALIDATOR ] CADE: Primary data validation passed. [2025-10-20T17:48:32.071884] [INFO ] [MAIN ] CAL: Wiping existing primary data before full refresh. [2025-10-20T17:48:32.071974] [INFO ] [LOCAL ] CAL: Starting deletion of primary data. [2025-10-20T17:48:32.072862] [INFO ] [MAIN ] CADE: Fetched primary data is valid. [2025-10-20T17:48:32.072998] [DEBUG] [LOCAL ] CADE: Saving primary data. [2025-10-20T17:48:32.073203] [DEBUG] [LOCAL ] CAL: Deleted primary data file. [2025-10-20T17:48:32.073276] [INFO ] [MAIN ] CAL: Starting full primary refresh. [2025-10-20T17:48:32.156723] [DEBUG] [LOCAL ] CADE: Trimmed 70658 records to 49658 (keeping 200 days of data) [2025-10-20T17:48:32.635235] [INFO ] [REMOTE ] CAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:32.663159] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-01. [2025-10-20T17:48:32.708048] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-10-20T17:48:32.792773] [DEBUG] [LOCAL ] CADE: Primary data saved (49658 records). [2025-10-20T17:48:32.792920] [INFO ] [MAIN ] CADE: Full history saved. [2025-10-20T17:48:32.793000] [INFO ] [MAIN ] Update Progress: 386/2367 symbols completed. CADE took 37.3s. Estimated remaining: 4h 8m 46s. Rate: 8.0 symbols/min [2025-10-20T17:48:32.994964] [INFO ] [MAIN ] CALX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:33.028143] [DEBUG] [LOCAL ] CALX: Loaded 38084 existing records from primary storage. [2025-10-20T17:48:33.028459] [INFO ] [MAIN ] CALX: Loaded 38084 existing records. [2025-10-20T17:48:33.053404] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-03. [2025-10-20T17:48:33.223629] [WARN ] [MAIN ] CALX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:33.223984] [INFO ] [MAIN ] CALX: Wiping existing primary data before full refresh. [2025-10-20T17:48:33.224036] [INFO ] [LOCAL ] CALX: Starting deletion of primary data. [2025-10-20T17:48:33.224243] [DEBUG] [LOCAL ] CALX: Deleted primary data file. [2025-10-20T17:48:33.224287] [INFO ] [MAIN ] CALX: Starting full primary refresh. [2025-10-20T17:48:33.602602] [INFO ] [REMOTE ] CALX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:33.632281] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T17:48:33.851344] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-05. [2025-10-20T17:48:34.297963] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:48:34.651400] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-01. [2025-10-20T17:48:35.000817] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T17:48:35.451367] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-04. [2025-10-20T17:48:35.623237] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:48:35.699773] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T17:48:36.251427] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-06. [2025-10-20T17:48:37.051399] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-02. [2025-10-20T17:48:37.358838] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T17:48:37.851516] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-05. [2025-10-20T17:48:38.058779] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T17:48:38.155093] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:48:38.651440] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-07. [2025-10-20T17:48:39.369813] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T17:48:39.451351] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-06. [2025-10-20T17:48:40.251378] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-03. [2025-10-20T17:48:40.644696] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T17:48:41.051435] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-08. [2025-10-20T17:48:41.247604] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T17:48:41.818016] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T17:48:41.851363] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-04. [2025-10-20T17:48:42.362417] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:48:42.651384] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-07. [2025-10-20T17:48:43.451374] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-09. [2025-10-20T17:48:43.549412] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:48:44.251403] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-02. [2025-10-20T17:48:44.568886] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T17:48:44.985806] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:48:45.051376] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-10. [2025-10-20T17:48:45.771015] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:48:45.851387] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-03. [2025-10-20T17:48:46.651381] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-05. [2025-10-20T17:48:47.152827] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:48:47.161586] [INFO ] [REMOTE ] CAG: Full fetch complete (77826 records). [2025-10-20T17:48:47.161861] [DEBUG] [VALIDATOR ] CAG: Validating primary data. [2025-10-20T17:48:47.456837] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-08. [2025-10-20T17:48:47.614370] [DEBUG] [VALIDATOR ] CAG: All expected trading days are present. [2025-10-20T17:48:47.623985] [DEBUG] [VALIDATOR ] CAG: Primary data validation passed. [2025-10-20T17:48:47.642408] [INFO ] [MAIN ] CAG: Fetched primary data is valid. [2025-10-20T17:48:47.645090] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T17:48:47.645259] [DEBUG] [LOCAL ] CAG: Saving primary data. [2025-10-20T17:48:47.658439] [DEBUG] [LOCAL ] CAG: Trimmed 77826 records to 53665 (keeping 200 days of data) [2025-10-20T17:48:48.015778] [DEBUG] [LOCAL ] CAG: Primary data saved (53665 records). [2025-10-20T17:48:48.016007] [INFO ] [MAIN ] CAG: Full history saved. [2025-10-20T17:48:48.016117] [INFO ] [MAIN ] Update Progress: 387/2367 symbols completed. CAG took 26.1s. Estimated remaining: 4h 9m 18s. Rate: 7.9 symbols/min [2025-10-20T17:48:48.202482] [INFO ] [MAIN ] CANG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:48.230511] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T17:48:48.251337] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-06. [2025-10-20T17:48:48.264565] [DEBUG] [LOCAL ] CANG: Loaded 27428 existing records from primary storage. [2025-10-20T17:48:48.264964] [INFO ] [MAIN ] CANG: Loaded 27428 existing records. [2025-10-20T17:48:48.455721] [WARN ] [MAIN ] CANG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:48.456057] [INFO ] [MAIN ] CANG: Wiping existing primary data before full refresh. [2025-10-20T17:48:48.456108] [INFO ] [LOCAL ] CANG: Starting deletion of primary data. [2025-10-20T17:48:48.456391] [DEBUG] [LOCAL ] CANG: Deleted primary data file. [2025-10-20T17:48:48.456451] [INFO ] [MAIN ] CANG: Starting full primary refresh. [2025-10-20T17:48:48.845732] [INFO ] [REMOTE ] CANG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:49.036777] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:48:49.051337] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-04. [2025-10-20T17:48:49.590502] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T17:48:49.851379] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-07. [2025-10-20T17:48:50.651353] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-09. [2025-10-20T17:48:50.750692] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:48:51.296739] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T17:48:51.451434] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-08. [2025-10-20T17:48:52.251364] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-01. [2025-10-20T17:48:52.357779] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T17:48:52.881518] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T17:48:53.051424] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-09. [2025-10-20T17:48:53.397341] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T17:48:53.851384] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-10. [2025-10-20T17:48:54.034346] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T17:48:54.651419] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-05. [2025-10-20T17:48:55.460174] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-10. [2025-10-20T17:48:55.485833] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:48:55.493986] [INFO ] [REMOTE ] CAH: Full fetch complete (74843 records). [2025-10-20T17:48:55.494119] [DEBUG] [VALIDATOR ] CAH: Validating primary data. [2025-10-20T17:48:55.933553] [DEBUG] [VALIDATOR ] CAH: All expected trading days are present. [2025-10-20T17:48:55.933702] [DEBUG] [VALIDATOR ] CAH: Primary data validation passed. [2025-10-20T17:48:55.934906] [INFO ] [MAIN ] CAH: Fetched primary data is valid. [2025-10-20T17:48:55.934977] [DEBUG] [LOCAL ] CAH: Saving primary data. [2025-10-20T17:48:55.937120] [DEBUG] [LOCAL ] CAH: Trimmed 74843 records to 51463 (keeping 200 days of data) [2025-10-20T17:48:56.256472] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-02. [2025-10-20T17:48:56.303372] [DEBUG] [LOCAL ] CAH: Primary data saved (51463 records). [2025-10-20T17:48:56.303625] [INFO ] [MAIN ] CAH: Full history saved. [2025-10-20T17:48:56.304125] [INFO ] [MAIN ] Update Progress: 388/2367 symbols completed. CAH took 32.3s. Estimated remaining: 4h 9m 14s. Rate: 7.9 symbols/min [2025-10-20T17:48:56.510873] [INFO ] [MAIN ] CAPL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:56.530158] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T17:48:56.546276] [DEBUG] [LOCAL ] CAPL: Loaded 5696 existing records from primary storage. [2025-10-20T17:48:56.546581] [INFO ] [MAIN ] CAPL: Loaded 5696 existing records. [2025-10-20T17:48:56.740402] [WARN ] [MAIN ] CAPL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:56.740720] [INFO ] [MAIN ] CAPL: Wiping existing primary data before full refresh. [2025-10-20T17:48:56.740791] [INFO ] [LOCAL ] CAPL: Starting deletion of primary data. [2025-10-20T17:48:56.741023] [DEBUG] [LOCAL ] CAPL: Deleted primary data file. [2025-10-20T17:48:56.741087] [INFO ] [MAIN ] CAPL: Starting full primary refresh. [2025-10-20T17:48:57.053363] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-06. [2025-10-20T17:48:57.100016] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T17:48:57.167097] [INFO ] [REMOTE ] CALX: Full fetch complete (54173 records). [2025-10-20T17:48:57.169896] [DEBUG] [VALIDATOR ] CALX: Validating primary data. [2025-10-20T17:48:57.194151] [INFO ] [REMOTE ] CAPL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:57.255239] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T17:48:57.653303] [DEBUG] [VALIDATOR ] CALX: All expected trading days are present. [2025-10-20T17:48:57.653409] [DEBUG] [VALIDATOR ] CALX: Primary data validation passed. [2025-10-20T17:48:57.654220] [INFO ] [MAIN ] CALX: Fetched primary data is valid. [2025-10-20T17:48:57.654310] [DEBUG] [LOCAL ] CALX: Saving primary data. [2025-10-20T17:48:57.655715] [DEBUG] [LOCAL ] CALX: Trimmed 54173 records to 37496 (keeping 200 days of data) [2025-10-20T17:48:57.880119] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-01. [2025-10-20T17:48:57.914350] [DEBUG] [LOCAL ] CALX: Primary data saved (37496 records). [2025-10-20T17:48:57.914611] [INFO ] [MAIN ] CALX: Full history saved. [2025-10-20T17:48:57.914758] [INFO ] [MAIN ] Update Progress: 389/2367 symbols completed. CALX took 25.1s. Estimated remaining: 4h 8m 36s. Rate: 8.0 symbols/min [2025-10-20T17:48:58.136773] [INFO ] [MAIN ] CARR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:58.198274] [DEBUG] [LOCAL ] CARR: Loaded 54382 existing records from primary storage. [2025-10-20T17:48:58.198739] [INFO ] [MAIN ] CARR: Loaded 54382 existing records. [2025-10-20T17:48:58.496381] [WARN ] [MAIN ] CARR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:58.498016] [INFO ] [MAIN ] CARR: Wiping existing primary data before full refresh. [2025-10-20T17:48:58.498407] [INFO ] [LOCAL ] CARR: Starting deletion of primary data. [2025-10-20T17:48:58.499009] [DEBUG] [LOCAL ] CARR: Deleted primary data file. [2025-10-20T17:48:58.499086] [INFO ] [MAIN ] CARR: Starting full primary refresh. [2025-10-20T17:48:58.660983] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T17:48:58.659385] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-03. [2025-10-20T17:48:58.785603] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:48:58.990249] [INFO ] [REMOTE ] CARR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:59.451433] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-02. [2025-10-20T17:48:59.789517] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T17:48:59.815133] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T17:49:00.251425] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-01. [2025-10-20T17:49:01.051431] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-03. [2025-10-20T17:49:01.851416] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-07. [2025-10-20T17:49:01.934885] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T17:49:02.272678] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T17:49:02.651429] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-04. [2025-10-20T17:49:03.297857] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:49:03.326277] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T17:49:03.451389] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-02. [2025-10-20T17:49:04.251384] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-05. [2025-10-20T17:49:04.748585] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-10-20T17:49:04.915224] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:49:05.051418] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-03. [2025-10-20T17:49:05.851418] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-08. [2025-10-20T17:49:06.569479] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:49:06.651420] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-04. [2025-10-20T17:49:07.069336] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-10-20T17:49:07.451462] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-04. [2025-10-20T17:49:08.251463] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-06. [2025-10-20T17:49:08.508337] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T17:49:08.666307] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:49:08.715704] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T17:49:09.051397] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-05. [2025-10-20T17:49:09.851451] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-05. [2025-10-20T17:49:10.651384] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-07. [2025-10-20T17:49:11.451448] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-09. [2025-10-20T17:49:11.658430] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:49:12.176711] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T17:49:12.251465] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-08. [2025-10-20T17:49:12.503888] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-05 (3.45 seconds). [2025-10-20T17:49:12.937258] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:49:13.051410] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-09. [2025-10-20T17:49:13.299297] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:49:13.553449] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T17:49:13.851427] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-10. [2025-10-20T17:49:14.651393] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-06. [2025-10-20T17:49:15.063980] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T17:49:15.070861] [INFO ] [REMOTE ] CAL: Full fetch complete (61837 records). [2025-10-20T17:49:15.071241] [DEBUG] [VALIDATOR ] CAL: Validating primary data. [2025-10-20T17:49:15.456731] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-10. [2025-10-20T17:49:15.504895] [DEBUG] [VALIDATOR ] CAL: All expected trading days are present. [2025-10-20T17:49:15.505004] [DEBUG] [VALIDATOR ] CAL: Primary data validation passed. [2025-10-20T17:49:15.505945] [INFO ] [MAIN ] CAL: Fetched primary data is valid. [2025-10-20T17:49:15.506013] [DEBUG] [LOCAL ] CAL: Saving primary data. [2025-10-20T17:49:15.507741] [DEBUG] [LOCAL ] CAL: Trimmed 61837 records to 42891 (keeping 200 days of data) [2025-10-20T17:49:15.788319] [DEBUG] [LOCAL ] CAL: Primary data saved (42891 records). [2025-10-20T17:49:15.788503] [INFO ] [MAIN ] CAL: Full history saved. [2025-10-20T17:49:15.788602] [INFO ] [MAIN ] Update Progress: 390/2367 symbols completed. CAL took 44.5s. Estimated remaining: 4h 9m 21s. Rate: 7.9 symbols/min [2025-10-20T17:49:15.984050] [INFO ] [MAIN ] CARS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:16.031994] [DEBUG] [LOCAL ] CARS: Loaded 40383 existing records from primary storage. [2025-10-20T17:49:16.032387] [INFO ] [MAIN ] CARS: Loaded 40383 existing records. [2025-10-20T17:49:16.233713] [WARN ] [MAIN ] CARS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:16.234034] [INFO ] [MAIN ] CARS: Wiping existing primary data before full refresh. [2025-10-20T17:49:16.234223] [INFO ] [LOCAL ] CARS: Starting deletion of primary data. [2025-10-20T17:49:16.234567] [DEBUG] [LOCAL ] CARS: Deleted primary data file. [2025-10-20T17:49:16.234626] [INFO ] [MAIN ] CARS: Starting full primary refresh. [2025-10-20T17:49:16.266321] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-06. [2025-10-20T17:49:16.559571] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T17:49:16.621091] [INFO ] [REMOTE ] CARS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:16.623046] [INFO ] [REMOTE ] CAPL: Full fetch complete (8692 records). [2025-10-20T17:49:16.623539] [DEBUG] [VALIDATOR ] CAPL: Validating primary data. [2025-10-20T17:49:16.624039] [FATAL] [VALIDATOR ] CAPL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:49:16.624186] [FATAL] [MAIN ] CAPL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:49:16.624522] [WARN ] [MAIN ] CAPL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:49:16.624628] [INFO ] [LOCAL ] CAPL: Starting deletion of primary data. [2025-10-20T17:49:16.624957] [DEBUG] [LOCAL ] CAPL: No primary data file found to delete. [2025-10-20T17:49:16.625031] [INFO ] [LOCAL ] CAPL: Starting deletion of derivative data. [2025-10-20T17:49:16.625280] [DEBUG] [LOCAL ] CAPL: Deleted derivative data file. [2025-10-20T17:49:16.625370] [INFO ] [MAIN ] Update Progress: 391/2367 symbols completed. CAPL took 20.3s. Estimated remaining: 4h 8m 40s. Rate: 7.9 symbols/min [2025-10-20T17:49:16.843705] [INFO ] [MAIN ] CAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:16.846607] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-06 (2.20 seconds). [2025-10-20T17:49:16.906124] [DEBUG] [LOCAL ] CAT: Loaded 53736 existing records from primary storage. [2025-10-20T17:49:16.906448] [INFO ] [MAIN ] CAT: Loaded 53736 existing records. [2025-10-20T17:49:17.056754] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-01. [2025-10-20T17:49:17.085826] [WARN ] [MAIN ] CAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:17.086156] [INFO ] [MAIN ] CAT: Wiping existing primary data before full refresh. [2025-10-20T17:49:17.086207] [INFO ] [LOCAL ] CAT: Starting deletion of primary data. [2025-10-20T17:49:17.086743] [DEBUG] [LOCAL ] CAT: Deleted primary data file. [2025-10-20T17:49:17.086798] [INFO ] [MAIN ] CAT: Starting full primary refresh. [2025-10-20T17:49:17.448225] [INFO ] [REMOTE ] CAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:17.467516] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T17:49:17.851445] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-01. [2025-10-20T17:49:18.479968] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:49:18.651441] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-07. [2025-10-20T17:49:19.451359] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-07. [2025-10-20T17:49:19.835964] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T17:49:20.251471] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-02. [2025-10-20T17:49:20.637931] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T17:49:20.744409] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T17:49:21.051382] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-08. [2025-10-20T17:49:21.580102] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:49:21.851445] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-02. [2025-10-20T17:49:22.651366] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-03. [2025-10-20T17:49:22.970807] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T17:49:23.451397] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-09. [2025-10-20T17:49:23.864750] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T17:49:24.197260] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:49:24.251433] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-04. [2025-10-20T17:49:25.028903] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T17:49:25.051354] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-03. [2025-10-20T17:49:25.764116] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T17:49:25.851389] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-08. [2025-10-20T17:49:26.651420] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-10. [2025-10-20T17:49:26.853386] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T17:49:26.930556] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T17:49:27.451375] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-04. [2025-10-20T17:49:28.251436] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-09. [2025-10-20T17:49:28.588992] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T17:49:28.597261] [INFO ] [REMOTE ] CARR: Full fetch complete (77772 records). [2025-10-20T17:49:28.597425] [DEBUG] [VALIDATOR ] CARR: Validating primary data. [2025-10-20T17:49:29.009212] [DEBUG] [VALIDATOR ] CARR: All expected trading days are present. [2025-10-20T17:49:29.009327] [DEBUG] [VALIDATOR ] CARR: Primary data validation passed. [2025-10-20T17:49:29.010543] [INFO ] [MAIN ] CARR: Fetched primary data is valid. [2025-10-20T17:49:29.010615] [DEBUG] [LOCAL ] CARR: Saving primary data. [2025-10-20T17:49:29.012793] [DEBUG] [LOCAL ] CARR: Trimmed 77772 records to 53603 (keeping 200 days of data) [2025-10-20T17:49:29.069141] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-05. [2025-10-20T17:49:29.374458] [DEBUG] [LOCAL ] CARR: Primary data saved (53603 records). [2025-10-20T17:49:29.374740] [INFO ] [MAIN ] CARR: Full history saved. [2025-10-20T17:49:29.374907] [INFO ] [MAIN ] Update Progress: 392/2367 symbols completed. CARR took 31.5s. Estimated remaining: 4h 8m 58s. Rate: 7.9 symbols/min [2025-10-20T17:49:29.578682] [INFO ] [MAIN ] CATO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:29.602490] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:49:29.617985] [DEBUG] [LOCAL ] CATO: Loaded 6799 existing records from primary storage. [2025-10-20T17:49:29.619097] [INFO ] [MAIN ] CATO: Loaded 6799 existing records. [2025-10-20T17:49:29.814051] [WARN ] [MAIN ] CATO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:29.814453] [INFO ] [MAIN ] CATO: Wiping existing primary data before full refresh. [2025-10-20T17:49:29.814534] [INFO ] [LOCAL ] CATO: Starting deletion of primary data. [2025-10-20T17:49:29.815011] [DEBUG] [LOCAL ] CATO: Deleted primary data file. [2025-10-20T17:49:29.842083] [INFO ] [MAIN ] CATO: Starting full primary refresh. [2025-10-20T17:49:29.858225] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-10. [2025-10-20T17:49:29.884249] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-04 (2.43 seconds). [2025-10-20T17:49:30.225040] [INFO ] [REMOTE ] CATO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:30.651368] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-05. [2025-10-20T17:49:30.746938] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:49:30.942495] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T17:49:30.947664] [INFO ] [REMOTE ] CANG: Full fetch complete (39819 records). [2025-10-20T17:49:30.947774] [DEBUG] [VALIDATOR ] CANG: Validating primary data. [2025-10-20T17:49:31.346946] [DEBUG] [VALIDATOR ] CANG: All expected trading days are present. [2025-10-20T17:49:31.347064] [DEBUG] [VALIDATOR ] CANG: Primary data validation passed. [2025-10-20T17:49:31.347618] [INFO ] [MAIN ] CANG: Fetched primary data is valid. [2025-10-20T17:49:31.347692] [DEBUG] [LOCAL ] CANG: Saving primary data. [2025-10-20T17:49:31.348529] [DEBUG] [LOCAL ] CANG: Trimmed 39819 records to 27136 (keeping 200 days of data) [2025-10-20T17:49:31.453123] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-06. [2025-10-20T17:49:31.491583] [DEBUG] [LOCAL ] CANG: Primary data saved (27136 records). [2025-10-20T17:49:31.491702] [INFO ] [MAIN ] CANG: Full history saved. [2025-10-20T17:49:31.491786] [INFO ] [MAIN ] Update Progress: 393/2367 symbols completed. CANG took 43.5s. Estimated remaining: 4h 8m 23s. Rate: 7.9 symbols/min [2025-10-20T17:49:31.657097] [INFO ] [MAIN ] CAVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:31.657240] [DEBUG] [LOCAL ] CAVA: No existing data file found in primary storage. [2025-10-20T17:49:31.657276] [WARN ] [MAIN ] CAVA: No existing data found, performing full refresh instead. [2025-10-20T17:49:31.657383] [INFO ] [MAIN ] CAVA: Wiping existing primary data before full refresh. [2025-10-20T17:49:31.657423] [INFO ] [LOCAL ] CAVA: Starting deletion of primary data. [2025-10-20T17:49:31.657462] [DEBUG] [LOCAL ] CAVA: No primary data file found to delete. [2025-10-20T17:49:31.657492] [INFO ] [MAIN ] CAVA: Starting full primary refresh. [2025-10-20T17:49:32.003973] [INFO ] [REMOTE ] CAVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:32.251351] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-01. [2025-10-20T17:49:32.942122] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T17:49:32.972031] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-05 (2.32 seconds). [2025-10-20T17:49:33.051376] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-06. [2025-10-20T17:49:33.851390] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-01. [2025-10-20T17:49:34.299190] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T17:49:34.371589] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T17:49:34.651367] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-07. [2025-10-20T17:49:35.058198] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-10-20T17:49:35.451387] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-02. [2025-10-20T17:49:36.059537] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T17:49:36.251411] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-07. [2025-10-20T17:49:37.059549] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-08. [2025-10-20T17:49:37.094479] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T17:49:37.851418] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-03. [2025-10-20T17:49:38.439151] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T17:49:38.568792] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:49:38.651405] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-09. [2025-10-20T17:49:39.451378] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-02. [2025-10-20T17:49:39.904689] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T17:49:39.950808] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T17:49:40.159378] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T17:49:40.251375] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-10. [2025-10-20T17:49:41.051388] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-03. [2025-10-20T17:49:41.720035] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T17:49:41.851406] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-04. [2025-10-20T17:49:41.918380] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T17:49:41.928231] [INFO ] [REMOTE ] CARS: Full fetch complete (56359 records). [2025-10-20T17:49:41.928464] [DEBUG] [VALIDATOR ] CARS: Validating primary data. [2025-10-20T17:49:42.315050] [DEBUG] [VALIDATOR ] CARS: All expected trading days are present. [2025-10-20T17:49:42.315141] [DEBUG] [VALIDATOR ] CARS: Primary data validation passed. [2025-10-20T17:49:42.315970] [INFO ] [MAIN ] CARS: Fetched primary data is valid. [2025-10-20T17:49:42.316042] [DEBUG] [LOCAL ] CARS: Saving primary data. [2025-10-20T17:49:42.317324] [DEBUG] [LOCAL ] CARS: Trimmed 56359 records to 39802 (keeping 200 days of data) [2025-10-20T17:49:42.530478] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T17:49:42.559690] [DEBUG] [LOCAL ] CARS: Primary data saved (39802 records). [2025-10-20T17:49:42.559867] [INFO ] [MAIN ] CARS: Full history saved. [2025-10-20T17:49:42.559956] [INFO ] [MAIN ] Update Progress: 394/2367 symbols completed. CARS took 26.8s. Estimated remaining: 4h 8m 33s. Rate: 7.9 symbols/min [2025-10-20T17:49:42.656492] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-04. [2025-10-20T17:49:42.730476] [INFO ] [MAIN ] CB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:42.774717] [DEBUG] [LOCAL ] CB: Loaded 50716 existing records from primary storage. [2025-10-20T17:49:42.775055] [INFO ] [MAIN ] CB: Loaded 50716 existing records. [2025-10-20T17:49:42.946589] [WARN ] [MAIN ] CB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:42.946844] [INFO ] [MAIN ] CB: Wiping existing primary data before full refresh. [2025-10-20T17:49:42.946891] [INFO ] [LOCAL ] CB: Starting deletion of primary data. [2025-10-20T17:49:42.947189] [DEBUG] [LOCAL ] CB: Deleted primary data file. [2025-10-20T17:49:42.947248] [INFO ] [MAIN ] CB: Starting full primary refresh. [2025-10-20T17:49:43.293843] [INFO ] [REMOTE ] CB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:43.451355] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-01. [2025-10-20T17:49:44.251377] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-08. [2025-10-20T17:49:44.887217] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T17:49:44.994604] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:49:45.051334] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-02. [2025-10-20T17:49:45.851391] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-05. [2025-10-20T17:49:46.426388] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-10-20T17:49:46.513816] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:49:46.651341] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-03. [2025-10-20T17:49:47.451421] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-09. [2025-10-20T17:49:47.748839] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T17:49:48.251415] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-05. [2025-10-20T17:49:48.474927] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T17:49:48.877761] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T17:49:49.051355] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-06. [2025-10-20T17:49:49.193218] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:49:49.851408] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-06. [2025-10-20T17:49:50.455443] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T17:49:50.651450] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-07. [2025-10-20T17:49:50.912412] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T17:49:51.196012] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-10-20T17:49:51.451437] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-08. [2025-10-20T17:49:52.225219] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T17:49:52.251353] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-09. [2025-10-20T17:49:53.051369] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-10. [2025-10-20T17:49:53.124934] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T17:49:53.851435] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-10. [2025-10-20T17:49:54.338290] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T17:49:54.340046] [INFO ] [REMOTE ] CATO: Full fetch complete (10733 records). [2025-10-20T17:49:54.340127] [DEBUG] [VALIDATOR ] CATO: Validating primary data. [2025-10-20T17:49:54.656483] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-04. [2025-10-20T17:49:54.773490] [DEBUG] [VALIDATOR ] CATO: All expected trading days are present. [2025-10-20T17:49:54.773599] [DEBUG] [VALIDATOR ] CATO: Primary data validation passed. [2025-10-20T17:49:54.774119] [INFO ] [MAIN ] CATO: Fetched primary data is valid. [2025-10-20T17:49:54.774191] [DEBUG] [LOCAL ] CATO: Saving primary data. [2025-10-20T17:49:54.774874] [DEBUG] [LOCAL ] CATO: Trimmed 10733 records to 6648 (keeping 200 days of data) [2025-10-20T17:49:54.823947] [DEBUG] [LOCAL ] CATO: Primary data saved (6648 records). [2025-10-20T17:49:54.824096] [INFO ] [MAIN ] CATO: Full history saved. [2025-10-20T17:49:54.824186] [INFO ] [MAIN ] Update Progress: 395/2367 symbols completed. CATO took 25.4s. Estimated remaining: 4h 8m 49s. Rate: 7.9 symbols/min [2025-10-20T17:49:55.046993] [INFO ] [MAIN ] CBAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:55.060424] [DEBUG] [LOCAL ] CBAN: Loaded 10619 existing records from primary storage. [2025-10-20T17:49:55.060716] [INFO ] [MAIN ] CBAN: Loaded 10619 existing records. [2025-10-20T17:49:55.298489] [WARN ] [MAIN ] CBAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:55.298808] [INFO ] [MAIN ] CBAN: Wiping existing primary data before full refresh. [2025-10-20T17:49:55.298930] [INFO ] [LOCAL ] CBAN: Starting deletion of primary data. [2025-10-20T17:49:55.299321] [DEBUG] [LOCAL ] CBAN: Deleted primary data file. [2025-10-20T17:49:55.299437] [INFO ] [MAIN ] CBAN: Starting full primary refresh. [2025-10-20T17:49:55.476561] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-07. [2025-10-20T17:49:55.488003] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-10 (2.44 seconds). [2025-10-20T17:49:55.655794] [INFO ] [REMOTE ] CAT: Full fetch complete (76899 records). [2025-10-20T17:49:55.687919] [DEBUG] [VALIDATOR ] CAT: Validating primary data. [2025-10-20T17:49:55.922687] [INFO ] [REMOTE ] CBAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:56.251967] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-01. [2025-10-20T17:49:56.257400] [DEBUG] [VALIDATOR ] CAT: All expected trading days are present. [2025-10-20T17:49:56.257719] [DEBUG] [VALIDATOR ] CAT: Primary data validation passed. [2025-10-20T17:49:56.259162] [INFO ] [MAIN ] CAT: Fetched primary data is valid. [2025-10-20T17:49:56.259574] [DEBUG] [LOCAL ] CAT: Saving primary data. [2025-10-20T17:49:56.261811] [DEBUG] [LOCAL ] CAT: Trimmed 76899 records to 52966 (keeping 200 days of data) [2025-10-20T17:49:56.445791] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:49:56.677441] [DEBUG] [LOCAL ] CAT: Primary data saved (52966 records). [2025-10-20T17:49:56.677663] [INFO ] [MAIN ] CAT: Full history saved. [2025-10-20T17:49:56.678190] [INFO ] [MAIN ] Update Progress: 396/2367 symbols completed. CAT took 40.1s. Estimated remaining: 4h 8m 13s. Rate: 7.9 symbols/min [2025-10-20T17:49:56.866431] [INFO ] [MAIN ] CBL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:56.885985] [DEBUG] [LOCAL ] CBL: Loaded 16467 existing records from primary storage. [2025-10-20T17:49:56.886324] [INFO ] [MAIN ] CBL: Loaded 16467 existing records. [2025-10-20T17:49:57.054421] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-05. [2025-10-20T17:49:57.089535] [WARN ] [MAIN ] CBL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:57.090837] [INFO ] [MAIN ] CBL: Wiping existing primary data before full refresh. [2025-10-20T17:49:57.090892] [INFO ] [LOCAL ] CBL: Starting deletion of primary data. [2025-10-20T17:49:57.091148] [DEBUG] [LOCAL ] CBL: Deleted primary data file. [2025-10-20T17:49:57.091207] [INFO ] [MAIN ] CBL: Starting full primary refresh. [2025-10-20T17:49:57.214412] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T17:49:57.488554] [INFO ] [REMOTE ] CBL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:57.851381] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-02. [2025-10-20T17:49:57.903622] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-07 (2.43 seconds). [2025-10-20T17:49:58.651391] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-01. [2025-10-20T17:49:58.835979] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:49:59.003159] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T17:49:59.451449] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-06. [2025-10-20T17:49:59.481158] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T17:50:00.251467] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-03. [2025-10-20T17:50:01.009841] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T17:50:01.051480] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-08. [2025-10-20T17:50:01.465715] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T17:50:01.851401] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-07. [2025-10-20T17:50:02.651725] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-02. [2025-10-20T17:50:03.257383] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T17:50:03.451491] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-09. [2025-10-20T17:50:03.726768] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:50:03.935529] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-10-20T17:50:04.251434] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-03. [2025-10-20T17:50:05.051362] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-08. [2025-10-20T17:50:05.326112] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T17:50:05.327060] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T17:50:05.851390] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-04. [2025-10-20T17:50:06.580612] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:50:06.606029] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T17:50:06.651390] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-04. [2025-10-20T17:50:07.451461] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-05. [2025-10-20T17:50:07.865488] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:50:08.251433] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-09. [2025-10-20T17:50:08.355921] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T17:50:09.051392] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-05. [2025-10-20T17:50:09.851404] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-10. [2025-10-20T17:50:10.022826] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:50:10.052325] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T17:50:10.651467] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-06. [2025-10-20T17:50:11.359809] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T17:50:11.451370] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-07. [2025-10-20T17:50:12.251388] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-10. [2025-10-20T17:50:12.477260] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T17:50:13.051444] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-08. [2025-10-20T17:50:13.851497] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-06. [2025-10-20T17:50:14.001768] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T17:50:14.009129] [INFO ] [REMOTE ] CB: Full fetch complete (72841 records). [2025-10-20T17:50:14.009239] [DEBUG] [VALIDATOR ] CB: Validating primary data. [2025-10-20T17:50:14.159478] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:50:14.522807] [DEBUG] [VALIDATOR ] CB: All expected trading days are present. [2025-10-20T17:50:14.523161] [DEBUG] [VALIDATOR ] CB: Primary data validation passed. [2025-10-20T17:50:14.524803] [INFO ] [MAIN ] CB: Fetched primary data is valid. [2025-10-20T17:50:14.532490] [DEBUG] [LOCAL ] CB: Saving primary data. [2025-10-20T17:50:14.541179] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T17:50:14.558895] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-10 (4.71 seconds). [2025-10-20T17:50:14.567299] [DEBUG] [LOCAL ] CB: Trimmed 72841 records to 49967 (keeping 200 days of data) [2025-10-20T17:50:14.567612] [INFO ] [REMOTE ] CAVA: Full fetch complete (77453 records). [2025-10-20T17:50:14.568234] [DEBUG] [VALIDATOR ] CAVA: Validating primary data. [2025-10-20T17:50:14.659893] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-07. [2025-10-20T17:50:15.313850] [DEBUG] [VALIDATOR ] CAVA: All expected trading days are present. [2025-10-20T17:50:15.326972] [DEBUG] [VALIDATOR ] CAVA: Primary data validation passed. [2025-10-20T17:50:15.338691] [INFO ] [MAIN ] CAVA: Fetched primary data is valid. [2025-10-20T17:50:15.338994] [DEBUG] [LOCAL ] CAVA: Saving primary data. [2025-10-20T17:50:15.338859] [DEBUG] [LOCAL ] CB: Primary data saved (49967 records). [2025-10-20T17:50:15.339469] [INFO ] [MAIN ] CB: Full history saved. [2025-10-20T17:50:15.339620] [INFO ] [MAIN ] Update Progress: 397/2367 symbols completed. CB took 32.8s. Estimated remaining: 4h 9m 1s. Rate: 7.9 symbols/min [2025-10-20T17:50:15.341165] [DEBUG] [LOCAL ] CAVA: Trimmed 77453 records to 53552 (keeping 200 days of data) [2025-10-20T17:50:15.453685] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-09. [2025-10-20T17:50:15.662013] [DEBUG] [LOCAL ] CAVA: Primary data saved (53552 records). [2025-10-20T17:50:15.662185] [INFO ] [MAIN ] CAVA: Full history saved. [2025-10-20T17:50:15.662387] [INFO ] [MAIN ] Update Progress: 398/2367 symbols completed. CAVA took 44.2s. Estimated remaining: 4h 8m 17s. Rate: 7.9 symbols/min [2025-10-20T17:50:15.827397] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T17:50:16.251406] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-08. [2025-10-20T17:50:16.368365] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T17:50:17.051416] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-10. [2025-10-20T17:50:17.123549] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T17:50:17.851363] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-09. [2025-10-20T17:50:18.155341] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T17:50:18.158073] [INFO ] [REMOTE ] CBL: Full fetch complete (24903 records). [2025-10-20T17:50:18.158146] [DEBUG] [VALIDATOR ] CBL: Validating primary data. [2025-10-20T17:50:18.526482] [DEBUG] [VALIDATOR ] CBL: All expected trading days are present. [2025-10-20T17:50:18.526622] [DEBUG] [VALIDATOR ] CBL: Primary data validation passed. [2025-10-20T17:50:18.527067] [INFO ] [MAIN ] CBL: Fetched primary data is valid. [2025-10-20T17:50:18.527114] [DEBUG] [LOCAL ] CBL: Saving primary data. [2025-10-20T17:50:18.527842] [DEBUG] [LOCAL ] CBL: Trimmed 24903 records to 16230 (keeping 200 days of data) [2025-10-20T17:50:18.613755] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T17:50:18.642024] [DEBUG] [LOCAL ] CBL: Primary data saved (16230 records). [2025-10-20T17:50:18.642164] [INFO ] [MAIN ] CBL: Full history saved. [2025-10-20T17:50:18.642242] [INFO ] [MAIN ] Update Progress: 399/2367 symbols completed. CBL took 22.0s. Estimated remaining: 4h 7m 47s. Rate: 7.9 symbols/min [2025-10-20T17:50:18.651333] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-10. [2025-10-20T17:50:20.195826] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T17:50:20.197852] [INFO ] [REMOTE ] CBAN: Full fetch complete (16230 records). [2025-10-20T17:50:20.197938] [DEBUG] [VALIDATOR ] CBAN: Validating primary data. [2025-10-20T17:50:20.575264] [DEBUG] [VALIDATOR ] CBAN: All expected trading days are present. [2025-10-20T17:50:20.575350] [DEBUG] [VALIDATOR ] CBAN: Primary data validation passed. [2025-10-20T17:50:20.575761] [INFO ] [MAIN ] CBAN: Fetched primary data is valid. [2025-10-20T17:50:20.575826] [DEBUG] [LOCAL ] CBAN: Saving primary data. [2025-10-20T17:50:20.576372] [DEBUG] [LOCAL ] CBAN: Trimmed 16230 records to 10299 (keeping 200 days of data) [2025-10-20T17:50:20.637598] [DEBUG] [LOCAL ] CBAN: Primary data saved (10299 records). [2025-10-20T17:50:20.637769] [INFO ] [MAIN ] CBAN: Full history saved. [2025-10-20T17:50:20.637853] [INFO ] [MAIN ] Update Progress: 400/2367 symbols completed. CBAN took 25.8s. Estimated remaining: 4h 7m 12s. Rate: 8.0 symbols/min [2025-10-20T17:50:21.179793] [INFO ] [MAIN ] CBT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:21.315733] [INFO ] [MAIN ] CBNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:21.393014] [INFO ] [MAIN ] CBRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:21.389393] [INFO ] [MAIN ] CBO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:21.394078] [DEBUG] [LOCAL ] CBO: No existing data file found in primary storage. [2025-10-20T17:50:21.396667] [DEBUG] [LOCAL ] CBNA: Loaded 2942 existing records from primary storage. [2025-10-20T17:50:21.397281] [INFO ] [MAIN ] CBNA: Loaded 2942 existing records. [2025-10-20T17:50:21.396891] [WARN ] [MAIN ] CBO: No existing data found, performing full refresh instead. [2025-10-20T17:50:21.408709] [INFO ] [MAIN ] CBO: Wiping existing primary data before full refresh. [2025-10-20T17:50:21.446203] [INFO ] [LOCAL ] CBO: Starting deletion of primary data. [2025-10-20T17:50:21.501898] [DEBUG] [LOCAL ] CBO: No primary data file found to delete. [2025-10-20T17:50:21.520783] [INFO ] [MAIN ] CBO: Starting full primary refresh. [2025-10-20T17:50:21.559017] [DEBUG] [LOCAL ] CBT: Loaded 31588 existing records from primary storage. [2025-10-20T17:50:21.578550] [INFO ] [MAIN ] CBT: Loaded 31588 existing records. [2025-10-20T17:50:21.791406] [WARN ] [MAIN ] CBNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:21.860358] [DEBUG] [LOCAL ] CBRE: Loaded 51089 existing records from primary storage. [2025-10-20T17:50:21.898404] [INFO ] [MAIN ] CBNA: Wiping existing primary data before full refresh. [2025-10-20T17:50:21.932457] [INFO ] [MAIN ] CBRE: Loaded 51089 existing records. [2025-10-20T17:50:21.924599] [INFO ] [LOCAL ] CBNA: Starting deletion of primary data. [2025-10-20T17:50:22.024982] [DEBUG] [LOCAL ] CBNA: Deleted primary data file. [2025-10-20T17:50:22.144124] [INFO ] [MAIN ] CBNA: Starting full primary refresh. [2025-10-20T17:50:22.229377] [WARN ] [MAIN ] CBT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:22.295139] [INFO ] [MAIN ] CBT: Wiping existing primary data before full refresh. [2025-10-20T17:50:22.342299] [INFO ] [LOCAL ] CBT: Starting deletion of primary data. [2025-10-20T17:50:22.376733] [DEBUG] [LOCAL ] CBT: Deleted primary data file. [2025-10-20T17:50:22.460670] [INFO ] [MAIN ] CBT: Starting full primary refresh. [2025-10-20T17:50:22.550754] [WARN ] [MAIN ] CBRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:22.566541] [INFO ] [REMOTE ] CBO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:22.576560] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-01. [2025-10-20T17:50:22.618071] [INFO ] [MAIN ] CBRE: Wiping existing primary data before full refresh. [2025-10-20T17:50:22.657289] [INFO ] [LOCAL ] CBRE: Starting deletion of primary data. [2025-10-20T17:50:22.733107] [DEBUG] [LOCAL ] CBRE: Deleted primary data file. [2025-10-20T17:50:22.760436] [INFO ] [MAIN ] CBRE: Starting full primary refresh. [2025-10-20T17:50:22.919430] [DEBUG] [REMOTE ] CBO: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:50:22.961935] [WARN ] [REMOTE ] CBO: No data returned for month 2025-01 [2025-10-20T17:50:22.978226] [FATAL] [REMOTE ] CBO: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:50:22.992168] [FATAL] [MAIN ] CBO: Failed to fetch primary data. [2025-10-20T17:50:23.009295] [WARN ] [MAIN ] CBO: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:50:23.064280] [INFO ] [LOCAL ] CBO: Starting deletion of primary data. [2025-10-20T17:50:23.198998] [INFO ] [REMOTE ] CBNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:23.242324] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-01. [2025-10-20T17:50:23.195710] [DEBUG] [LOCAL ] CBO: No primary data file found to delete. [2025-10-20T17:50:23.266560] [INFO ] [LOCAL ] CBO: Starting deletion of derivative data. [2025-10-20T17:50:23.381346] [DEBUG] [LOCAL ] CBO: No derivative data file found to delete. [2025-10-20T17:50:23.401089] [INFO ] [MAIN ] Update Progress: 401/2367 symbols completed. CBO took 2.8s. Estimated remaining: 4h 6m 41s. Rate: 8.0 symbols/min [2025-10-20T17:50:23.677074] [INFO ] [REMOTE ] CBT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:23.722959] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-01. [2025-10-20T17:50:23.717476] [INFO ] [REMOTE ] CBRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:23.737209] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-01. [2025-10-20T17:50:23.841565] [INFO ] [MAIN ] CBU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:23.866178] [DEBUG] [LOCAL ] CBU: Loaded 23143 existing records from primary storage. [2025-10-20T17:50:23.866479] [INFO ] [MAIN ] CBU: Loaded 23143 existing records. [2025-10-20T17:50:24.080730] [WARN ] [MAIN ] CBU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:24.085639] [INFO ] [MAIN ] CBU: Wiping existing primary data before full refresh. [2025-10-20T17:50:24.085737] [INFO ] [LOCAL ] CBU: Starting deletion of primary data. [2025-10-20T17:50:24.086568] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T17:50:24.086713] [DEBUG] [LOCAL ] CBU: Deleted primary data file. [2025-10-20T17:50:24.087061] [INFO ] [MAIN ] CBU: Starting full primary refresh. [2025-10-20T17:50:24.122339] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-02. [2025-10-20T17:50:24.483663] [INFO ] [REMOTE ] CBU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:24.483963] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-01. [2025-10-20T17:50:24.577241] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T17:50:24.579927] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-03. [2025-10-20T17:50:24.841208] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:50:25.036565] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T17:50:25.051384] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-04. [2025-10-20T17:50:25.346810] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-10-20T17:50:25.665365] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:50:25.711696] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T17:50:25.851447] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-02. [2025-10-20T17:50:26.651397] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-05. [2025-10-20T17:50:27.289261] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-10-20T17:50:27.355900] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T17:50:27.451378] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-06. [2025-10-20T17:50:27.862965] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-10-20T17:50:28.251389] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-02. [2025-10-20T17:50:29.051392] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-03. [2025-10-20T17:50:29.645410] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T17:50:29.851383] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-03. [2025-10-20T17:50:30.466575] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:50:30.651368] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-04. [2025-10-20T17:50:31.231629] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T17:50:31.451369] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-02. [2025-10-20T17:50:32.149237] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T17:50:32.251380] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-07. [2025-10-20T17:50:32.507707] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T17:50:32.619064] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-10-20T17:50:33.051377] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-03. [2025-10-20T17:50:33.851417] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-04. [2025-10-20T17:50:34.142570] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:50:34.651408] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-05. [2025-10-20T17:50:35.035969] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T17:50:35.451372] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-08. [2025-10-20T17:50:35.881601] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T17:50:36.157826] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:50:36.251382] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-04. [2025-10-20T17:50:37.051375] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-09. [2025-10-20T17:50:37.478237] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-10-20T17:50:37.633305] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:50:37.851364] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-05. [2025-10-20T17:50:38.651431] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-10. [2025-10-20T17:50:39.074302] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T17:50:39.075241] [INFO ] [REMOTE ] CBNA: Full fetch complete (4335 records). [2025-10-20T17:50:39.075312] [DEBUG] [VALIDATOR ] CBNA: Validating primary data. [2025-10-20T17:50:39.412343] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:50:39.454350] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-06. [2025-10-20T17:50:39.490757] [DEBUG] [VALIDATOR ] CBNA: All expected trading days are present. [2025-10-20T17:50:39.490833] [DEBUG] [VALIDATOR ] CBNA: Primary data validation passed. [2025-10-20T17:50:39.490973] [INFO ] [MAIN ] CBNA: Fetched primary data is valid. [2025-10-20T17:50:39.491013] [DEBUG] [LOCAL ] CBNA: Saving primary data. [2025-10-20T17:50:39.491416] [DEBUG] [LOCAL ] CBNA: Trimmed 4335 records to 2900 (keeping 200 days of data) [2025-10-20T17:50:39.509947] [DEBUG] [LOCAL ] CBNA: Primary data saved (2900 records). [2025-10-20T17:50:39.510082] [INFO ] [MAIN ] CBNA: Full history saved. [2025-10-20T17:50:39.510203] [INFO ] [MAIN ] Update Progress: 402/2367 symbols completed. CBNA took 18.9s. Estimated remaining: 4h 7m 16s. Rate: 7.9 symbols/min [2025-10-20T17:50:39.695275] [INFO ] [MAIN ] CBX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:39.695442] [DEBUG] [LOCAL ] CBX: No existing data file found in primary storage. [2025-10-20T17:50:39.695483] [WARN ] [MAIN ] CBX: No existing data found, performing full refresh instead. [2025-10-20T17:50:39.695595] [INFO ] [MAIN ] CBX: Wiping existing primary data before full refresh. [2025-10-20T17:50:39.695637] [INFO ] [LOCAL ] CBX: Starting deletion of primary data. [2025-10-20T17:50:39.695726] [DEBUG] [LOCAL ] CBX: No primary data file found to delete. [2025-10-20T17:50:39.695765] [INFO ] [MAIN ] CBX: Starting full primary refresh. [2025-10-20T17:50:40.057063] [INFO ] [REMOTE ] CBX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:40.251374] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-05. [2025-10-20T17:50:40.825271] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:50:41.051363] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-07. [2025-10-20T17:50:41.262940] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T17:50:41.851417] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-06. [2025-10-20T17:50:42.655253] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-06. [2025-10-20T17:50:42.660393] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T17:50:42.693877] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T17:50:43.451455] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-07. [2025-10-20T17:50:43.630905] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T17:50:44.251373] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-01. [2025-10-20T17:50:44.441632] [DEBUG] [REMOTE ] CBX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:50:44.442004] [WARN ] [REMOTE ] CBX: No data returned for month 2025-01 [2025-10-20T17:50:44.442106] [FATAL] [REMOTE ] CBX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:50:44.442191] [FATAL] [MAIN ] CBX: Failed to fetch primary data. [2025-10-20T17:50:44.442277] [WARN ] [MAIN ] CBX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:50:44.442376] [INFO ] [LOCAL ] CBX: Starting deletion of primary data. [2025-10-20T17:50:44.442460] [DEBUG] [LOCAL ] CBX: No primary data file found to delete. [2025-10-20T17:50:44.442499] [INFO ] [LOCAL ] CBX: Starting deletion of derivative data. [2025-10-20T17:50:44.442537] [DEBUG] [LOCAL ] CBX: No derivative data file found to delete. [2025-10-20T17:50:44.442581] [INFO ] [MAIN ] Update Progress: 403/2367 symbols completed. CBX took 4.9s. Estimated remaining: 4h 6m 55s. Rate: 8.0 symbols/min [2025-10-20T17:50:44.651187] [INFO ] [MAIN ] CBZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:44.685236] [DEBUG] [LOCAL ] CBZ: Loaded 34777 existing records from primary storage. [2025-10-20T17:50:44.685532] [INFO ] [MAIN ] CBZ: Loaded 34777 existing records. [2025-10-20T17:50:44.839871] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:50:44.899476] [WARN ] [MAIN ] CBZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:44.899966] [INFO ] [MAIN ] CBZ: Wiping existing primary data before full refresh. [2025-10-20T17:50:44.900014] [INFO ] [LOCAL ] CBZ: Starting deletion of primary data. [2025-10-20T17:50:44.900188] [DEBUG] [LOCAL ] CBZ: Deleted primary data file. [2025-10-20T17:50:44.900227] [INFO ] [MAIN ] CBZ: Starting full primary refresh. [2025-10-20T17:50:45.076113] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-07. [2025-10-20T17:50:45.260871] [INFO ] [REMOTE ] CBZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:45.851413] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-08. [2025-10-20T17:50:46.267063] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:50:46.651411] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-01. [2025-10-20T17:50:47.119716] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T17:50:47.451387] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-09. [2025-10-20T17:50:47.818373] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T17:50:48.251396] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-08. [2025-10-20T17:50:48.765843] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T17:50:49.051431] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-10. [2025-10-20T17:50:49.791775] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T17:50:49.851360] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-08. [2025-10-20T17:50:50.402021] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T17:50:50.407752] [INFO ] [REMOTE ] CBT: Full fetch complete (45846 records). [2025-10-20T17:50:50.407938] [DEBUG] [VALIDATOR ] CBT: Validating primary data. [2025-10-20T17:50:50.657067] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-09. [2025-10-20T17:50:50.790085] [DEBUG] [VALIDATOR ] CBT: All expected trading days are present. [2025-10-20T17:50:50.790173] [DEBUG] [VALIDATOR ] CBT: Primary data validation passed. [2025-10-20T17:50:50.791012] [INFO ] [MAIN ] CBT: Fetched primary data is valid. [2025-10-20T17:50:50.791075] [DEBUG] [LOCAL ] CBT: Saving primary data. [2025-10-20T17:50:50.792530] [DEBUG] [LOCAL ] CBT: Trimmed 45846 records to 31150 (keeping 200 days of data) [2025-10-20T17:50:51.014387] [DEBUG] [LOCAL ] CBT: Primary data saved (31150 records). [2025-10-20T17:50:51.014538] [INFO ] [MAIN ] CBT: Full history saved. [2025-10-20T17:50:51.014619] [INFO ] [MAIN ] Update Progress: 404/2367 symbols completed. CBT took 30.4s. Estimated remaining: 4h 6m 43s. Rate: 8.0 symbols/min [2025-10-20T17:50:51.203439] [INFO ] [MAIN ] CC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:51.220721] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T17:50:51.281875] [DEBUG] [LOCAL ] CC: Loaded 53789 existing records from primary storage. [2025-10-20T17:50:51.282234] [INFO ] [MAIN ] CC: Loaded 53789 existing records. [2025-10-20T17:50:51.457112] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-02. [2025-10-20T17:50:51.462219] [WARN ] [MAIN ] CC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:51.463615] [INFO ] [MAIN ] CC: Wiping existing primary data before full refresh. [2025-10-20T17:50:51.463779] [INFO ] [LOCAL ] CC: Starting deletion of primary data. [2025-10-20T17:50:51.464758] [DEBUG] [LOCAL ] CC: Deleted primary data file. [2025-10-20T17:50:51.464837] [INFO ] [MAIN ] CC: Starting full primary refresh. [2025-10-20T17:50:51.886339] [INFO ] [REMOTE ] CC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:52.084505] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:50:52.251354] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-10. [2025-10-20T17:50:52.653481] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T17:50:53.051495] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-09. [2025-10-20T17:50:53.513864] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T17:50:53.521793] [INFO ] [REMOTE ] CBRE: Full fetch complete (73785 records). [2025-10-20T17:50:53.521960] [DEBUG] [VALIDATOR ] CBRE: Validating primary data. [2025-10-20T17:50:53.861018] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-01. [2025-10-20T17:50:53.928605] [DEBUG] [VALIDATOR ] CBRE: All expected trading days are present. [2025-10-20T17:50:53.928746] [DEBUG] [VALIDATOR ] CBRE: Primary data validation passed. [2025-10-20T17:50:53.929585] [INFO ] [MAIN ] CBRE: Fetched primary data is valid. [2025-10-20T17:50:53.929631] [DEBUG] [LOCAL ] CBRE: Saving primary data. [2025-10-20T17:50:53.931686] [DEBUG] [LOCAL ] CBRE: Trimmed 73785 records to 50305 (keeping 200 days of data) [2025-10-20T17:50:54.266481] [DEBUG] [LOCAL ] CBRE: Primary data saved (50305 records). [2025-10-20T17:50:54.267339] [INFO ] [MAIN ] CBRE: Full history saved. [2025-10-20T17:50:54.269170] [INFO ] [MAIN ] Update Progress: 405/2367 symbols completed. CBRE took 33.6s. Estimated remaining: 4h 6m 15s. Rate: 8.0 symbols/min [2025-10-20T17:50:54.269014] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T17:50:54.466727] [INFO ] [MAIN ] CCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:54.520993] [DEBUG] [LOCAL ] CCI: Loaded 53890 existing records from primary storage. [2025-10-20T17:50:54.521346] [INFO ] [MAIN ] CCI: Loaded 53890 existing records. [2025-10-20T17:50:54.662008] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-10. [2025-10-20T17:50:54.703738] [WARN ] [MAIN ] CCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:54.704819] [INFO ] [MAIN ] CCI: Wiping existing primary data before full refresh. [2025-10-20T17:50:54.704872] [INFO ] [LOCAL ] CCI: Starting deletion of primary data. [2025-10-20T17:50:54.705380] [DEBUG] [LOCAL ] CCI: Deleted primary data file. [2025-10-20T17:50:54.705436] [INFO ] [MAIN ] CCI: Starting full primary refresh. [2025-10-20T17:50:55.075844] [INFO ] [REMOTE ] CCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:55.451404] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-03. [2025-10-20T17:50:55.480308] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T17:50:55.744300] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T17:50:55.751113] [INFO ] [REMOTE ] CBU: Full fetch complete (32722 records). [2025-10-20T17:50:55.751304] [DEBUG] [VALIDATOR ] CBU: Validating primary data. [2025-10-20T17:50:56.165138] [DEBUG] [VALIDATOR ] CBU: All expected trading days are present. [2025-10-20T17:50:56.165220] [DEBUG] [VALIDATOR ] CBU: Primary data validation passed. [2025-10-20T17:50:56.165736] [INFO ] [MAIN ] CBU: Fetched primary data is valid. [2025-10-20T17:50:56.165798] [DEBUG] [LOCAL ] CBU: Saving primary data. [2025-10-20T17:50:56.166638] [DEBUG] [LOCAL ] CBU: Trimmed 32722 records to 22862 (keeping 200 days of data) [2025-10-20T17:50:56.255967] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-02. [2025-10-20T17:50:56.308634] [DEBUG] [LOCAL ] CBU: Primary data saved (22862 records). [2025-10-20T17:50:56.308789] [INFO ] [MAIN ] CBU: Full history saved. [2025-10-20T17:50:56.308869] [INFO ] [MAIN ] Update Progress: 406/2367 symbols completed. CBU took 32.9s. Estimated remaining: 4h 5m 41s. Rate: 8.0 symbols/min [2025-10-20T17:50:56.480036] [INFO ] [MAIN ] CCIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:56.480227] [DEBUG] [LOCAL ] CCIA: No existing data file found in primary storage. [2025-10-20T17:50:56.480334] [WARN ] [MAIN ] CCIA: No existing data found, performing full refresh instead. [2025-10-20T17:50:56.480456] [INFO ] [MAIN ] CCIA: Wiping existing primary data before full refresh. [2025-10-20T17:50:56.480499] [INFO ] [LOCAL ] CCIA: Starting deletion of primary data. [2025-10-20T17:50:56.480540] [DEBUG] [LOCAL ] CCIA: No primary data file found to delete. [2025-10-20T17:50:56.480574] [INFO ] [MAIN ] CCIA: Starting full primary refresh. [2025-10-20T17:50:56.850431] [INFO ] [REMOTE ] CCIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:56.870282] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:50:57.051404] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-01. [2025-10-20T17:50:57.283867] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T17:50:57.623957] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T17:50:57.851423] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-04. [2025-10-20T17:50:58.651377] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-01. [2025-10-20T17:50:59.179516] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:50:59.451438] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-02. [2025-10-20T17:50:59.769454] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T17:51:00.251468] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-05. [2025-10-20T17:51:00.326002] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T17:51:01.051430] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-03. [2025-10-20T17:51:01.661693] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T17:51:01.851449] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-02. [2025-10-20T17:51:02.651419] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-06. [2025-10-20T17:51:02.968256] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T17:51:03.315615] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:51:03.451453] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-03. [2025-10-20T17:51:03.646596] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T17:51:03.797687] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T17:51:04.251383] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-04. [2025-10-20T17:51:04.546606] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-10-20T17:51:05.051394] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-04. [2025-10-20T17:51:05.851397] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-03. [2025-10-20T17:51:06.651369] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-05. [2025-10-20T17:51:06.944056] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T17:51:07.114488] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T17:51:07.451355] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-07. [2025-10-20T17:51:07.658836] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T17:51:08.251373] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-04. [2025-10-20T17:51:08.809988] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:51:09.051387] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-05. [2025-10-20T17:51:09.851431] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-08. [2025-10-20T17:51:10.291511] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T17:51:10.609626] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:51:10.651335] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-06. [2025-10-20T17:51:11.353743] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T17:51:11.451351] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-05. [2025-10-20T17:51:12.251449] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-06. [2025-10-20T17:51:12.288666] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T17:51:12.713778] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T17:51:13.051366] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-09. [2025-10-20T17:51:13.262219] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:51:13.851565] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-06. [2025-10-20T17:51:14.268262] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T17:51:14.651379] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-07. [2025-10-20T17:51:15.094974] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-10-20T17:51:15.451341] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-07. [2025-10-20T17:51:15.558041] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T17:51:16.251380] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-07. [2025-10-20T17:51:17.051355] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-10. [2025-10-20T17:51:17.561449] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T17:51:17.683995] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T17:51:17.851433] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-08. [2025-10-20T17:51:18.139566] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-10-20T17:51:18.496822] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T17:51:18.504597] [INFO ] [REMOTE ] CBZ: Full fetch complete (46238 records). [2025-10-20T17:51:18.504793] [DEBUG] [VALIDATOR ] CBZ: Validating primary data. [2025-10-20T17:51:18.657636] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-08. [2025-10-20T17:51:18.888135] [DEBUG] [VALIDATOR ] CBZ: All expected trading days are present. [2025-10-20T17:51:18.888215] [DEBUG] [VALIDATOR ] CBZ: Primary data validation passed. [2025-10-20T17:51:18.891773] [INFO ] [MAIN ] CBZ: Fetched primary data is valid. [2025-10-20T17:51:18.891868] [DEBUG] [LOCAL ] CBZ: Saving primary data. [2025-10-20T17:51:18.892950] [DEBUG] [LOCAL ] CBZ: Trimmed 46238 records to 34284 (keeping 200 days of data) [2025-10-20T17:51:19.084997] [DEBUG] [LOCAL ] CBZ: Primary data saved (34284 records). [2025-10-20T17:51:19.085146] [INFO ] [MAIN ] CBZ: Full history saved. [2025-10-20T17:51:19.085235] [INFO ] [MAIN ] Update Progress: 407/2367 symbols completed. CBZ took 34.6s. Estimated remaining: 4h 6m 47s. Rate: 7.9 symbols/min [2025-10-20T17:51:19.259548] [INFO ] [MAIN ] CCIF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:19.274062] [DEBUG] [LOCAL ] CCIF: Loaded 15711 existing records from primary storage. [2025-10-20T17:51:19.274297] [INFO ] [MAIN ] CCIF: Loaded 15711 existing records. [2025-10-20T17:51:19.458761] [WARN ] [MAIN ] CCIF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:19.458906] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-09. [2025-10-20T17:51:19.459303] [INFO ] [MAIN ] CCIF: Wiping existing primary data before full refresh. [2025-10-20T17:51:19.460268] [INFO ] [LOCAL ] CCIF: Starting deletion of primary data. [2025-10-20T17:51:19.460455] [DEBUG] [LOCAL ] CCIF: Deleted primary data file. [2025-10-20T17:51:19.460496] [INFO ] [MAIN ] CCIF: Starting full primary refresh. [2025-10-20T17:51:19.815188] [INFO ] [REMOTE ] CCIF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:20.033840] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T17:51:20.251391] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-08. [2025-10-20T17:51:20.476259] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T17:51:21.051382] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-01. [2025-10-20T17:51:21.766922] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T17:51:21.851356] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-02. [2025-10-20T17:51:22.037395] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T17:51:22.503818] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:51:22.651358] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-03. [2025-10-20T17:51:23.451364] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-09. [2025-10-20T17:51:23.469820] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T17:51:24.251410] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-04. [2025-10-20T17:51:25.051385] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-09. [2025-10-20T17:51:25.259296] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T17:51:25.284868] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T17:51:25.851381] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-05. [2025-10-20T17:51:26.651434] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-10. [2025-10-20T17:51:26.858523] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T17:51:26.994615] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T17:51:27.116317] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T17:51:27.117429] [INFO ] [REMOTE ] CCIA: Full fetch complete (2166 records). [2025-10-20T17:51:27.117503] [DEBUG] [VALIDATOR ] CCIA: Validating primary data. [2025-10-20T17:51:27.316754] [WARN ] [VALIDATOR ] CCIA: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T17:51:27.316965] [FATAL] [MAIN ] CCIA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:51:27.317033] [WARN ] [MAIN ] CCIA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:51:27.317084] [INFO ] [LOCAL ] CCIA: Starting deletion of primary data. [2025-10-20T17:51:27.317148] [DEBUG] [LOCAL ] CCIA: No primary data file found to delete. [2025-10-20T17:51:27.317184] [INFO ] [LOCAL ] CCIA: Starting deletion of derivative data. [2025-10-20T17:51:27.317223] [DEBUG] [LOCAL ] CCIA: No derivative data file found to delete. [2025-10-20T17:51:27.317283] [INFO ] [MAIN ] Update Progress: 408/2367 symbols completed. CCIA took 31.0s. Estimated remaining: 4h 6m 43s. Rate: 7.9 symbols/min [2025-10-20T17:51:27.461823] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-10. [2025-10-20T17:51:27.487842] [INFO ] [MAIN ] CCJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:27.539312] [DEBUG] [LOCAL ] CCJ: Loaded 54468 existing records from primary storage. [2025-10-20T17:51:27.539778] [INFO ] [MAIN ] CCJ: Loaded 54468 existing records. [2025-10-20T17:51:27.726532] [WARN ] [MAIN ] CCJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:27.726776] [INFO ] [MAIN ] CCJ: Wiping existing primary data before full refresh. [2025-10-20T17:51:27.726828] [INFO ] [LOCAL ] CCJ: Starting deletion of primary data. [2025-10-20T17:51:27.727164] [DEBUG] [LOCAL ] CCJ: Deleted primary data file. [2025-10-20T17:51:27.727221] [INFO ] [MAIN ] CCJ: Starting full primary refresh. [2025-10-20T17:51:28.081960] [INFO ] [REMOTE ] CCJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:28.251389] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-01. [2025-10-20T17:51:29.051355] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-10. [2025-10-20T17:51:29.151105] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:51:29.160477] [INFO ] [REMOTE ] CC: Full fetch complete (76146 records). [2025-10-20T17:51:29.160743] [DEBUG] [VALIDATOR ] CC: Validating primary data. [2025-10-20T17:51:29.569287] [DEBUG] [VALIDATOR ] CC: All expected trading days are present. [2025-10-20T17:51:29.569384] [DEBUG] [VALIDATOR ] CC: Primary data validation passed. [2025-10-20T17:51:29.570244] [INFO ] [MAIN ] CC: Fetched primary data is valid. [2025-10-20T17:51:29.570309] [DEBUG] [LOCAL ] CC: Saving primary data. [2025-10-20T17:51:29.571756] [DEBUG] [LOCAL ] CC: Trimmed 76146 records to 53030 (keeping 200 days of data) [2025-10-20T17:51:29.863170] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-06. [2025-10-20T17:51:29.888252] [DEBUG] [LOCAL ] CC: Primary data saved (53030 records). [2025-10-20T17:51:29.888627] [INFO ] [MAIN ] CC: Full history saved. [2025-10-20T17:51:29.888751] [INFO ] [MAIN ] Update Progress: 409/2367 symbols completed. CC took 38.9s. Estimated remaining: 4h 6m 11s. Rate: 8.0 symbols/min [2025-10-20T17:51:30.060256] [INFO ] [MAIN ] CCK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:30.100439] [DEBUG] [LOCAL ] CCK: Loaded 47154 existing records from primary storage. [2025-10-20T17:51:30.100772] [INFO ] [MAIN ] CCK: Loaded 47154 existing records. [2025-10-20T17:51:30.278218] [WARN ] [MAIN ] CCK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:30.278517] [INFO ] [MAIN ] CCK: Wiping existing primary data before full refresh. [2025-10-20T17:51:30.278580] [INFO ] [LOCAL ] CCK: Starting deletion of primary data. [2025-10-20T17:51:30.278872] [DEBUG] [LOCAL ] CCK: Deleted primary data file. [2025-10-20T17:51:30.278927] [INFO ] [MAIN ] CCK: Starting full primary refresh. [2025-10-20T17:51:30.644116] [INFO ] [REMOTE ] CCK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:30.658431] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-01. [2025-10-20T17:51:30.683745] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-01 (2.43 seconds). [2025-10-20T17:51:30.901121] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:51:31.128002] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T17:51:31.135897] [INFO ] [REMOTE ] CCI: Full fetch complete (77098 records). [2025-10-20T17:51:31.136019] [DEBUG] [VALIDATOR ] CCI: Validating primary data. [2025-10-20T17:51:31.464965] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-07. [2025-10-20T17:51:31.523727] [DEBUG] [VALIDATOR ] CCI: All expected trading days are present. [2025-10-20T17:51:31.523815] [DEBUG] [VALIDATOR ] CCI: Primary data validation passed. [2025-10-20T17:51:31.524614] [INFO ] [MAIN ] CCI: Fetched primary data is valid. [2025-10-20T17:51:31.524691] [DEBUG] [LOCAL ] CCI: Saving primary data. [2025-10-20T17:51:31.526302] [DEBUG] [LOCAL ] CCI: Trimmed 77098 records to 53116 (keeping 200 days of data) [2025-10-20T17:51:31.851370] [DEBUG] [LOCAL ] CCI: Primary data saved (53116 records). [2025-10-20T17:51:31.851510] [INFO ] [MAIN ] CCI: Full history saved. [2025-10-20T17:51:31.851835] [INFO ] [MAIN ] Update Progress: 410/2367 symbols completed. CCI took 37.6s. Estimated remaining: 4h 5m 37s. Rate: 8.0 symbols/min [2025-10-20T17:51:32.020289] [INFO ] [MAIN ] CCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:32.071108] [DEBUG] [LOCAL ] CCL: Loaded 54580 existing records from primary storage. [2025-10-20T17:51:32.071419] [INFO ] [MAIN ] CCL: Loaded 54580 existing records. [2025-10-20T17:51:32.255267] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-02. [2025-10-20T17:51:32.262623] [WARN ] [MAIN ] CCL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:32.281600] [INFO ] [MAIN ] CCL: Wiping existing primary data before full refresh. [2025-10-20T17:51:32.282304] [INFO ] [LOCAL ] CCL: Starting deletion of primary data. [2025-10-20T17:51:32.283612] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:51:32.292457] [DEBUG] [LOCAL ] CCL: Deleted primary data file. [2025-10-20T17:51:32.294070] [INFO ] [MAIN ] CCL: Starting full primary refresh. [2025-10-20T17:51:32.505286] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T17:51:32.701201] [INFO ] [REMOTE ] CCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:33.051407] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-08. [2025-10-20T17:51:33.851453] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-01. [2025-10-20T17:51:33.947863] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T17:51:34.217434] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-10-20T17:51:34.651418] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-09. [2025-10-20T17:51:35.433326] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T17:51:35.451349] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-10. [2025-10-20T17:51:36.188446] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-01 (2.34 seconds). [2025-10-20T17:51:36.220664] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T17:51:36.223085] [INFO ] [REMOTE ] CCIF: Full fetch complete (21876 records). [2025-10-20T17:51:36.223161] [DEBUG] [VALIDATOR ] CCIF: Validating primary data. [2025-10-20T17:51:36.256460] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-02. [2025-10-20T17:51:36.594543] [DEBUG] [VALIDATOR ] CCIF: All expected trading days are present. [2025-10-20T17:51:36.594622] [DEBUG] [VALIDATOR ] CCIF: Primary data validation passed. [2025-10-20T17:51:36.595026] [INFO ] [MAIN ] CCIF: Fetched primary data is valid. [2025-10-20T17:51:36.595078] [DEBUG] [LOCAL ] CCIF: Saving primary data. [2025-10-20T17:51:36.595732] [DEBUG] [LOCAL ] CCIF: Trimmed 21876 records to 15294 (keeping 200 days of data) [2025-10-20T17:51:36.684893] [DEBUG] [LOCAL ] CCIF: Primary data saved (15294 records). [2025-10-20T17:51:36.685037] [INFO ] [MAIN ] CCIF: Full history saved. [2025-10-20T17:51:36.685116] [INFO ] [MAIN ] Update Progress: 411/2367 symbols completed. CCIF took 17.6s. Estimated remaining: 4h 5m 17s. Rate: 8.0 symbols/min [2025-10-20T17:51:36.878926] [INFO ] [MAIN ] CCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:36.920565] [DEBUG] [LOCAL ] CCO: Loaded 49089 existing records from primary storage. [2025-10-20T17:51:36.920934] [INFO ] [MAIN ] CCO: Loaded 49089 existing records. [2025-10-20T17:51:37.060437] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-03. [2025-10-20T17:51:37.103242] [WARN ] [MAIN ] CCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:37.103729] [INFO ] [MAIN ] CCO: Wiping existing primary data before full refresh. [2025-10-20T17:51:37.103798] [INFO ] [LOCAL ] CCO: Starting deletion of primary data. [2025-10-20T17:51:37.104067] [DEBUG] [LOCAL ] CCO: Deleted primary data file. [2025-10-20T17:51:37.104123] [INFO ] [MAIN ] CCO: Starting full primary refresh. [2025-10-20T17:51:37.473871] [INFO ] [REMOTE ] CCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:37.680245] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:51:37.851377] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-03. [2025-10-20T17:51:38.651330] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-02. [2025-10-20T17:51:39.161339] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T17:51:39.208897] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-10-20T17:51:39.451372] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-04. [2025-10-20T17:51:40.251392] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-04. [2025-10-20T17:51:40.877218] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-02 (2.23 seconds). [2025-10-20T17:51:41.051407] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-03. [2025-10-20T17:51:41.133153] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T17:51:41.799225] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:51:41.851399] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-05. [2025-10-20T17:51:42.651385] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-05. [2025-10-20T17:51:43.451361] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-01. [2025-10-20T17:51:43.524449] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-03 (2.47 seconds). [2025-10-20T17:51:43.627159] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T17:51:43.846977] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T17:51:44.251423] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-06. [2025-10-20T17:51:45.056842] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-06. [2025-10-20T17:51:45.061382] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:51:45.851439] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-02. [2025-10-20T17:51:46.116057] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T17:51:46.250714] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:51:46.651412] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-07. [2025-10-20T17:51:47.244447] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T17:51:47.451364] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-03. [2025-10-20T17:51:48.251423] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-04. [2025-10-20T17:51:48.652958] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T17:51:48.857150] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T17:51:49.051486] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-04. [2025-10-20T17:51:49.851458] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-08. [2025-10-20T17:51:50.456275] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-10-20T17:51:50.651397] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-05. [2025-10-20T17:51:50.845419] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:51:51.451429] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-05. [2025-10-20T17:51:51.777068] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T17:51:52.251433] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-07. [2025-10-20T17:51:52.653398] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T17:51:53.051456] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-09. [2025-10-20T17:51:53.170182] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:51:53.492558] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T17:51:53.851458] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-08. [2025-10-20T17:51:54.651434] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-06. [2025-10-20T17:51:55.033231] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T17:51:55.359573] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:51:55.451360] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-09. [2025-10-20T17:51:56.251420] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-10. [2025-10-20T17:51:56.982509] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-06 (2.33 seconds). [2025-10-20T17:51:57.051343] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-07. [2025-10-20T17:51:57.096432] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:51:57.851370] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-06. [2025-10-20T17:51:58.651437] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-10. [2025-10-20T17:51:58.762110] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-10 (2.51 seconds). [2025-10-20T17:51:58.770442] [INFO ] [REMOTE ] CCJ: Full fetch complete (77840 records). [2025-10-20T17:51:58.770615] [DEBUG] [VALIDATOR ] CCJ: Validating primary data. [2025-10-20T17:51:59.176738] [DEBUG] [VALIDATOR ] CCJ: All expected trading days are present. [2025-10-20T17:51:59.177105] [DEBUG] [VALIDATOR ] CCJ: Primary data validation passed. [2025-10-20T17:51:59.178017] [INFO ] [MAIN ] CCJ: Fetched primary data is valid. [2025-10-20T17:51:59.178082] [DEBUG] [LOCAL ] CCJ: Saving primary data. [2025-10-20T17:51:59.179622] [DEBUG] [LOCAL ] CCJ: Trimmed 77840 records to 53690 (keeping 200 days of data) [2025-10-20T17:51:59.529631] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-07 (2.48 seconds). [2025-10-20T17:51:59.535549] [DEBUG] [LOCAL ] CCJ: Primary data saved (53690 records). [2025-10-20T17:51:59.546709] [INFO ] [MAIN ] CCJ: Full history saved. [2025-10-20T17:51:59.547736] [INFO ] [MAIN ] Update Progress: 412/2367 symbols completed. CCJ took 32.2s. Estimated remaining: 4h 6m 22s. Rate: 7.9 symbols/min [2025-10-20T17:51:59.576598] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-08. [2025-10-20T17:51:59.642560] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T17:51:59.763087] [INFO ] [MAIN ] CCRD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:59.772890] [DEBUG] [LOCAL ] CCRD: Loaded 9647 existing records from primary storage. [2025-10-20T17:51:59.773117] [INFO ] [MAIN ] CCRD: Loaded 9647 existing records. [2025-10-20T17:51:59.947999] [WARN ] [MAIN ] CCRD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:59.948233] [INFO ] [MAIN ] CCRD: Wiping existing primary data before full refresh. [2025-10-20T17:51:59.948289] [INFO ] [LOCAL ] CCRD: Starting deletion of primary data. [2025-10-20T17:51:59.948478] [DEBUG] [LOCAL ] CCRD: Deleted primary data file. [2025-10-20T17:51:59.948530] [INFO ] [MAIN ] CCRD: Starting full primary refresh. [2025-10-20T17:52:00.251942] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-07. [2025-10-20T17:52:00.326911] [INFO ] [REMOTE ] CCRD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:00.409850] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:52:00.417103] [INFO ] [REMOTE ] CCK: Full fetch complete (68007 records). [2025-10-20T17:52:00.417261] [DEBUG] [VALIDATOR ] CCK: Validating primary data. [2025-10-20T17:52:00.809770] [DEBUG] [VALIDATOR ] CCK: All expected trading days are present. [2025-10-20T17:52:00.809855] [DEBUG] [VALIDATOR ] CCK: Primary data validation passed. [2025-10-20T17:52:00.810573] [INFO ] [MAIN ] CCK: Fetched primary data is valid. [2025-10-20T17:52:00.810612] [DEBUG] [LOCAL ] CCK: Saving primary data. [2025-10-20T17:52:00.811898] [DEBUG] [LOCAL ] CCK: Trimmed 68007 records to 46482 (keeping 200 days of data) [2025-10-20T17:52:01.053708] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-01. [2025-10-20T17:52:01.109596] [DEBUG] [LOCAL ] CCK: Primary data saved (46482 records). [2025-10-20T17:52:01.109773] [INFO ] [MAIN ] CCK: Full history saved. [2025-10-20T17:52:01.109866] [INFO ] [MAIN ] Update Progress: 413/2367 symbols completed. CCK took 31.2s. Estimated remaining: 4h 5m 46s. Rate: 8.0 symbols/min [2025-10-20T17:52:01.280881] [INFO ] [MAIN ] CCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:01.310352] [DEBUG] [LOCAL ] CCS: Loaded 30600 existing records from primary storage. [2025-10-20T17:52:01.310627] [INFO ] [MAIN ] CCS: Loaded 30600 existing records. [2025-10-20T17:52:01.487523] [WARN ] [MAIN ] CCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:01.487755] [INFO ] [MAIN ] CCS: Wiping existing primary data before full refresh. [2025-10-20T17:52:01.487806] [INFO ] [LOCAL ] CCS: Starting deletion of primary data. [2025-10-20T17:52:01.488176] [DEBUG] [LOCAL ] CCS: Deleted primary data file. [2025-10-20T17:52:01.488233] [INFO ] [MAIN ] CCS: Starting full primary refresh. [2025-10-20T17:52:01.794286] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T17:52:01.854953] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-02. [2025-10-20T17:52:01.909840] [INFO ] [REMOTE ] CCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:02.062510] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-08 (2.49 seconds). [2025-10-20T17:52:02.110446] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T17:52:02.336206] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T17:52:02.651374] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-01. [2025-10-20T17:52:03.451446] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-08. [2025-10-20T17:52:04.127931] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T17:52:04.251450] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-03. [2025-10-20T17:52:04.825729] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T17:52:04.825566] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T17:52:05.051403] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-09. [2025-10-20T17:52:05.851484] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-02. [2025-10-20T17:52:06.630042] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T17:52:06.651355] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-10. [2025-10-20T17:52:06.868145] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T17:52:07.451379] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-09. [2025-10-20T17:52:08.251404] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-04. [2025-10-20T17:52:08.425580] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T17:52:08.433087] [INFO ] [REMOTE ] CCO: Full fetch complete (67203 records). [2025-10-20T17:52:08.433235] [DEBUG] [VALIDATOR ] CCO: Validating primary data. [2025-10-20T17:52:08.888806] [DEBUG] [VALIDATOR ] CCO: All expected trading days are present. [2025-10-20T17:52:08.893231] [DEBUG] [VALIDATOR ] CCO: Primary data validation passed. [2025-10-20T17:52:08.894055] [INFO ] [MAIN ] CCO: Fetched primary data is valid. [2025-10-20T17:52:08.894800] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T17:52:08.894617] [DEBUG] [LOCAL ] CCO: Saving primary data. [2025-10-20T17:52:08.898179] [DEBUG] [LOCAL ] CCO: Trimmed 67203 records to 48483 (keeping 200 days of data) [2025-10-20T17:52:09.057558] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-05. [2025-10-20T17:52:09.165772] [DEBUG] [LOCAL ] CCO: Primary data saved (48483 records). [2025-10-20T17:52:09.165892] [INFO ] [MAIN ] CCO: Full history saved. [2025-10-20T17:52:09.165965] [INFO ] [MAIN ] Update Progress: 414/2367 symbols completed. CCO took 32.5s. Estimated remaining: 4h 5m 41s. Rate: 7.9 symbols/min [2025-10-20T17:52:09.333826] [INFO ] [MAIN ] CCU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:09.373953] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:52:09.395146] [DEBUG] [LOCAL ] CCU: Loaded 22640 existing records from primary storage. [2025-10-20T17:52:09.395484] [INFO ] [MAIN ] CCU: Loaded 22640 existing records. [2025-10-20T17:52:09.604603] [WARN ] [MAIN ] CCU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:09.604882] [INFO ] [MAIN ] CCU: Wiping existing primary data before full refresh. [2025-10-20T17:52:09.604951] [INFO ] [LOCAL ] CCU: Starting deletion of primary data. [2025-10-20T17:52:09.605146] [DEBUG] [LOCAL ] CCU: Deleted primary data file. [2025-10-20T17:52:09.605203] [INFO ] [MAIN ] CCU: Starting full primary refresh. [2025-10-20T17:52:09.777110] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T17:52:09.856830] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-10. [2025-10-20T17:52:10.026067] [INFO ] [REMOTE ] CCU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:10.651369] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-03. [2025-10-20T17:52:11.451368] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-06. [2025-10-20T17:52:12.054499] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T17:52:12.070590] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T17:52:12.251396] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-01. [2025-10-20T17:52:12.417372] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-10 (2.56 seconds). [2025-10-20T17:52:12.425927] [INFO ] [REMOTE ] CCL: Full fetch complete (77980 records). [2025-10-20T17:52:12.426096] [DEBUG] [VALIDATOR ] CCL: Validating primary data. [2025-10-20T17:52:12.826543] [DEBUG] [VALIDATOR ] CCL: All expected trading days are present. [2025-10-20T17:52:12.826696] [DEBUG] [VALIDATOR ] CCL: Primary data validation passed. [2025-10-20T17:52:12.827517] [INFO ] [MAIN ] CCL: Fetched primary data is valid. [2025-10-20T17:52:12.827568] [DEBUG] [LOCAL ] CCL: Saving primary data. [2025-10-20T17:52:12.829100] [DEBUG] [LOCAL ] CCL: Trimmed 77980 records to 53800 (keeping 200 days of data) [2025-10-20T17:52:13.060090] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-07. [2025-10-20T17:52:13.156831] [DEBUG] [LOCAL ] CCL: Primary data saved (53800 records). [2025-10-20T17:52:13.156987] [INFO ] [MAIN ] CCL: Full history saved. [2025-10-20T17:52:13.157447] [INFO ] [MAIN ] Update Progress: 415/2367 symbols completed. CCL took 41.3s. Estimated remaining: 4h 5m 16s. Rate: 8.0 symbols/min [2025-10-20T17:52:13.330222] [INFO ] [MAIN ] CCZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:13.330499] [DEBUG] [LOCAL ] CCZ: No existing data file found in primary storage. [2025-10-20T17:52:13.330632] [WARN ] [MAIN ] CCZ: No existing data found, performing full refresh instead. [2025-10-20T17:52:13.330774] [INFO ] [MAIN ] CCZ: Wiping existing primary data before full refresh. [2025-10-20T17:52:13.330818] [INFO ] [LOCAL ] CCZ: Starting deletion of primary data. [2025-10-20T17:52:13.330876] [DEBUG] [LOCAL ] CCZ: No primary data file found to delete. [2025-10-20T17:52:13.330986] [INFO ] [MAIN ] CCZ: Starting full primary refresh. [2025-10-20T17:52:13.453360] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:52:13.738877] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T17:52:13.744513] [INFO ] [REMOTE ] CCZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:13.851362] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-08. [2025-10-20T17:52:14.651368] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-04. [2025-10-20T17:52:14.861321] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:52:15.451383] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-02. [2025-10-20T17:52:16.181847] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T17:52:16.251374] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-05. [2025-10-20T17:52:16.564567] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T17:52:17.051371] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-09. [2025-10-20T17:52:17.551604] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T17:52:17.851403] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-01. [2025-10-20T17:52:17.935974] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T17:52:18.181274] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T17:52:18.651383] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-06. [2025-10-20T17:52:19.451440] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-03. [2025-10-20T17:52:19.667232] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T17:52:20.251423] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-02. [2025-10-20T17:52:20.538107] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-10-20T17:52:20.761497] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T17:52:21.051407] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-04. [2025-10-20T17:52:21.851392] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-03. [2025-10-20T17:52:22.074492] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-03 (0.22 seconds). [2025-10-20T17:52:22.396386] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:52:22.651376] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-05. [2025-10-20T17:52:23.451422] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-07. [2025-10-20T17:52:24.018711] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:52:24.251369] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-06. [2025-10-20T17:52:24.810525] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:52:25.051351] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-10. [2025-10-20T17:52:25.286597] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:52:25.726336] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:52:25.728142] [INFO ] [REMOTE ] CCRD: Full fetch complete (11827 records). [2025-10-20T17:52:25.728210] [DEBUG] [VALIDATOR ] CCRD: Validating primary data. [2025-10-20T17:52:25.865737] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-04. [2025-10-20T17:52:26.105069] [DEBUG] [VALIDATOR ] CCRD: All expected trading days are present. [2025-10-20T17:52:26.105144] [DEBUG] [VALIDATOR ] CCRD: Primary data validation passed. [2025-10-20T17:52:26.105425] [INFO ] [MAIN ] CCRD: Fetched primary data is valid. [2025-10-20T17:52:26.105464] [DEBUG] [LOCAL ] CCRD: Saving primary data. [2025-10-20T17:52:26.106024] [DEBUG] [LOCAL ] CCRD: Trimmed 11827 records to 9603 (keeping 200 days of data) [2025-10-20T17:52:26.165018] [DEBUG] [LOCAL ] CCRD: Primary data saved (9603 records). [2025-10-20T17:52:26.165144] [INFO ] [MAIN ] CCRD: Full history saved. [2025-10-20T17:52:26.165218] [INFO ] [MAIN ] Update Progress: 416/2367 symbols completed. CCRD took 26.6s. Estimated remaining: 4h 5m 35s. Rate: 7.9 symbols/min [2025-10-20T17:52:26.213369] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T17:52:26.345738] [INFO ] [MAIN ] CDE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:26.393561] [DEBUG] [LOCAL ] CDE: Loaded 54423 existing records from primary storage. [2025-10-20T17:52:26.393894] [INFO ] [MAIN ] CDE: Loaded 54423 existing records. [2025-10-20T17:52:26.568354] [WARN ] [MAIN ] CDE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:26.568559] [INFO ] [MAIN ] CDE: Wiping existing primary data before full refresh. [2025-10-20T17:52:26.568604] [INFO ] [LOCAL ] CDE: Starting deletion of primary data. [2025-10-20T17:52:26.568863] [DEBUG] [LOCAL ] CDE: Deleted primary data file. [2025-10-20T17:52:26.568919] [INFO ] [MAIN ] CDE: Starting full primary refresh. [2025-10-20T17:52:26.656544] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-07. [2025-10-20T17:52:26.939744] [INFO ] [REMOTE ] CDE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:27.451424] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-01. [2025-10-20T17:52:27.928492] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T17:52:28.251418] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-08. [2025-10-20T17:52:29.051411] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-08. [2025-10-20T17:52:29.157598] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T17:52:29.289382] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T17:52:29.851390] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-05. [2025-10-20T17:52:30.051693] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T17:52:30.162808] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T17:52:30.651390] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-09. [2025-10-20T17:52:31.451466] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-09. [2025-10-20T17:52:31.939867] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T17:52:32.251435] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-10. [2025-10-20T17:52:32.433453] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T17:52:33.051425] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-02. [2025-10-20T17:52:33.548171] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T17:52:33.555539] [INFO ] [REMOTE ] CCS: Full fetch complete (44811 records). [2025-10-20T17:52:33.555689] [DEBUG] [VALIDATOR ] CCS: Validating primary data. [2025-10-20T17:52:33.856521] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-06. [2025-10-20T17:52:33.932158] [DEBUG] [VALIDATOR ] CCS: All expected trading days are present. [2025-10-20T17:52:33.932298] [DEBUG] [VALIDATOR ] CCS: Primary data validation passed. [2025-10-20T17:52:33.932933] [INFO ] [MAIN ] CCS: Fetched primary data is valid. [2025-10-20T17:52:33.932996] [DEBUG] [LOCAL ] CCS: Saving primary data. [2025-10-20T17:52:33.934101] [DEBUG] [LOCAL ] CCS: Trimmed 44811 records to 30084 (keeping 200 days of data) [2025-10-20T17:52:34.110881] [DEBUG] [LOCAL ] CCS: Primary data saved (30084 records). [2025-10-20T17:52:34.111011] [INFO ] [MAIN ] CCS: Full history saved. [2025-10-20T17:52:34.111088] [INFO ] [MAIN ] Update Progress: 417/2367 symbols completed. CCS took 33.0s. Estimated remaining: 4h 5m 29s. Rate: 7.9 symbols/min [2025-10-20T17:52:34.179115] [DEBUG] [REMOTE ] CCZ: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:52:34.200837] [WARN ] [REMOTE ] CCZ: No data returned for month 2025-06 [2025-10-20T17:52:34.211346] [FATAL] [REMOTE ] CCZ: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:52:34.211439] [FATAL] [MAIN ] CCZ: Failed to fetch primary data. [2025-10-20T17:52:34.211809] [WARN ] [MAIN ] CCZ: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:52:34.211892] [INFO ] [LOCAL ] CCZ: Starting deletion of primary data. [2025-10-20T17:52:34.222441] [DEBUG] [LOCAL ] CCZ: No primary data file found to delete. [2025-10-20T17:52:34.227664] [INFO ] [LOCAL ] CCZ: Starting deletion of derivative data. [2025-10-20T17:52:34.237957] [DEBUG] [LOCAL ] CCZ: No derivative data file found to delete. [2025-10-20T17:52:34.250428] [INFO ] [MAIN ] Update Progress: 418/2367 symbols completed. CCZ took 21.1s. Estimated remaining: 4h 4m 47s. Rate: 8.0 symbols/min [2025-10-20T17:52:34.347000] [INFO ] [MAIN ] CDLR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:34.444830] [DEBUG] [LOCAL ] CDLR: Loaded 10294 existing records from primary storage. [2025-10-20T17:52:34.462718] [INFO ] [MAIN ] CDLR: Loaded 10294 existing records. [2025-10-20T17:52:34.463574] [INFO ] [MAIN ] CDP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:34.650084] [WARN ] [MAIN ] CDLR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:34.650736] [INFO ] [MAIN ] CDLR: Wiping existing primary data before full refresh. [2025-10-20T17:52:34.650924] [INFO ] [LOCAL ] CDLR: Starting deletion of primary data. [2025-10-20T17:52:34.651131] [DEBUG] [LOCAL ] CDLR: Deleted primary data file. [2025-10-20T17:52:34.651222] [INFO ] [MAIN ] CDLR: Starting full primary refresh. [2025-10-20T17:52:34.662421] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-10. [2025-10-20T17:52:34.963552] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T17:52:34.960695] [DEBUG] [LOCAL ] CDP: Loaded 43123 existing records from primary storage. [2025-10-20T17:52:34.970264] [INFO ] [MAIN ] CDP: Loaded 43123 existing records. [2025-10-20T17:52:35.267182] [INFO ] [REMOTE ] CDLR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:35.290677] [WARN ] [MAIN ] CDP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:35.290916] [INFO ] [MAIN ] CDP: Wiping existing primary data before full refresh. [2025-10-20T17:52:35.290967] [INFO ] [LOCAL ] CDP: Starting deletion of primary data. [2025-10-20T17:52:35.291194] [DEBUG] [LOCAL ] CDP: Deleted primary data file. [2025-10-20T17:52:35.291249] [INFO ] [MAIN ] CDP: Starting full primary refresh. [2025-10-20T17:52:35.455390] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-01. [2025-10-20T17:52:35.662511] [INFO ] [REMOTE ] CDP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:35.934065] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T17:52:35.937701] [INFO ] [REMOTE ] CCU: Full fetch complete (32526 records). [2025-10-20T17:52:35.937804] [DEBUG] [VALIDATOR ] CCU: Validating primary data. [2025-10-20T17:52:36.256695] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-03. [2025-10-20T17:52:36.345839] [DEBUG] [VALIDATOR ] CCU: All expected trading days are present. [2025-10-20T17:52:36.346299] [DEBUG] [VALIDATOR ] CCU: Primary data validation passed. [2025-10-20T17:52:36.346194] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T17:52:36.347286] [INFO ] [MAIN ] CCU: Fetched primary data is valid. [2025-10-20T17:52:36.349148] [DEBUG] [LOCAL ] CCU: Saving primary data. [2025-10-20T17:52:36.351061] [DEBUG] [LOCAL ] CCU: Trimmed 32526 records to 21982 (keeping 200 days of data) [2025-10-20T17:52:36.488267] [DEBUG] [LOCAL ] CCU: Primary data saved (21982 records). [2025-10-20T17:52:36.488441] [INFO ] [MAIN ] CCU: Full history saved. [2025-10-20T17:52:36.488533] [INFO ] [MAIN ] Update Progress: 419/2367 symbols completed. CCU took 27.3s. Estimated remaining: 4h 4m 15s. Rate: 8.0 symbols/min [2025-10-20T17:52:36.660565] [INFO ] [MAIN ] CDRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:36.683682] [DEBUG] [LOCAL ] CDRE: Loaded 25250 existing records from primary storage. [2025-10-20T17:52:36.683962] [INFO ] [MAIN ] CDRE: Loaded 25250 existing records. [2025-10-20T17:52:36.861453] [WARN ] [MAIN ] CDRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:36.861714] [INFO ] [MAIN ] CDRE: Wiping existing primary data before full refresh. [2025-10-20T17:52:36.861770] [INFO ] [LOCAL ] CDRE: Starting deletion of primary data. [2025-10-20T17:52:36.862084] [DEBUG] [LOCAL ] CDRE: Deleted primary data file. [2025-10-20T17:52:36.862139] [INFO ] [MAIN ] CDRE: Starting full primary refresh. [2025-10-20T17:52:37.056618] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-01. [2025-10-20T17:52:37.226995] [INFO ] [REMOTE ] CDRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:37.851458] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-01. [2025-10-20T17:52:37.998219] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:52:38.651367] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-04. [2025-10-20T17:52:38.719592] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:52:38.745527] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T17:52:39.451437] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-02. [2025-10-20T17:52:40.251478] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-02. [2025-10-20T17:52:40.811986] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T17:52:40.868033] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:52:40.983696] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T17:52:41.051389] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-03. [2025-10-20T17:52:41.851424] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-05. [2025-10-20T17:52:42.385528] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T17:52:42.651437] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-03. [2025-10-20T17:52:43.451424] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-02. [2025-10-20T17:52:43.520713] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T17:52:43.789172] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T17:52:44.102841] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:52:44.251460] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-04. [2025-10-20T17:52:45.051422] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-04. [2025-10-20T17:52:45.856602] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-03. [2025-10-20T17:52:45.887298] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:52:45.926757] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T17:52:46.651444] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-05. [2025-10-20T17:52:46.757052] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T17:52:47.451383] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-06. [2025-10-20T17:52:48.048823] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:52:48.251407] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-05. [2025-10-20T17:52:49.051391] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-04. [2025-10-20T17:52:49.282793] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T17:52:49.375009] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:52:49.801623] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T17:52:49.851342] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-05. [2025-10-20T17:52:50.472060] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T17:52:50.651422] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-06. [2025-10-20T17:52:51.451460] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-06. [2025-10-20T17:52:52.153865] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T17:52:52.224334] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T17:52:52.251385] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-07. [2025-10-20T17:52:53.051397] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-07. [2025-10-20T17:52:53.110268] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T17:52:53.851436] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-07. [2025-10-20T17:52:54.282618] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T17:52:54.651386] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-08. [2025-10-20T17:52:55.451471] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-08. [2025-10-20T17:52:56.001460] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-07 (2.15 seconds). [2025-10-20T17:52:56.100361] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T17:52:56.251443] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-08. [2025-10-20T17:52:56.408008] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:52:57.051377] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-09. [2025-10-20T17:52:57.851431] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-06. [2025-10-20T17:52:58.067747] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T17:52:58.222080] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T17:52:58.644376] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:52:58.651333] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-07. [2025-10-20T17:52:59.174634] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T17:52:59.451369] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-09. [2025-10-20T17:53:00.251439] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-08. [2025-10-20T17:53:01.051429] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-09. [2025-10-20T17:53:01.077074] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T17:53:01.607017] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-10-20T17:53:01.851367] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-10. [2025-10-20T17:53:02.273428] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T17:53:02.651480] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-10. [2025-10-20T17:53:03.173074] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T17:53:03.179467] [INFO ] [REMOTE ] CDP: Full fetch complete (61721 records). [2025-10-20T17:53:03.179681] [DEBUG] [VALIDATOR ] CDP: Validating primary data. [2025-10-20T17:53:03.180505] [FATAL] [VALIDATOR ] CDP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:53:03.180596] [FATAL] [MAIN ] CDP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:53:03.180664] [WARN ] [MAIN ] CDP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:53:03.180744] [INFO ] [LOCAL ] CDP: Starting deletion of primary data. [2025-10-20T17:53:03.180809] [DEBUG] [LOCAL ] CDP: No primary data file found to delete. [2025-10-20T17:53:03.180844] [INFO ] [LOCAL ] CDP: Starting deletion of derivative data. [2025-10-20T17:53:03.181032] [DEBUG] [LOCAL ] CDP: Deleted derivative data file. [2025-10-20T17:53:03.181370] [INFO ] [MAIN ] Update Progress: 420/2367 symbols completed. CDP took 28.9s. Estimated remaining: 4h 5m 36s. Rate: 7.9 symbols/min [2025-10-20T17:53:03.361271] [INFO ] [MAIN ] CE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:03.409155] [DEBUG] [LOCAL ] CE: Loaded 52065 existing records from primary storage. [2025-10-20T17:53:03.409467] [INFO ] [MAIN ] CE: Loaded 52065 existing records. [2025-10-20T17:53:03.459152] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-09. [2025-10-20T17:53:03.617604] [WARN ] [MAIN ] CE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:03.617907] [INFO ] [MAIN ] CE: Wiping existing primary data before full refresh. [2025-10-20T17:53:03.617956] [INFO ] [LOCAL ] CE: Starting deletion of primary data. [2025-10-20T17:53:03.618346] [DEBUG] [LOCAL ] CE: Deleted primary data file. [2025-10-20T17:53:03.618404] [INFO ] [MAIN ] CE: Starting full primary refresh. [2025-10-20T17:53:03.991951] [INFO ] [REMOTE ] CE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:04.251451] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-01. [2025-10-20T17:53:04.345600] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T17:53:04.639592] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T17:53:04.647702] [INFO ] [REMOTE ] CDE: Full fetch complete (77787 records). [2025-10-20T17:53:04.647831] [DEBUG] [VALIDATOR ] CDE: Validating primary data. [2025-10-20T17:53:05.053260] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-10. [2025-10-20T17:53:05.106915] [DEBUG] [VALIDATOR ] CDE: All expected trading days are present. [2025-10-20T17:53:05.106996] [DEBUG] [VALIDATOR ] CDE: Primary data validation passed. [2025-10-20T17:53:05.107866] [INFO ] [MAIN ] CDE: Fetched primary data is valid. [2025-10-20T17:53:05.107927] [DEBUG] [LOCAL ] CDE: Saving primary data. [2025-10-20T17:53:05.109430] [DEBUG] [LOCAL ] CDE: Trimmed 77787 records to 53643 (keeping 200 days of data) [2025-10-20T17:53:05.408463] [DEBUG] [LOCAL ] CDE: Primary data saved (53643 records). [2025-10-20T17:53:05.408597] [INFO ] [MAIN ] CDE: Full history saved. [2025-10-20T17:53:05.408695] [INFO ] [MAIN ] Update Progress: 421/2367 symbols completed. CDE took 39.2s. Estimated remaining: 4h 5m 4s. Rate: 7.9 symbols/min [2025-10-20T17:53:05.579286] [INFO ] [MAIN ] CEE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:05.586700] [DEBUG] [LOCAL ] CEE: Loaded 5462 existing records from primary storage. [2025-10-20T17:53:05.586879] [INFO ] [MAIN ] CEE: Loaded 5462 existing records. [2025-10-20T17:53:05.765538] [WARN ] [MAIN ] CEE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:05.765860] [INFO ] [MAIN ] CEE: Wiping existing primary data before full refresh. [2025-10-20T17:53:05.766021] [INFO ] [LOCAL ] CEE: Starting deletion of primary data. [2025-10-20T17:53:05.766202] [DEBUG] [LOCAL ] CEE: Deleted primary data file. [2025-10-20T17:53:05.766265] [INFO ] [MAIN ] CEE: Starting full primary refresh. [2025-10-20T17:53:05.862514] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-10. [2025-10-20T17:53:05.998015] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T17:53:06.078788] [INFO ] [REMOTE ] CDLR: Full fetch complete (14119 records). [2025-10-20T17:53:06.106373] [DEBUG] [VALIDATOR ] CDLR: Validating primary data. [2025-10-20T17:53:06.177717] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:53:06.181275] [INFO ] [REMOTE ] CEE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:06.578372] [DEBUG] [VALIDATOR ] CDLR: All expected trading days are present. [2025-10-20T17:53:06.578466] [DEBUG] [VALIDATOR ] CDLR: Primary data validation passed. [2025-10-20T17:53:06.578988] [INFO ] [MAIN ] CDLR: Fetched primary data is valid. [2025-10-20T17:53:06.579050] [DEBUG] [LOCAL ] CDLR: Saving primary data. [2025-10-20T17:53:06.579572] [DEBUG] [LOCAL ] CDLR: Trimmed 14119 records to 10221 (keeping 200 days of data) [2025-10-20T17:53:06.639120] [DEBUG] [LOCAL ] CDLR: Primary data saved (10221 records). [2025-10-20T17:53:06.639273] [INFO ] [MAIN ] CDLR: Full history saved. [2025-10-20T17:53:06.639356] [INFO ] [MAIN ] Update Progress: 422/2367 symbols completed. CDLR took 32.5s. Estimated remaining: 4h 4m 27s. Rate: 8.0 symbols/min [2025-10-20T17:53:06.662048] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-01. [2025-10-20T17:53:06.820571] [INFO ] [MAIN ] CEPU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:06.848498] [DEBUG] [LOCAL ] CEPU: Loaded 27225 existing records from primary storage. [2025-10-20T17:53:06.848817] [INFO ] [MAIN ] CEPU: Loaded 27225 existing records. [2025-10-20T17:53:07.036668] [WARN ] [MAIN ] CEPU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:07.037048] [INFO ] [MAIN ] CEPU: Wiping existing primary data before full refresh. [2025-10-20T17:53:07.037172] [INFO ] [LOCAL ] CEPU: Starting deletion of primary data. [2025-10-20T17:53:07.037410] [DEBUG] [LOCAL ] CEPU: Deleted primary data file. [2025-10-20T17:53:07.042667] [INFO ] [MAIN ] CEPU: Starting full primary refresh. [2025-10-20T17:53:07.114570] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T17:53:07.154291] [INFO ] [REMOTE ] CDRE: Full fetch complete (35110 records). [2025-10-20T17:53:07.156877] [DEBUG] [VALIDATOR ] CDRE: Validating primary data. [2025-10-20T17:53:07.470625] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-02. [2025-10-20T17:53:07.512052] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T17:53:07.729251] [INFO ] [REMOTE ] CEPU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:07.807705] [DEBUG] [VALIDATOR ] CDRE: All expected trading days are present. [2025-10-20T17:53:07.807789] [DEBUG] [VALIDATOR ] CDRE: Primary data validation passed. [2025-10-20T17:53:07.808294] [INFO ] [MAIN ] CDRE: Fetched primary data is valid. [2025-10-20T17:53:07.808364] [DEBUG] [LOCAL ] CDRE: Saving primary data. [2025-10-20T17:53:07.809131] [DEBUG] [LOCAL ] CDRE: Trimmed 35110 records to 24879 (keeping 200 days of data) [2025-10-20T17:53:07.958083] [DEBUG] [LOCAL ] CDRE: Primary data saved (24879 records). [2025-10-20T17:53:07.958225] [INFO ] [MAIN ] CDRE: Full history saved. [2025-10-20T17:53:07.958310] [INFO ] [MAIN ] Update Progress: 423/2367 symbols completed. CDRE took 31.5s. Estimated remaining: 4h 3m 51s. Rate: 8.0 symbols/min [2025-10-20T17:53:08.138146] [INFO ] [MAIN ] CF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:08.184016] [DEBUG] [LOCAL ] CF: Loaded 52902 existing records from primary storage. [2025-10-20T17:53:08.184329] [INFO ] [MAIN ] CF: Loaded 52902 existing records. [2025-10-20T17:53:08.256732] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-02. [2025-10-20T17:53:08.359798] [WARN ] [MAIN ] CF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:08.359996] [INFO ] [MAIN ] CF: Wiping existing primary data before full refresh. [2025-10-20T17:53:08.360043] [INFO ] [LOCAL ] CF: Starting deletion of primary data. [2025-10-20T17:53:08.360226] [DEBUG] [LOCAL ] CF: Deleted primary data file. [2025-10-20T17:53:08.360264] [INFO ] [MAIN ] CF: Starting full primary refresh. [2025-10-20T17:53:08.702235] [INFO ] [REMOTE ] CF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:09.051415] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-01. [2025-10-20T17:53:09.188732] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:53:09.219258] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T17:53:09.851437] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-01. [2025-10-20T17:53:10.651425] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-03. [2025-10-20T17:53:10.744093] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:53:11.274060] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:53:11.451362] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-02. [2025-10-20T17:53:12.251347] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-02. [2025-10-20T17:53:12.564772] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T17:53:12.608165] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:53:13.051373] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-03. [2025-10-20T17:53:13.785707] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T17:53:13.851365] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-03. [2025-10-20T17:53:13.926295] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T17:53:14.651387] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-04. [2025-10-20T17:53:14.770868] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-10-20T17:53:15.451439] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-04. [2025-10-20T17:53:16.097422] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:53:16.251409] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-05. [2025-10-20T17:53:16.603828] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T17:53:16.959824] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T17:53:17.051385] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-03. [2025-10-20T17:53:17.851373] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-04. [2025-10-20T17:53:18.574536] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:53:18.644358] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T17:53:18.651448] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-05. [2025-10-20T17:53:19.451421] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-06. [2025-10-20T17:53:19.902614] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T17:53:20.226227] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T17:53:20.251391] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-05. [2025-10-20T17:53:21.051356] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-04. [2025-10-20T17:53:21.851379] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-06. [2025-10-20T17:53:22.023072] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:53:22.651372] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-06. [2025-10-20T17:53:22.979988] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T17:53:23.008907] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T17:53:23.451424] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-05. [2025-10-20T17:53:24.259387] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-07. [2025-10-20T17:53:24.287620] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T17:53:25.051380] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-07. [2025-10-20T17:53:25.317866] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T17:53:25.461544] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T17:53:25.752766] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T17:53:25.851394] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-08. [2025-10-20T17:53:26.504752] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T17:53:26.651395] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-07. [2025-10-20T17:53:27.451423] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-06. [2025-10-20T17:53:28.251399] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-09. [2025-10-20T17:53:28.486779] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T17:53:28.831746] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T17:53:28.997422] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:53:29.051352] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-07. [2025-10-20T17:53:29.851403] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-08. [2025-10-20T17:53:30.523982] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T17:53:30.651374] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-08. [2025-10-20T17:53:30.974353] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:53:31.451408] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-08. [2025-10-20T17:53:32.047926] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:53:32.251385] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-10. [2025-10-20T17:53:32.930622] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T17:53:32.931840] [INFO ] [REMOTE ] CEE: Full fetch complete (10175 records). [2025-10-20T17:53:32.931911] [DEBUG] [VALIDATOR ] CEE: Validating primary data. [2025-10-20T17:53:33.056773] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-09. [2025-10-20T17:53:33.342337] [DEBUG] [VALIDATOR ] CEE: All expected trading days are present. [2025-10-20T17:53:33.342533] [DEBUG] [VALIDATOR ] CEE: Primary data validation passed. [2025-10-20T17:53:33.342899] [INFO ] [MAIN ] CEE: Fetched primary data is valid. [2025-10-20T17:53:33.342989] [DEBUG] [LOCAL ] CEE: Saving primary data. [2025-10-20T17:53:33.343780] [DEBUG] [LOCAL ] CEE: Trimmed 10175 records to 5304 (keeping 200 days of data) [2025-10-20T17:53:33.411751] [DEBUG] [LOCAL ] CEE: Primary data saved (5304 records). [2025-10-20T17:53:33.412805] [INFO ] [MAIN ] CEE: Full history saved. [2025-10-20T17:53:33.413429] [INFO ] [MAIN ] Update Progress: 424/2367 symbols completed. CEE took 28.0s. Estimated remaining: 4h 5m 5s. Rate: 7.9 symbols/min [2025-10-20T17:53:33.417845] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T17:53:33.605668] [INFO ] [MAIN ] CFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:33.656264] [DEBUG] [LOCAL ] CFG: Loaded 54403 existing records from primary storage. [2025-10-20T17:53:33.656622] [INFO ] [MAIN ] CFG: Loaded 54403 existing records. [2025-10-20T17:53:33.842891] [WARN ] [MAIN ] CFG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:33.843109] [INFO ] [MAIN ] CFG: Wiping existing primary data before full refresh. [2025-10-20T17:53:33.843156] [INFO ] [LOCAL ] CFG: Starting deletion of primary data. [2025-10-20T17:53:33.843516] [DEBUG] [LOCAL ] CFG: Deleted primary data file. [2025-10-20T17:53:33.843555] [INFO ] [MAIN ] CFG: Starting full primary refresh. [2025-10-20T17:53:33.856580] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-09. [2025-10-20T17:53:34.226219] [INFO ] [REMOTE ] CFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:34.268132] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T17:53:34.651392] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-01. [2025-10-20T17:53:35.451351] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-09. [2025-10-20T17:53:35.627619] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:53:36.170335] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T17:53:36.251386] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-10. [2025-10-20T17:53:37.050758] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:53:37.060982] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-10. [2025-10-20T17:53:37.314058] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T17:53:37.319369] [INFO ] [REMOTE ] CEPU: Full fetch complete (39845 records). [2025-10-20T17:53:37.319516] [DEBUG] [VALIDATOR ] CEPU: Validating primary data. [2025-10-20T17:53:37.685601] [DEBUG] [VALIDATOR ] CEPU: All expected trading days are present. [2025-10-20T17:53:37.685707] [DEBUG] [VALIDATOR ] CEPU: Primary data validation passed. [2025-10-20T17:53:37.686248] [INFO ] [MAIN ] CEPU: Fetched primary data is valid. [2025-10-20T17:53:37.686293] [DEBUG] [LOCAL ] CEPU: Saving primary data. [2025-10-20T17:53:37.687126] [DEBUG] [LOCAL ] CEPU: Trimmed 39845 records to 26925 (keeping 200 days of data) [2025-10-20T17:53:37.836410] [DEBUG] [LOCAL ] CEPU: Primary data saved (26925 records). [2025-10-20T17:53:37.836604] [INFO ] [MAIN ] CEPU: Full history saved. [2025-10-20T17:53:37.836752] [INFO ] [MAIN ] Update Progress: 425/2367 symbols completed. CEPU took 31.2s. Estimated remaining: 4h 4m 43s. Rate: 7.9 symbols/min [2025-10-20T17:53:37.857445] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-10. [2025-10-20T17:53:38.006278] [INFO ] [MAIN ] CFND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:38.006414] [DEBUG] [LOCAL ] CFND: No existing data file found in primary storage. [2025-10-20T17:53:38.006451] [WARN ] [MAIN ] CFND: No existing data found, performing full refresh instead. [2025-10-20T17:53:38.006581] [INFO ] [MAIN ] CFND: Wiping existing primary data before full refresh. [2025-10-20T17:53:38.006637] [INFO ] [LOCAL ] CFND: Starting deletion of primary data. [2025-10-20T17:53:38.006735] [DEBUG] [LOCAL ] CFND: No primary data file found to delete. [2025-10-20T17:53:38.006773] [INFO ] [MAIN ] CFND: Starting full primary refresh. [2025-10-20T17:53:38.368534] [INFO ] [REMOTE ] CFND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:38.651393] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-02. [2025-10-20T17:53:38.963047] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T17:53:38.975992] [INFO ] [REMOTE ] CF: Full fetch complete (75588 records). [2025-10-20T17:53:38.976300] [DEBUG] [VALIDATOR ] CF: Validating primary data. [2025-10-20T17:53:39.467174] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-01. [2025-10-20T17:53:39.544570] [DEBUG] [VALIDATOR ] CF: All expected trading days are present. [2025-10-20T17:53:39.544784] [DEBUG] [VALIDATOR ] CF: Primary data validation passed. [2025-10-20T17:53:39.545745] [INFO ] [MAIN ] CF: Fetched primary data is valid. [2025-10-20T17:53:39.545811] [DEBUG] [LOCAL ] CF: Saving primary data. [2025-10-20T17:53:39.547378] [DEBUG] [LOCAL ] CF: Trimmed 75588 records to 52126 (keeping 200 days of data) [2025-10-20T17:53:39.803085] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:53:39.808827] [WARN ] [REMOTE ] CFND: No data returned for month 2025-01 [2025-10-20T17:53:39.813816] [FATAL] [REMOTE ] CFND: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:53:39.817169] [FATAL] [MAIN ] CFND: Failed to fetch primary data. [2025-10-20T17:53:39.824337] [WARN ] [MAIN ] CFND: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:53:39.826930] [INFO ] [LOCAL ] CFND: Starting deletion of primary data. [2025-10-20T17:53:39.827290] [DEBUG] [LOCAL ] CFND: No primary data file found to delete. [2025-10-20T17:53:39.827419] [INFO ] [LOCAL ] CFND: Starting deletion of derivative data. [2025-10-20T17:53:39.867314] [DEBUG] [LOCAL ] CFND: No derivative data file found to delete. [2025-10-20T17:53:39.899839] [INFO ] [MAIN ] Update Progress: 426/2367 symbols completed. CFND took 2.1s. Estimated remaining: 4h 4m 11s. Rate: 7.9 symbols/min [2025-10-20T17:53:39.912207] [DEBUG] [LOCAL ] CF: Primary data saved (52126 records). [2025-10-20T17:53:39.942566] [INFO ] [MAIN ] CF: Full history saved. [2025-10-20T17:53:40.015219] [INFO ] [MAIN ] Update Progress: 427/2367 symbols completed. CF took 32.1s. Estimated remaining: 4h 3m 30s. Rate: 8.0 symbols/min [2025-10-20T17:53:40.027970] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-10 (2.17 seconds). [2025-10-20T17:53:40.116906] [INFO ] [REMOTE ] CE: Full fetch complete (74838 records). [2025-10-20T17:53:40.126832] [DEBUG] [VALIDATOR ] CE: Validating primary data. [2025-10-20T17:53:40.333460] [INFO ] [MAIN ] CFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:40.450111] [INFO ] [MAIN ] CGAU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:40.750088] [DEBUG] [VALIDATOR ] CE: All expected trading days are present. [2025-10-20T17:53:40.759304] [DEBUG] [VALIDATOR ] CE: Primary data validation passed. [2025-10-20T17:53:40.760279] [INFO ] [MAIN ] CE: Fetched primary data is valid. [2025-10-20T17:53:40.760625] [DEBUG] [LOCAL ] CE: Saving primary data. [2025-10-20T17:53:40.800297] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-02 (2.15 seconds). [2025-10-20T17:53:40.798671] [DEBUG] [LOCAL ] CE: Trimmed 74838 records to 51312 (keeping 200 days of data) [2025-10-20T17:53:40.798472] [DEBUG] [LOCAL ] CFR: Loaded 33178 existing records from primary storage. [2025-10-20T17:53:40.822603] [INFO ] [MAIN ] CFR: Loaded 33178 existing records. [2025-10-20T17:53:40.823850] [DEBUG] [LOCAL ] CGAU: Loaded 44311 existing records from primary storage. [2025-10-20T17:53:40.887936] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-03. [2025-10-20T17:53:40.908601] [INFO ] [MAIN ] CGAU: Loaded 44311 existing records. [2025-10-20T17:53:41.253218] [WARN ] [MAIN ] CFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:41.277476] [INFO ] [MAIN ] CFR: Wiping existing primary data before full refresh. [2025-10-20T17:53:41.336393] [INFO ] [LOCAL ] CFR: Starting deletion of primary data. [2025-10-20T17:53:41.376116] [WARN ] [MAIN ] CGAU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:41.403639] [DEBUG] [LOCAL ] CFR: Deleted primary data file. [2025-10-20T17:53:41.404748] [INFO ] [MAIN ] CGAU: Wiping existing primary data before full refresh. [2025-10-20T17:53:41.404614] [INFO ] [MAIN ] CFR: Starting full primary refresh. [2025-10-20T17:53:41.404878] [INFO ] [LOCAL ] CGAU: Starting deletion of primary data. [2025-10-20T17:53:41.423813] [DEBUG] [LOCAL ] CGAU: Deleted primary data file. [2025-10-20T17:53:41.446456] [INFO ] [MAIN ] CGAU: Starting full primary refresh. [2025-10-20T17:53:42.166792] [INFO ] [REMOTE ] CFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:42.126285] [DEBUG] [LOCAL ] CE: Primary data saved (51312 records). [2025-10-20T17:53:42.171638] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-01. [2025-10-20T17:53:42.172597] [INFO ] [MAIN ] CE: Full history saved. [2025-10-20T17:53:42.173382] [INFO ] [MAIN ] Update Progress: 428/2367 symbols completed. CE took 39.0s. Estimated remaining: 4h 2m 58s. Rate: 8.0 symbols/min [2025-10-20T17:53:42.359741] [INFO ] [REMOTE ] CGAU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:42.376402] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-01. [2025-10-20T17:53:42.431241] [INFO ] [MAIN ] CGV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:42.435181] [DEBUG] [LOCAL ] CGV: Loaded 1418 existing records from primary storage. [2025-10-20T17:53:42.435334] [INFO ] [MAIN ] CGV: Loaded 1418 existing records. [2025-10-20T17:53:42.612696] [WARN ] [MAIN ] CGV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:42.613079] [INFO ] [MAIN ] CGV: Wiping existing primary data before full refresh. [2025-10-20T17:53:42.613141] [INFO ] [LOCAL ] CGV: Starting deletion of primary data. [2025-10-20T17:53:42.613263] [DEBUG] [LOCAL ] CGV: Deleted primary data file. [2025-10-20T17:53:42.613301] [INFO ] [MAIN ] CGV: Starting full primary refresh. [2025-10-20T17:53:43.011257] [INFO ] [REMOTE ] CGV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:43.014433] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-01. [2025-10-20T17:53:43.016923] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-03 (2.13 seconds). [2025-10-20T17:53:43.393251] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-01 (0.38 seconds). [2025-10-20T17:53:43.451431] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-04. [2025-10-20T17:53:43.789397] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T17:53:44.251412] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-02. [2025-10-20T17:53:44.292807] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T17:53:44.578890] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T17:53:44.931268] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:53:45.051338] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-05. [2025-10-20T17:53:45.851385] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-02. [2025-10-20T17:53:46.651384] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-02. [2025-10-20T17:53:46.929974] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:53:47.084690] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T17:53:47.451390] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-06. [2025-10-20T17:53:47.765927] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T17:53:48.251389] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-03. [2025-10-20T17:53:49.051435] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-03. [2025-10-20T17:53:49.187950] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T17:53:49.537291] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T17:53:49.656095] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T17:53:49.851375] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-04. [2025-10-20T17:53:50.651430] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-07. [2025-10-20T17:53:51.280956] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T17:53:51.451380] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-05. [2025-10-20T17:53:51.927897] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T17:53:52.251469] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-03. [2025-10-20T17:53:52.824421] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:53:53.051453] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-04. [2025-10-20T17:53:53.506614] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-10-20T17:53:53.736760] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T17:53:53.851417] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-04. [2025-10-20T17:53:54.651393] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-05. [2025-10-20T17:53:55.006477] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-10-20T17:53:55.225574] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T17:53:55.451381] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-08. [2025-10-20T17:53:56.251416] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-06. [2025-10-20T17:53:56.682152] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T17:53:57.051409] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-06. [2025-10-20T17:53:57.285404] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T17:53:57.851378] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-09. [2025-10-20T17:53:58.651403] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-05. [2025-10-20T17:53:58.689670] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T17:53:59.347669] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T17:53:59.451395] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-10. [2025-10-20T17:54:00.014881] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T17:54:00.251438] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-07. [2025-10-20T17:54:01.051754] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-06. [2025-10-20T17:54:01.417942] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T17:54:01.426824] [INFO ] [REMOTE ] CFG: Full fetch complete (77760 records). [2025-10-20T17:54:01.426997] [DEBUG] [VALIDATOR ] CFG: Validating primary data. [2025-10-20T17:54:01.854800] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-07. [2025-10-20T17:54:01.879172] [DEBUG] [VALIDATOR ] CFG: All expected trading days are present. [2025-10-20T17:54:01.880956] [DEBUG] [VALIDATOR ] CFG: Primary data validation passed. [2025-10-20T17:54:01.882022] [INFO ] [MAIN ] CFG: Fetched primary data is valid. [2025-10-20T17:54:01.882116] [DEBUG] [LOCAL ] CFG: Saving primary data. [2025-10-20T17:54:01.895271] [DEBUG] [LOCAL ] CFG: Trimmed 77760 records to 53623 (keeping 200 days of data) [2025-10-20T17:54:01.970912] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T17:54:02.260826] [DEBUG] [LOCAL ] CFG: Primary data saved (53623 records). [2025-10-20T17:54:02.260961] [INFO ] [MAIN ] CFG: Full history saved. [2025-10-20T17:54:02.261038] [INFO ] [MAIN ] Update Progress: 429/2367 symbols completed. CFG took 28.8s. Estimated remaining: 4h 3m 47s. Rate: 7.9 symbols/min [2025-10-20T17:54:02.382039] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:54:02.457773] [INFO ] [MAIN ] CHCT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:02.451848] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T17:54:02.489019] [DEBUG] [LOCAL ] CHCT: Loaded 24518 existing records from primary storage. [2025-10-20T17:54:02.489411] [INFO ] [MAIN ] CHCT: Loaded 24518 existing records. [2025-10-20T17:54:02.667765] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-08. [2025-10-20T17:54:02.673085] [WARN ] [MAIN ] CHCT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:02.674377] [INFO ] [MAIN ] CHCT: Wiping existing primary data before full refresh. [2025-10-20T17:54:02.674440] [INFO ] [LOCAL ] CHCT: Starting deletion of primary data. [2025-10-20T17:54:02.675024] [DEBUG] [LOCAL ] CHCT: Deleted primary data file. [2025-10-20T17:54:02.675299] [INFO ] [MAIN ] CHCT: Starting full primary refresh. [2025-10-20T17:54:03.061713] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-10-20T17:54:03.067660] [INFO ] [REMOTE ] CHCT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:03.451491] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-07. [2025-10-20T17:54:04.251470] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-08. [2025-10-20T17:54:04.451885] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T17:54:05.051494] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-08. [2025-10-20T17:54:05.782075] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:54:05.851334] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-09. [2025-10-20T17:54:06.550799] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T17:54:06.651355] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-09. [2025-10-20T17:54:07.103795] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-10-20T17:54:07.451354] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-10. [2025-10-20T17:54:07.517154] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T17:54:07.961978] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T17:54:07.962933] [INFO ] [REMOTE ] CGV: Full fetch complete (2243 records). [2025-10-20T17:54:07.963006] [DEBUG] [VALIDATOR ] CGV: Validating primary data. [2025-10-20T17:54:08.256875] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-10. [2025-10-20T17:54:08.369084] [DEBUG] [VALIDATOR ] CGV: All expected trading days are present. [2025-10-20T17:54:08.369187] [DEBUG] [VALIDATOR ] CGV: Primary data validation passed. [2025-10-20T17:54:08.369329] [INFO ] [MAIN ] CGV: Fetched primary data is valid. [2025-10-20T17:54:08.369396] [DEBUG] [LOCAL ] CGV: Saving primary data. [2025-10-20T17:54:08.369796] [DEBUG] [LOCAL ] CGV: Trimmed 2243 records to 1384 (keeping 200 days of data) [2025-10-20T17:54:08.379250] [DEBUG] [LOCAL ] CGV: Primary data saved (1384 records). [2025-10-20T17:54:08.379391] [INFO ] [MAIN ] CGV: Full history saved. [2025-10-20T17:54:08.379480] [INFO ] [MAIN ] Update Progress: 430/2367 symbols completed. CGV took 26.2s. Estimated remaining: 4h 3m 33s. Rate: 8.0 symbols/min [2025-10-20T17:54:08.565432] [INFO ] [MAIN ] CHD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:08.632140] [DEBUG] [LOCAL ] CHD: Loaded 53066 existing records from primary storage. [2025-10-20T17:54:08.632637] [INFO ] [MAIN ] CHD: Loaded 53066 existing records. [2025-10-20T17:54:08.892897] [WARN ] [MAIN ] CHD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:08.893227] [INFO ] [MAIN ] CHD: Wiping existing primary data before full refresh. [2025-10-20T17:54:08.893305] [INFO ] [LOCAL ] CHD: Starting deletion of primary data. [2025-10-20T17:54:08.893774] [DEBUG] [LOCAL ] CHD: Deleted primary data file. [2025-10-20T17:54:08.893835] [INFO ] [MAIN ] CHD: Starting full primary refresh. [2025-10-20T17:54:09.056542] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-09. [2025-10-20T17:54:09.353310] [INFO ] [REMOTE ] CHD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:09.851443] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-01. [2025-10-20T17:54:10.233993] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T17:54:10.242345] [INFO ] [REMOTE ] CGAU: Full fetch complete (61920 records). [2025-10-20T17:54:10.242574] [DEBUG] [VALIDATOR ] CGAU: Validating primary data. [2025-10-20T17:54:10.443104] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T17:54:10.656009] [DEBUG] [VALIDATOR ] CGAU: All expected trading days are present. [2025-10-20T17:54:10.655812] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-10. [2025-10-20T17:54:10.656864] [DEBUG] [VALIDATOR ] CGAU: Primary data validation passed. [2025-10-20T17:54:10.657677] [INFO ] [MAIN ] CGAU: Fetched primary data is valid. [2025-10-20T17:54:10.657738] [DEBUG] [LOCAL ] CGAU: Saving primary data. [2025-10-20T17:54:10.659049] [DEBUG] [LOCAL ] CGAU: Trimmed 61920 records to 43818 (keeping 200 days of data) [2025-10-20T17:54:10.943947] [DEBUG] [LOCAL ] CGAU: Primary data saved (43818 records). [2025-10-20T17:54:10.944196] [INFO ] [MAIN ] CGAU: Full history saved. [2025-10-20T17:54:10.944404] [INFO ] [MAIN ] Update Progress: 431/2367 symbols completed. CGAU took 30.9s. Estimated remaining: 4h 3m 3s. Rate: 8.0 symbols/min [2025-10-20T17:54:11.125712] [INFO ] [MAIN ] CHE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:11.143670] [DEBUG] [LOCAL ] CHE: Loaded 15548 existing records from primary storage. [2025-10-20T17:54:11.144031] [INFO ] [MAIN ] CHE: Loaded 15548 existing records. [2025-10-20T17:54:11.347320] [WARN ] [MAIN ] CHE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:11.351055] [INFO ] [MAIN ] CHE: Wiping existing primary data before full refresh. [2025-10-20T17:54:11.351164] [INFO ] [LOCAL ] CHE: Starting deletion of primary data. [2025-10-20T17:54:11.350923] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:54:11.357436] [DEBUG] [LOCAL ] CHE: Deleted primary data file. [2025-10-20T17:54:11.357582] [INFO ] [MAIN ] CHE: Starting full primary refresh. [2025-10-20T17:54:11.456590] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-01. [2025-10-20T17:54:11.751130] [INFO ] [REMOTE ] CHE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:12.251344] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-02. [2025-10-20T17:54:12.653841] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T17:54:12.658688] [INFO ] [REMOTE ] CFR: Full fetch complete (48533 records). [2025-10-20T17:54:12.658807] [DEBUG] [VALIDATOR ] CFR: Validating primary data. [2025-10-20T17:54:13.046575] [DEBUG] [VALIDATOR ] CFR: All expected trading days are present. [2025-10-20T17:54:13.046676] [DEBUG] [VALIDATOR ] CFR: Primary data validation passed. [2025-10-20T17:54:13.047262] [INFO ] [MAIN ] CFR: Fetched primary data is valid. [2025-10-20T17:54:13.047333] [DEBUG] [LOCAL ] CFR: Saving primary data. [2025-10-20T17:54:13.048382] [DEBUG] [LOCAL ] CFR: Trimmed 48533 records to 32637 (keeping 200 days of data) [2025-10-20T17:54:13.102857] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-01. [2025-10-20T17:54:13.282143] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:54:13.297433] [DEBUG] [LOCAL ] CFR: Primary data saved (32637 records). [2025-10-20T17:54:13.297569] [INFO ] [MAIN ] CFR: Full history saved. [2025-10-20T17:54:13.297671] [INFO ] [MAIN ] Update Progress: 432/2367 symbols completed. CFR took 33.4s. Estimated remaining: 4h 2m 32s. Rate: 8.0 symbols/min [2025-10-20T17:54:13.474875] [INFO ] [MAIN ] CHGG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:13.518762] [DEBUG] [LOCAL ] CHGG: Loaded 48419 existing records from primary storage. [2025-10-20T17:54:13.519056] [INFO ] [MAIN ] CHGG: Loaded 48419 existing records. [2025-10-20T17:54:13.693020] [WARN ] [MAIN ] CHGG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:13.693233] [INFO ] [MAIN ] CHGG: Wiping existing primary data before full refresh. [2025-10-20T17:54:13.693282] [INFO ] [LOCAL ] CHGG: Starting deletion of primary data. [2025-10-20T17:54:13.693539] [DEBUG] [LOCAL ] CHGG: Deleted primary data file. [2025-10-20T17:54:13.693596] [INFO ] [MAIN ] CHGG: Starting full primary refresh. [2025-10-20T17:54:13.851736] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-02. [2025-10-20T17:54:14.076058] [INFO ] [REMOTE ] CHGG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:14.091859] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T17:54:14.313293] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:54:14.651466] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-01. [2025-10-20T17:54:15.452271] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-03. [2025-10-20T17:54:15.457688] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T17:54:16.251351] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-03. [2025-10-20T17:54:16.454384] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T17:54:16.546572] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T17:54:17.051428] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-02. [2025-10-20T17:54:17.851361] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-04. [2025-10-20T17:54:18.014035] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T17:54:18.651424] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-04. [2025-10-20T17:54:18.797924] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T17:54:19.102954] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T17:54:19.451456] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-03. [2025-10-20T17:54:20.251365] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-02. [2025-10-20T17:54:20.318173] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T17:54:20.901499] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:54:21.001817] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:54:21.051372] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-03. [2025-10-20T17:54:21.851406] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-04. [2025-10-20T17:54:21.929221] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T17:54:22.651414] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-04. [2025-10-20T17:54:23.451383] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-05. [2025-10-20T17:54:23.490851] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:54:23.720906] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T17:54:24.251378] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-05. [2025-10-20T17:54:24.963061] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:54:25.051394] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-05. [2025-10-20T17:54:25.074839] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T17:54:25.851412] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-06. [2025-10-20T17:54:26.479915] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T17:54:26.651382] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-05. [2025-10-20T17:54:27.348832] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:54:27.451387] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-06. [2025-10-20T17:54:27.745464] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T17:54:28.151074] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T17:54:28.251452] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-07. [2025-10-20T17:54:29.051444] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-07. [2025-10-20T17:54:29.563619] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T17:54:29.851406] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-06. [2025-10-20T17:54:30.651396] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-08. [2025-10-20T17:54:30.831677] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T17:54:30.833962] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T17:54:31.451375] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-06. [2025-10-20T17:54:31.541568] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T17:54:32.251400] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-07. [2025-10-20T17:54:33.051354] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-09. [2025-10-20T17:54:33.341745] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T17:54:33.434355] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-10-20T17:54:33.851409] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-07. [2025-10-20T17:54:34.087916] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:54:34.651493] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-08. [2025-10-20T17:54:35.451455] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-08. [2025-10-20T17:54:35.486836] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T17:54:35.598099] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T17:54:36.251369] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-09. [2025-10-20T17:54:36.970712] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:54:37.051410] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-09. [2025-10-20T17:54:37.330434] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T17:54:37.851418] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-10. [2025-10-20T17:54:38.651376] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-10. [2025-10-20T17:54:38.889660] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T17:54:39.000454] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:54:39.003491] [INFO ] [REMOTE ] CHE: Full fetch complete (20409 records). [2025-10-20T17:54:39.003804] [DEBUG] [VALIDATOR ] CHE: Validating primary data. [2025-10-20T17:54:39.369801] [DEBUG] [VALIDATOR ] CHE: All expected trading days are present. [2025-10-20T17:54:39.369891] [DEBUG] [VALIDATOR ] CHE: Primary data validation passed. [2025-10-20T17:54:39.370298] [INFO ] [MAIN ] CHE: Fetched primary data is valid. [2025-10-20T17:54:39.370338] [DEBUG] [LOCAL ] CHE: Saving primary data. [2025-10-20T17:54:39.370919] [DEBUG] [LOCAL ] CHE: Trimmed 20409 records to 15304 (keeping 200 days of data) [2025-10-20T17:54:39.451722] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-10. [2025-10-20T17:54:39.465179] [DEBUG] [LOCAL ] CHE: Primary data saved (15304 records). [2025-10-20T17:54:39.466710] [INFO ] [MAIN ] CHE: Full history saved. [2025-10-20T17:54:39.466817] [INFO ] [MAIN ] Update Progress: 433/2367 symbols completed. CHE took 28.5s. Estimated remaining: 4h 3m 48s. Rate: 7.9 symbols/min [2025-10-20T17:54:39.638828] [INFO ] [MAIN ] CHH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:39.669387] [DEBUG] [LOCAL ] CHH: Loaded 32905 existing records from primary storage. [2025-10-20T17:54:39.669698] [INFO ] [MAIN ] CHH: Loaded 32905 existing records. [2025-10-20T17:54:39.844162] [WARN ] [MAIN ] CHH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:39.844362] [INFO ] [MAIN ] CHH: Wiping existing primary data before full refresh. [2025-10-20T17:54:39.844449] [INFO ] [LOCAL ] CHH: Starting deletion of primary data. [2025-10-20T17:54:39.844743] [DEBUG] [LOCAL ] CHH: Deleted primary data file. [2025-10-20T17:54:39.844806] [INFO ] [MAIN ] CHH: Starting full primary refresh. [2025-10-20T17:54:40.201900] [INFO ] [REMOTE ] CHH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:40.251346] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-01. [2025-10-20T17:54:40.275288] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T17:54:40.279531] [INFO ] [REMOTE ] CHCT: Full fetch complete (34788 records). [2025-10-20T17:54:40.279609] [DEBUG] [VALIDATOR ] CHCT: Validating primary data. [2025-10-20T17:54:40.652203] [DEBUG] [VALIDATOR ] CHCT: All expected trading days are present. [2025-10-20T17:54:40.652289] [DEBUG] [VALIDATOR ] CHCT: Primary data validation passed. [2025-10-20T17:54:40.652890] [INFO ] [MAIN ] CHCT: Fetched primary data is valid. [2025-10-20T17:54:40.652952] [DEBUG] [LOCAL ] CHCT: Saving primary data. [2025-10-20T17:54:40.653814] [DEBUG] [LOCAL ] CHCT: Trimmed 34788 records to 24343 (keeping 200 days of data) [2025-10-20T17:54:40.799417] [DEBUG] [LOCAL ] CHCT: Primary data saved (24343 records). [2025-10-20T17:54:40.799560] [INFO ] [MAIN ] CHCT: Full history saved. [2025-10-20T17:54:40.799661] [INFO ] [MAIN ] Update Progress: 434/2367 symbols completed. CHCT took 38.5s. Estimated remaining: 4h 3m 13s. Rate: 7.9 symbols/min [2025-10-20T17:54:40.973719] [INFO ] [MAIN ] CHMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:40.994714] [DEBUG] [LOCAL ] CHMI: Loaded 21855 existing records from primary storage. [2025-10-20T17:54:40.994965] [INFO ] [MAIN ] CHMI: Loaded 21855 existing records. [2025-10-20T17:54:41.059311] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-08. [2025-10-20T17:54:41.181780] [WARN ] [MAIN ] CHMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:41.181982] [INFO ] [MAIN ] CHMI: Wiping existing primary data before full refresh. [2025-10-20T17:54:41.182028] [INFO ] [LOCAL ] CHMI: Starting deletion of primary data. [2025-10-20T17:54:41.182195] [DEBUG] [LOCAL ] CHMI: Deleted primary data file. [2025-10-20T17:54:41.182232] [INFO ] [MAIN ] CHMI: Starting full primary refresh. [2025-10-20T17:54:41.598832] [INFO ] [REMOTE ] CHMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:41.603676] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T17:54:41.616618] [INFO ] [REMOTE ] CHD: Full fetch complete (74954 records). [2025-10-20T17:54:41.616909] [DEBUG] [VALIDATOR ] CHD: Validating primary data. [2025-10-20T17:54:41.747277] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:54:41.855383] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-02. [2025-10-20T17:54:42.096870] [DEBUG] [VALIDATOR ] CHD: All expected trading days are present. [2025-10-20T17:54:42.096988] [DEBUG] [VALIDATOR ] CHD: Primary data validation passed. [2025-10-20T17:54:42.098019] [INFO ] [MAIN ] CHD: Fetched primary data is valid. [2025-10-20T17:54:42.098092] [DEBUG] [LOCAL ] CHD: Saving primary data. [2025-10-20T17:54:42.100140] [DEBUG] [LOCAL ] CHD: Trimmed 74954 records to 52329 (keeping 200 days of data) [2025-10-20T17:54:42.443682] [DEBUG] [LOCAL ] CHD: Primary data saved (52329 records). [2025-10-20T17:54:42.443886] [INFO ] [MAIN ] CHD: Full history saved. [2025-10-20T17:54:42.444306] [INFO ] [MAIN ] Update Progress: 435/2367 symbols completed. CHD took 34.1s. Estimated remaining: 4h 2m 39s. Rate: 8.0 symbols/min [2025-10-20T17:54:42.630785] [INFO ] [MAIN ] CHN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:42.640980] [DEBUG] [LOCAL ] CHN: Loaded 5065 existing records from primary storage. [2025-10-20T17:54:42.641286] [INFO ] [MAIN ] CHN: Loaded 5065 existing records. [2025-10-20T17:54:42.657717] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-01. [2025-10-20T17:54:42.845949] [WARN ] [MAIN ] CHN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:42.846295] [INFO ] [MAIN ] CHN: Wiping existing primary data before full refresh. [2025-10-20T17:54:42.846355] [INFO ] [LOCAL ] CHN: Starting deletion of primary data. [2025-10-20T17:54:42.846502] [DEBUG] [LOCAL ] CHN: Deleted primary data file. [2025-10-20T17:54:42.846539] [INFO ] [MAIN ] CHN: Starting full primary refresh. [2025-10-20T17:54:43.193747] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T17:54:43.317068] [INFO ] [REMOTE ] CHN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:43.320785] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T17:54:43.451388] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-03. [2025-10-20T17:54:43.680120] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T17:54:44.251398] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-02. [2025-10-20T17:54:44.838265] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:54:44.897586] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:54:45.051360] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-04. [2025-10-20T17:54:45.851344] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-01. [2025-10-20T17:54:46.651361] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-03. [2025-10-20T17:54:46.653384] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T17:54:46.663726] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T17:54:47.451391] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-02. [2025-10-20T17:54:47.830010] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T17:54:47.968832] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T17:54:48.251451] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-09. [2025-10-20T17:54:49.051424] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-05. [2025-10-20T17:54:49.851380] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-03. [2025-10-20T17:54:50.109880] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T17:54:50.192994] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:54:50.304007] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T17:54:50.651376] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-06. [2025-10-20T17:54:51.451412] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-10. [2025-10-20T17:54:52.071792] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T17:54:52.251429] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-04. [2025-10-20T17:54:52.704544] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T17:54:53.051442] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-07. [2025-10-20T17:54:53.176918] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:54:53.183965] [INFO ] [REMOTE ] CHGG: Full fetch complete (70163 records). [2025-10-20T17:54:53.184076] [DEBUG] [VALIDATOR ] CHGG: Validating primary data. [2025-10-20T17:54:53.609503] [DEBUG] [VALIDATOR ] CHGG: All expected trading days are present. [2025-10-20T17:54:53.609602] [DEBUG] [VALIDATOR ] CHGG: Primary data validation passed. [2025-10-20T17:54:53.610542] [INFO ] [MAIN ] CHGG: Fetched primary data is valid. [2025-10-20T17:54:53.610623] [DEBUG] [LOCAL ] CHGG: Saving primary data. [2025-10-20T17:54:53.612257] [DEBUG] [LOCAL ] CHGG: Trimmed 70163 records to 47673 (keeping 200 days of data) [2025-10-20T17:54:53.852854] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-04. [2025-10-20T17:54:53.916539] [DEBUG] [LOCAL ] CHGG: Primary data saved (47673 records). [2025-10-20T17:54:53.916722] [INFO ] [MAIN ] CHGG: Full history saved. [2025-10-20T17:54:53.916821] [INFO ] [MAIN ] Update Progress: 436/2367 symbols completed. CHGG took 40.6s. Estimated remaining: 4h 2m 49s. Rate: 8.0 symbols/min [2025-10-20T17:54:54.087791] [INFO ] [MAIN ] CHPT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:54.132817] [DEBUG] [LOCAL ] CHPT: Loaded 48210 existing records from primary storage. [2025-10-20T17:54:54.133152] [INFO ] [MAIN ] CHPT: Loaded 48210 existing records. [2025-10-20T17:54:54.312765] [WARN ] [MAIN ] CHPT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:54.313274] [INFO ] [MAIN ] CHPT: Wiping existing primary data before full refresh. [2025-10-20T17:54:54.313388] [INFO ] [LOCAL ] CHPT: Starting deletion of primary data. [2025-10-20T17:54:54.313727] [DEBUG] [LOCAL ] CHPT: Deleted primary data file. [2025-10-20T17:54:54.313892] [INFO ] [MAIN ] CHPT: Starting full primary refresh. [2025-10-20T17:54:54.615500] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T17:54:54.656516] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-05. [2025-10-20T17:54:54.713818] [INFO ] [REMOTE ] CHPT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:55.245633] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T17:54:55.405028] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:54:55.451408] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-05. [2025-10-20T17:54:56.249203] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T17:54:56.254041] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-08. [2025-10-20T17:54:57.051449] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-01. [2025-10-20T17:54:57.549795] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T17:54:57.851445] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-06. [2025-10-20T17:54:58.554664] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T17:54:58.651469] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-07. [2025-10-20T17:54:59.052515] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:54:59.223133] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T17:54:59.451390] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-08. [2025-10-20T17:54:59.890842] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T17:55:00.251412] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-09. [2025-10-20T17:55:00.920592] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T17:55:01.051450] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-02. [2025-10-20T17:55:01.851470] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-09. [2025-10-20T17:55:02.651450] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-06. [2025-10-20T17:55:03.364769] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T17:55:03.441071] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-02 (2.39 seconds). [2025-10-20T17:55:03.451332] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-03. [2025-10-20T17:55:03.757830] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T17:55:04.251383] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-10. [2025-10-20T17:55:05.051471] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-10. [2025-10-20T17:55:05.544545] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T17:55:05.604985] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T17:55:05.610691] [INFO ] [REMOTE ] CHH: Full fetch complete (47486 records). [2025-10-20T17:55:05.610796] [DEBUG] [VALIDATOR ] CHH: Validating primary data. [2025-10-20T17:55:05.805192] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T17:55:05.823621] [INFO ] [REMOTE ] CHN: Full fetch complete (7551 records). [2025-10-20T17:55:05.856707] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-04. [2025-10-20T17:55:05.862105] [DEBUG] [VALIDATOR ] CHN: Validating primary data. [2025-10-20T17:55:06.215944] [DEBUG] [VALIDATOR ] CHH: All expected trading days are present. [2025-10-20T17:55:06.231264] [DEBUG] [VALIDATOR ] CHH: Primary data validation passed. [2025-10-20T17:55:06.287494] [INFO ] [MAIN ] CHH: Fetched primary data is valid. [2025-10-20T17:55:06.311571] [DEBUG] [LOCAL ] CHH: Saving primary data. [2025-10-20T17:55:06.362170] [DEBUG] [LOCAL ] CHH: Trimmed 47486 records to 32329 (keeping 200 days of data) [2025-10-20T17:55:06.494225] [DEBUG] [VALIDATOR ] CHN: All expected trading days are present. [2025-10-20T17:55:06.519073] [DEBUG] [VALIDATOR ] CHN: Primary data validation passed. [2025-10-20T17:55:06.519560] [INFO ] [MAIN ] CHN: Fetched primary data is valid. [2025-10-20T17:55:06.520795] [DEBUG] [LOCAL ] CHN: Saving primary data. [2025-10-20T17:55:06.527509] [DEBUG] [LOCAL ] CHN: Trimmed 7551 records to 5002 (keeping 200 days of data) [2025-10-20T17:55:06.584531] [DEBUG] [LOCAL ] CHN: Primary data saved (5002 records). [2025-10-20T17:55:06.586969] [INFO ] [MAIN ] CHN: Full history saved. [2025-10-20T17:55:06.589680] [INFO ] [MAIN ] Update Progress: 437/2367 symbols completed. CHN took 24.1s. Estimated remaining: 4h 3m 4s. Rate: 7.9 symbols/min [2025-10-20T17:55:06.652190] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-07. [2025-10-20T17:55:06.666811] [DEBUG] [LOCAL ] CHH: Primary data saved (32329 records). [2025-10-20T17:55:06.666896] [INFO ] [MAIN ] CHH: Full history saved. [2025-10-20T17:55:06.666968] [INFO ] [MAIN ] Update Progress: 438/2367 symbols completed. CHH took 27.2s. Estimated remaining: 4h 2m 24s. Rate: 8.0 symbols/min [2025-10-20T17:55:07.832897] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T17:55:07.855877] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-05. [2025-10-20T17:55:07.856870] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T17:55:08.251420] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-08. [2025-10-20T17:55:08.838929] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T17:55:09.051466] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-09. [2025-10-20T17:55:10.165720] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-05 (2.31 seconds). [2025-10-20T17:55:10.178109] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-06. [2025-10-20T17:55:10.209782] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T17:55:10.651415] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-10. [2025-10-20T17:55:12.031795] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T17:55:12.035143] [INFO ] [REMOTE ] CHMI: Full fetch complete (31632 records). [2025-10-20T17:55:12.035213] [DEBUG] [VALIDATOR ] CHMI: Validating primary data. [2025-10-20T17:55:12.399808] [DEBUG] [VALIDATOR ] CHMI: All expected trading days are present. [2025-10-20T17:55:12.399897] [DEBUG] [VALIDATOR ] CHMI: Primary data validation passed. [2025-10-20T17:55:12.400384] [INFO ] [MAIN ] CHMI: Fetched primary data is valid. [2025-10-20T17:55:12.400423] [DEBUG] [LOCAL ] CHMI: Saving primary data. [2025-10-20T17:55:12.401129] [DEBUG] [LOCAL ] CHMI: Trimmed 31632 records to 21466 (keeping 200 days of data) [2025-10-20T17:55:12.519211] [DEBUG] [LOCAL ] CHMI: Primary data saved (21466 records). [2025-10-20T17:55:12.519327] [INFO ] [MAIN ] CHMI: Full history saved. [2025-10-20T17:55:12.519397] [INFO ] [MAIN ] Update Progress: 439/2367 symbols completed. CHMI took 31.7s. Estimated remaining: 4h 2m 9s. Rate: 8.0 symbols/min [2025-10-20T17:55:12.900697] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-06 (2.72 seconds). [2025-10-20T17:55:12.911366] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-07. [2025-10-20T17:55:14.984917] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T17:55:14.996452] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-08. [2025-10-20T17:55:16.521377] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:55:16.530352] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-09. [2025-10-20T17:55:18.011618] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T17:55:18.020127] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-10. [2025-10-20T17:55:19.715711] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T17:55:19.721686] [INFO ] [REMOTE ] CHPT: Full fetch complete (71419 records). [2025-10-20T17:55:19.721798] [DEBUG] [VALIDATOR ] CHPT: Validating primary data. [2025-10-20T17:55:20.119812] [DEBUG] [VALIDATOR ] CHPT: All expected trading days are present. [2025-10-20T17:55:20.119921] [DEBUG] [VALIDATOR ] CHPT: Primary data validation passed. [2025-10-20T17:55:20.120731] [INFO ] [MAIN ] CHPT: Fetched primary data is valid. [2025-10-20T17:55:20.120793] [DEBUG] [LOCAL ] CHPT: Saving primary data. [2025-10-20T17:55:20.122193] [DEBUG] [LOCAL ] CHPT: Trimmed 71419 records to 47293 (keeping 200 days of data) [2025-10-20T17:55:20.385283] [DEBUG] [LOCAL ] CHPT: Primary data saved (47293 records). [2025-10-20T17:55:20.385412] [INFO ] [MAIN ] CHPT: Full history saved. [2025-10-20T17:55:20.385499] [INFO ] [MAIN ] Update Progress: 440/2367 symbols completed. CHPT took 26.5s. Estimated remaining: 4h 2m 2s. Rate: 8.0 symbols/min [2025-10-20T17:55:21.010268] [INFO ] [MAIN ] CIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:21.024510] [INFO ] [MAIN ] CHT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:21.088732] [INFO ] [MAIN ] CI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:21.092734] [INFO ] [MAIN ] CHWY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:21.115564] [DEBUG] [LOCAL ] CIA: Loaded 16820 existing records from primary storage. [2025-10-20T17:55:21.116184] [INFO ] [MAIN ] CIA: Loaded 16820 existing records. [2025-10-20T17:55:21.192416] [DEBUG] [LOCAL ] CHT: Loaded 19756 existing records from primary storage. [2025-10-20T17:55:21.215899] [INFO ] [MAIN ] CHT: Loaded 19756 existing records. [2025-10-20T17:55:21.408355] [WARN ] [MAIN ] CIA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:21.443840] [INFO ] [MAIN ] CIA: Wiping existing primary data before full refresh. [2025-10-20T17:55:21.470197] [INFO ] [LOCAL ] CIA: Starting deletion of primary data. [2025-10-20T17:55:21.468828] [DEBUG] [LOCAL ] CI: Loaded 50399 existing records from primary storage. [2025-10-20T17:55:21.486042] [INFO ] [MAIN ] CI: Loaded 50399 existing records. [2025-10-20T17:55:21.539013] [DEBUG] [LOCAL ] CHWY: Loaded 54435 existing records from primary storage. [2025-10-20T17:55:21.538234] [DEBUG] [LOCAL ] CIA: Deleted primary data file. [2025-10-20T17:55:21.544382] [INFO ] [MAIN ] CIA: Starting full primary refresh. [2025-10-20T17:55:21.562480] [INFO ] [MAIN ] CHWY: Loaded 54435 existing records. [2025-10-20T17:55:21.645718] [WARN ] [MAIN ] CHT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:21.781324] [INFO ] [MAIN ] CHT: Wiping existing primary data before full refresh. [2025-10-20T17:55:21.800103] [INFO ] [LOCAL ] CHT: Starting deletion of primary data. [2025-10-20T17:55:21.851767] [DEBUG] [LOCAL ] CHT: Deleted primary data file. [2025-10-20T17:55:21.855013] [INFO ] [MAIN ] CHT: Starting full primary refresh. [2025-10-20T17:55:22.066317] [WARN ] [MAIN ] CI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:22.134204] [INFO ] [MAIN ] CI: Wiping existing primary data before full refresh. [2025-10-20T17:55:22.144586] [INFO ] [LOCAL ] CI: Starting deletion of primary data. [2025-10-20T17:55:22.148954] [WARN ] [MAIN ] CHWY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:22.188196] [DEBUG] [LOCAL ] CI: Deleted primary data file. [2025-10-20T17:55:22.181263] [INFO ] [MAIN ] CHWY: Wiping existing primary data before full refresh. [2025-10-20T17:55:22.239841] [INFO ] [MAIN ] CI: Starting full primary refresh. [2025-10-20T17:55:22.234636] [INFO ] [LOCAL ] CHWY: Starting deletion of primary data. [2025-10-20T17:55:22.282366] [DEBUG] [LOCAL ] CHWY: Deleted primary data file. [2025-10-20T17:55:22.419668] [INFO ] [MAIN ] CHWY: Starting full primary refresh. [2025-10-20T17:55:22.691948] [INFO ] [REMOTE ] CIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:22.800585] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-01. [2025-10-20T17:55:23.225180] [INFO ] [REMOTE ] CHT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:23.262634] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-01. [2025-10-20T17:55:23.334894] [INFO ] [REMOTE ] CI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:23.347407] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-01. [2025-10-20T17:55:23.424340] [INFO ] [REMOTE ] CHWY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:23.425119] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-01. [2025-10-20T17:55:23.769423] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T17:55:23.774357] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-02. [2025-10-20T17:55:24.354423] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T17:55:24.358984] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-03. [2025-10-20T17:55:24.412521] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T17:55:24.416283] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-02. [2025-10-20T17:55:24.783386] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:55:24.793463] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-02. [2025-10-20T17:55:25.024710] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:55:25.028730] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-03. [2025-10-20T17:55:25.369972] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:55:25.373969] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-04. [2025-10-20T17:55:25.536931] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-10-20T17:55:25.547397] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-02. [2025-10-20T17:55:25.940953] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T17:55:25.946907] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-04. [2025-10-20T17:55:25.961090] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T17:55:25.965446] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-05. [2025-10-20T17:55:26.216813] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:55:26.230548] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-03. [2025-10-20T17:55:26.612790] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T17:55:26.616324] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-06. [2025-10-20T17:55:26.961985] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T17:55:26.972638] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-03. [2025-10-20T17:55:27.050635] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T17:55:27.451355] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-05. [2025-10-20T17:55:27.593788] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T17:55:28.003847] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:55:28.251461] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-07. [2025-10-20T17:55:28.536738] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T17:55:28.988548] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T17:55:29.012768] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T17:55:29.051329] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-08. [2025-10-20T17:55:29.851400] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-04. [2025-10-20T17:55:30.013527] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:55:30.651373] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-06. [2025-10-20T17:55:31.422533] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T17:55:31.451327] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-05. [2025-10-20T17:55:31.870399] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T17:55:32.251390] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-07. [2025-10-20T17:55:33.051388] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-09. [2025-10-20T17:55:33.191616] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:55:33.339361] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T17:55:33.851385] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-08. [2025-10-20T17:55:34.118603] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T17:55:34.651423] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-06. [2025-10-20T17:55:34.704632] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T17:55:35.451404] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-04. [2025-10-20T17:55:36.251443] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-09. [2025-10-20T17:55:36.358897] [DEBUG] [REMOTE ] CHT: Error fetching primary data for 2025-09: 'API call returned an empty response.' [2025-10-20T17:55:36.359229] [WARN ] [REMOTE ] CHT: No data returned for month 2025-09 [2025-10-20T17:55:36.359324] [FATAL] [REMOTE ] CHT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:55:36.359400] [FATAL] [MAIN ] CHT: Failed to fetch primary data. [2025-10-20T17:55:36.359462] [WARN ] [MAIN ] CHT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:55:36.359757] [INFO ] [LOCAL ] CHT: Starting deletion of primary data. [2025-10-20T17:55:36.359856] [DEBUG] [LOCAL ] CHT: No primary data file found to delete. [2025-10-20T17:55:36.359920] [INFO ] [LOCAL ] CHT: Starting deletion of derivative data. [2025-10-20T17:55:36.360129] [DEBUG] [LOCAL ] CHT: Deleted derivative data file. [2025-10-20T17:55:36.360214] [INFO ] [MAIN ] Update Progress: 441/2367 symbols completed. CHT took 16.0s. Estimated remaining: 4h 2m 32s. Rate: 7.9 symbols/min [2025-10-20T17:55:36.585676] [INFO ] [MAIN ] CIB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:36.623543] [DEBUG] [LOCAL ] CIB: Loaded 30952 existing records from primary storage. [2025-10-20T17:55:36.623992] [INFO ] [MAIN ] CIB: Loaded 30952 existing records. [2025-10-20T17:55:36.871959] [WARN ] [MAIN ] CIB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:36.872333] [INFO ] [MAIN ] CIB: Wiping existing primary data before full refresh. [2025-10-20T17:55:36.872520] [INFO ] [LOCAL ] CIB: Starting deletion of primary data. [2025-10-20T17:55:36.872827] [DEBUG] [LOCAL ] CIB: Deleted primary data file. [2025-10-20T17:55:36.872898] [INFO ] [MAIN ] CIB: Starting full primary refresh. [2025-10-20T17:55:37.056946] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-10. [2025-10-20T17:55:37.095142] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-06 (2.44 seconds). [2025-10-20T17:55:37.438372] [INFO ] [REMOTE ] CIB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:37.470543] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T17:55:37.851414] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-05. [2025-10-20T17:55:38.091729] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T17:55:38.095147] [INFO ] [REMOTE ] CIA: Full fetch complete (23370 records). [2025-10-20T17:55:38.095274] [DEBUG] [VALIDATOR ] CIA: Validating primary data. [2025-10-20T17:55:38.538026] [DEBUG] [VALIDATOR ] CIA: All expected trading days are present. [2025-10-20T17:55:38.538139] [DEBUG] [VALIDATOR ] CIA: Primary data validation passed. [2025-10-20T17:55:38.538983] [INFO ] [MAIN ] CIA: Fetched primary data is valid. [2025-10-20T17:55:38.539050] [DEBUG] [LOCAL ] CIA: Saving primary data. [2025-10-20T17:55:38.540097] [DEBUG] [LOCAL ] CIA: Trimmed 23370 records to 16550 (keeping 200 days of data) [2025-10-20T17:55:38.641390] [DEBUG] [LOCAL ] CIA: Primary data saved (16550 records). [2025-10-20T17:55:38.641541] [INFO ] [MAIN ] CIA: Full history saved. [2025-10-20T17:55:38.641854] [INFO ] [MAIN ] Update Progress: 442/2367 symbols completed. CIA took 18.3s. Estimated remaining: 4h 2m 1s. Rate: 8.0 symbols/min [2025-10-20T17:55:38.656618] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-01. [2025-10-20T17:55:38.846619] [INFO ] [MAIN ] CICB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:38.846869] [DEBUG] [LOCAL ] CICB: No existing data file found in primary storage. [2025-10-20T17:55:38.846914] [WARN ] [MAIN ] CICB: No existing data found, performing full refresh instead. [2025-10-20T17:55:38.847034] [INFO ] [MAIN ] CICB: Wiping existing primary data before full refresh. [2025-10-20T17:55:38.847076] [INFO ] [LOCAL ] CICB: Starting deletion of primary data. [2025-10-20T17:55:38.847119] [DEBUG] [LOCAL ] CICB: No primary data file found to delete. [2025-10-20T17:55:38.847153] [INFO ] [MAIN ] CICB: Starting full primary refresh. [2025-10-20T17:55:39.234268] [INFO ] [REMOTE ] CICB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:39.451366] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-01. [2025-10-20T17:55:39.727789] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T17:55:39.900402] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T17:55:40.178771] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T17:55:40.251466] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-02. [2025-10-20T17:55:41.051452] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-07. [2025-10-20T17:55:41.497459] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:55:41.851447] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-03. [2025-10-20T17:55:42.651442] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-06. [2025-10-20T17:55:42.854579] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T17:55:43.120694] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T17:55:43.451397] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-02. [2025-10-20T17:55:43.720128] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-02 (0.27 seconds). [2025-10-20T17:55:43.906319] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T17:55:44.251366] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-03. [2025-10-20T17:55:44.650070] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T17:55:45.051415] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-04. [2025-10-20T17:55:45.851454] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-07. [2025-10-20T17:55:46.654875] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-08. [2025-10-20T17:55:46.665261] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:55:47.451390] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-05. [2025-10-20T17:55:47.742303] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T17:55:48.251410] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-08. [2025-10-20T17:55:48.520635] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:55:48.562831] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:55:49.051420] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-09. [2025-10-20T17:55:49.851449] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-06. [2025-10-20T17:55:49.993618] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:55:50.651431] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-09. [2025-10-20T17:55:50.811165] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T17:55:51.038058] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T17:55:51.451377] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-04. [2025-10-20T17:55:51.869530] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T17:55:52.251373] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-07. [2025-10-20T17:55:52.394979] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:55:53.040054] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T17:55:53.051345] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-08. [2025-10-20T17:55:53.851436] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-10. [2025-10-20T17:55:54.362474] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:55:54.488269] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T17:55:54.496488] [INFO ] [REMOTE ] CI: Full fetch complete (72106 records). [2025-10-20T17:55:54.496764] [DEBUG] [VALIDATOR ] CI: Validating primary data. [2025-10-20T17:55:54.656578] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-09. [2025-10-20T17:55:54.911289] [DEBUG] [VALIDATOR ] CI: All expected trading days are present. [2025-10-20T17:55:54.911386] [DEBUG] [VALIDATOR ] CI: Primary data validation passed. [2025-10-20T17:55:54.912228] [INFO ] [MAIN ] CI: Fetched primary data is valid. [2025-10-20T17:55:54.912298] [DEBUG] [LOCAL ] CI: Saving primary data. [2025-10-20T17:55:54.913738] [DEBUG] [LOCAL ] CI: Trimmed 72106 records to 49691 (keeping 200 days of data) [2025-10-20T17:55:55.214398] [DEBUG] [LOCAL ] CI: Primary data saved (49691 records). [2025-10-20T17:55:55.214530] [INFO ] [MAIN ] CI: Full history saved. [2025-10-20T17:55:55.214608] [INFO ] [MAIN ] Update Progress: 443/2367 symbols completed. CI took 34.8s. Estimated remaining: 4h 2m 33s. Rate: 7.9 symbols/min [2025-10-20T17:55:55.380200] [INFO ] [MAIN ] CIEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:55.426675] [DEBUG] [LOCAL ] CIEN: Loaded 52910 existing records from primary storage. [2025-10-20T17:55:55.426975] [INFO ] [MAIN ] CIEN: Loaded 52910 existing records. [2025-10-20T17:55:55.457601] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-05. [2025-10-20T17:55:55.604161] [WARN ] [MAIN ] CIEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:55.604358] [INFO ] [MAIN ] CIEN: Wiping existing primary data before full refresh. [2025-10-20T17:55:55.604405] [INFO ] [LOCAL ] CIEN: Starting deletion of primary data. [2025-10-20T17:55:55.604701] [DEBUG] [LOCAL ] CIEN: Deleted primary data file. [2025-10-20T17:55:55.604757] [INFO ] [MAIN ] CIEN: Starting full primary refresh. [2025-10-20T17:55:55.893034] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T17:55:55.976273] [INFO ] [REMOTE ] CIEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:56.026607] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T17:55:56.251431] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-01. [2025-10-20T17:55:57.051414] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-10. [2025-10-20T17:55:57.851376] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-06. [2025-10-20T17:55:57.915524] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:55:58.336746] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T17:55:58.651452] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-10. [2025-10-20T17:55:59.115945] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T17:55:59.124533] [INFO ] [REMOTE ] CHWY: Full fetch complete (77781 records). [2025-10-20T17:55:59.124722] [DEBUG] [VALIDATOR ] CHWY: Validating primary data. [2025-10-20T17:55:59.464758] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-07. [2025-10-20T17:55:59.526528] [DEBUG] [VALIDATOR ] CHWY: All expected trading days are present. [2025-10-20T17:55:59.526717] [DEBUG] [VALIDATOR ] CHWY: Primary data validation passed. [2025-10-20T17:55:59.527706] [INFO ] [MAIN ] CHWY: Fetched primary data is valid. [2025-10-20T17:55:59.527845] [DEBUG] [LOCAL ] CHWY: Saving primary data. [2025-10-20T17:55:59.529834] [DEBUG] [LOCAL ] CHWY: Trimmed 77781 records to 53656 (keeping 200 days of data) [2025-10-20T17:55:59.651956] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:55:59.661872] [INFO ] [REMOTE ] CIB: Full fetch complete (45339 records). [2025-10-20T17:55:59.662876] [DEBUG] [VALIDATOR ] CIB: Validating primary data. [2025-10-20T17:55:59.911691] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T17:56:00.231821] [DEBUG] [VALIDATOR ] CIB: All expected trading days are present. [2025-10-20T17:56:00.232246] [DEBUG] [VALIDATOR ] CIB: Primary data validation passed. [2025-10-20T17:56:00.233572] [INFO ] [MAIN ] CIB: Fetched primary data is valid. [2025-10-20T17:56:00.233910] [DEBUG] [LOCAL ] CIB: Saving primary data. [2025-10-20T17:56:00.240044] [DEBUG] [LOCAL ] CIB: Trimmed 45339 records to 30372 (keeping 200 days of data) [2025-10-20T17:56:00.294701] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-02. [2025-10-20T17:56:00.454888] [DEBUG] [LOCAL ] CHWY: Primary data saved (53656 records). [2025-10-20T17:56:00.456898] [INFO ] [MAIN ] CHWY: Full history saved. [2025-10-20T17:56:00.457886] [INFO ] [MAIN ] Update Progress: 444/2367 symbols completed. CHWY took 40.1s. Estimated remaining: 4h 2m 15s. Rate: 7.9 symbols/min [2025-10-20T17:56:00.674827] [DEBUG] [LOCAL ] CIB: Primary data saved (30372 records). [2025-10-20T17:56:00.697964] [INFO ] [MAIN ] CIB: Full history saved. [2025-10-20T17:56:00.701418] [INFO ] [MAIN ] Update Progress: 445/2367 symbols completed. CIB took 24.3s. Estimated remaining: 4h 1m 36s. Rate: 8.0 symbols/min [2025-10-20T17:56:00.701058] [INFO ] [MAIN ] CIF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:00.816118] [DEBUG] [LOCAL ] CIF: Loaded 4087 existing records from primary storage. [2025-10-20T17:56:00.837181] [INFO ] [MAIN ] CIF: Loaded 4087 existing records. [2025-10-20T17:56:00.987955] [INFO ] [MAIN ] CIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:01.056545] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-08. [2025-10-20T17:56:01.073041] [WARN ] [MAIN ] CIF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:01.073462] [INFO ] [MAIN ] CIF: Wiping existing primary data before full refresh. [2025-10-20T17:56:01.073534] [INFO ] [LOCAL ] CIF: Starting deletion of primary data. [2025-10-20T17:56:01.073750] [DEBUG] [LOCAL ] CIF: Deleted primary data file. [2025-10-20T17:56:01.073808] [INFO ] [MAIN ] CIF: Starting full primary refresh. [2025-10-20T17:56:01.322273] [DEBUG] [LOCAL ] CIG: Loaded 45005 existing records from primary storage. [2025-10-20T17:56:01.355732] [INFO ] [MAIN ] CIG: Loaded 45005 existing records. [2025-10-20T17:56:01.653852] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T17:56:01.730146] [INFO ] [REMOTE ] CIF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:01.752718] [WARN ] [MAIN ] CIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:01.754565] [INFO ] [MAIN ] CIG: Wiping existing primary data before full refresh. [2025-10-20T17:56:01.754633] [INFO ] [LOCAL ] CIG: Starting deletion of primary data. [2025-10-20T17:56:01.755297] [DEBUG] [LOCAL ] CIG: Deleted primary data file. [2025-10-20T17:56:01.755371] [INFO ] [MAIN ] CIG: Starting full primary refresh. [2025-10-20T17:56:01.856545] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-09. [2025-10-20T17:56:02.221243] [INFO ] [REMOTE ] CIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:02.264602] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-10-20T17:56:02.314201] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T17:56:02.651420] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-10. [2025-10-20T17:56:03.092595] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T17:56:03.093567] [INFO ] [REMOTE ] CICB: Full fetch complete (1970 records). [2025-10-20T17:56:03.093636] [DEBUG] [VALIDATOR ] CICB: Validating primary data. [2025-10-20T17:56:03.273359] [WARN ] [VALIDATOR ] CICB: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-10-20T17:56:03.273533] [FATAL] [MAIN ] CICB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:56:03.273598] [WARN ] [MAIN ] CICB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:56:03.273690] [INFO ] [LOCAL ] CICB: Starting deletion of primary data. [2025-10-20T17:56:03.273759] [DEBUG] [LOCAL ] CICB: No primary data file found to delete. [2025-10-20T17:56:03.273795] [INFO ] [LOCAL ] CICB: Starting deletion of derivative data. [2025-10-20T17:56:03.273833] [DEBUG] [LOCAL ] CICB: No derivative data file found to delete. [2025-10-20T17:56:03.273886] [INFO ] [MAIN ] Update Progress: 446/2367 symbols completed. CICB took 24.6s. Estimated remaining: 4h 1m 7s. Rate: 8.0 symbols/min [2025-10-20T17:56:03.443942] [INFO ] [MAIN ] CII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:03.453428] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-01. [2025-10-20T17:56:03.461889] [DEBUG] [LOCAL ] CII: Loaded 15546 existing records from primary storage. [2025-10-20T17:56:03.462331] [INFO ] [MAIN ] CII: Loaded 15546 existing records. [2025-10-20T17:56:03.639440] [WARN ] [MAIN ] CII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:03.639664] [INFO ] [MAIN ] CII: Wiping existing primary data before full refresh. [2025-10-20T17:56:03.639724] [INFO ] [LOCAL ] CII: Starting deletion of primary data. [2025-10-20T17:56:03.639918] [DEBUG] [LOCAL ] CII: Deleted primary data file. [2025-10-20T17:56:03.639973] [INFO ] [MAIN ] CII: Starting full primary refresh. [2025-10-20T17:56:03.992544] [INFO ] [REMOTE ] CII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:04.251388] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-01. [2025-10-20T17:56:04.981750] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T17:56:05.015850] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:56:05.051329] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-02. [2025-10-20T17:56:05.851378] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-01. [2025-10-20T17:56:06.369854] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T17:56:06.651405] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-03. [2025-10-20T17:56:06.679638] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T17:56:07.451385] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-02. [2025-10-20T17:56:08.251377] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-03. [2025-10-20T17:56:09.051376] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-02. [2025-10-20T17:56:09.361058] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T17:56:09.851395] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-03. [2025-10-20T17:56:10.227090] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-03 (3.58 seconds). [2025-10-20T17:56:10.282708] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T17:56:10.284559] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:56:10.307423] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T17:56:10.651550] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-04. [2025-10-20T17:56:11.363710] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T17:56:11.451384] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-04. [2025-10-20T17:56:12.251424] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-03. [2025-10-20T17:56:13.051409] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-05. [2025-10-20T17:56:13.094446] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:56:13.322053] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T17:56:13.456860] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T17:56:13.851439] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-04. [2025-10-20T17:56:14.651638] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-06. [2025-10-20T17:56:15.151928] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-10-20T17:56:15.451378] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-05. [2025-10-20T17:56:15.710826] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T17:56:16.251400] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-05. [2025-10-20T17:56:16.876350] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T17:56:17.051391] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-07. [2025-10-20T17:56:17.663113] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T17:56:17.851405] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-08. [2025-10-20T17:56:17.923551] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T17:56:18.416113] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T17:56:18.651607] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-09. [2025-10-20T17:56:19.252910] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T17:56:19.451357] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-10. [2025-10-20T17:56:19.994479] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T17:56:19.995808] [INFO ] [REMOTE ] CIF: Full fetch complete (6576 records). [2025-10-20T17:56:19.995885] [DEBUG] [VALIDATOR ] CIF: Validating primary data. [2025-10-20T17:56:20.259545] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-04. [2025-10-20T17:56:20.386233] [DEBUG] [VALIDATOR ] CIF: All expected trading days are present. [2025-10-20T17:56:20.386338] [DEBUG] [VALIDATOR ] CIF: Primary data validation passed. [2025-10-20T17:56:20.386677] [INFO ] [MAIN ] CIF: Fetched primary data is valid. [2025-10-20T17:56:20.386743] [DEBUG] [LOCAL ] CIF: Saving primary data. [2025-10-20T17:56:20.387197] [DEBUG] [LOCAL ] CIF: Trimmed 6576 records to 4025 (keeping 200 days of data) [2025-10-20T17:56:20.411476] [DEBUG] [LOCAL ] CIF: Primary data saved (4025 records). [2025-10-20T17:56:20.411613] [INFO ] [MAIN ] CIF: Full history saved. [2025-10-20T17:56:20.411742] [INFO ] [MAIN ] Update Progress: 447/2367 symbols completed. CIF took 20.0s. Estimated remaining: 4h 1m 41s. Rate: 7.9 symbols/min [2025-10-20T17:56:20.596844] [INFO ] [MAIN ] CIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:20.638045] [DEBUG] [LOCAL ] CIM: Loaded 38246 existing records from primary storage. [2025-10-20T17:56:20.638533] [INFO ] [MAIN ] CIM: Loaded 38246 existing records. [2025-10-20T17:56:20.881973] [WARN ] [MAIN ] CIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:20.882306] [INFO ] [MAIN ] CIM: Wiping existing primary data before full refresh. [2025-10-20T17:56:20.882363] [INFO ] [LOCAL ] CIM: Starting deletion of primary data. [2025-10-20T17:56:20.883027] [DEBUG] [LOCAL ] CIM: Deleted primary data file. [2025-10-20T17:56:20.883088] [INFO ] [MAIN ] CIM: Starting full primary refresh. [2025-10-20T17:56:21.062571] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-06. [2025-10-20T17:56:21.296289] [INFO ] [REMOTE ] CIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:21.448972] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T17:56:21.851373] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-05. [2025-10-20T17:56:22.651375] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-01. [2025-10-20T17:56:22.806822] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T17:56:22.928908] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T17:56:23.451370] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-06. [2025-10-20T17:56:24.251410] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-06. [2025-10-20T17:56:24.776241] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T17:56:25.051397] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-07. [2025-10-20T17:56:25.815416] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T17:56:25.851407] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-07. [2025-10-20T17:56:26.200570] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-01 (3.55 seconds). [2025-10-20T17:56:26.388071] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-10-20T17:56:26.651434] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-07. [2025-10-20T17:56:27.386880] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:56:27.451394] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-08. [2025-10-20T17:56:28.251383] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-08. [2025-10-20T17:56:28.359496] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T17:56:28.385791] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T17:56:29.051416] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-09. [2025-10-20T17:56:29.851369] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-02. [2025-10-20T17:56:29.900100] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:56:30.058452] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T17:56:30.651387] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-10. [2025-10-20T17:56:30.972206] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:56:31.451367] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-08. [2025-10-20T17:56:31.715933] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T17:56:31.720612] [INFO ] [REMOTE ] CII: Full fetch complete (22942 records). [2025-10-20T17:56:31.720824] [DEBUG] [VALIDATOR ] CII: Validating primary data. [2025-10-20T17:56:32.150678] [DEBUG] [VALIDATOR ] CII: All expected trading days are present. [2025-10-20T17:56:32.150792] [DEBUG] [VALIDATOR ] CII: Primary data validation passed. [2025-10-20T17:56:32.151219] [INFO ] [MAIN ] CII: Fetched primary data is valid. [2025-10-20T17:56:32.151259] [DEBUG] [LOCAL ] CII: Saving primary data. [2025-10-20T17:56:32.151926] [DEBUG] [LOCAL ] CII: Trimmed 22942 records to 15328 (keeping 200 days of data) [2025-10-20T17:56:32.244616] [DEBUG] [LOCAL ] CII: Primary data saved (15328 records). [2025-10-20T17:56:32.244789] [INFO ] [MAIN ] CII: Full history saved. [2025-10-20T17:56:32.244873] [INFO ] [MAIN ] Update Progress: 448/2367 symbols completed. CII took 29.0s. Estimated remaining: 4h 1m 52s. Rate: 7.9 symbols/min [2025-10-20T17:56:32.257930] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-09. [2025-10-20T17:56:32.419571] [INFO ] [MAIN ] CIMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:32.419769] [DEBUG] [LOCAL ] CIMN: No existing data file found in primary storage. [2025-10-20T17:56:32.419814] [WARN ] [MAIN ] CIMN: No existing data found, performing full refresh instead. [2025-10-20T17:56:32.419927] [INFO ] [MAIN ] CIMN: Wiping existing primary data before full refresh. [2025-10-20T17:56:32.419969] [INFO ] [LOCAL ] CIMN: Starting deletion of primary data. [2025-10-20T17:56:32.420008] [DEBUG] [LOCAL ] CIMN: No primary data file found to delete. [2025-10-20T17:56:32.420039] [INFO ] [MAIN ] CIMN: Starting full primary refresh. [2025-10-20T17:56:32.782824] [INFO ] [REMOTE ] CIMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:33.004958] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T17:56:33.051437] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-01. [2025-10-20T17:56:33.690863] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T17:56:33.851373] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-03. [2025-10-20T17:56:34.032089] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:56:34.651475] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-02. [2025-10-20T17:56:35.268433] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T17:56:35.305549] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T17:56:35.451384] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-10. [2025-10-20T17:56:36.251401] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-09. [2025-10-20T17:56:36.869941] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T17:56:36.879575] [INFO ] [REMOTE ] CIG: Full fetch complete (62509 records). [2025-10-20T17:56:36.879867] [DEBUG] [VALIDATOR ] CIG: Validating primary data. [2025-10-20T17:56:37.057487] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-04. [2025-10-20T17:56:37.318776] [DEBUG] [VALIDATOR ] CIG: All expected trading days are present. [2025-10-20T17:56:37.318879] [DEBUG] [VALIDATOR ] CIG: Primary data validation passed. [2025-10-20T17:56:37.319715] [INFO ] [MAIN ] CIG: Fetched primary data is valid. [2025-10-20T17:56:37.319784] [DEBUG] [LOCAL ] CIG: Saving primary data. [2025-10-20T17:56:37.321089] [DEBUG] [LOCAL ] CIG: Trimmed 62509 records to 44324 (keeping 200 days of data) [2025-10-20T17:56:37.570443] [DEBUG] [LOCAL ] CIG: Primary data saved (44324 records). [2025-10-20T17:56:37.570570] [INFO ] [MAIN ] CIG: Full history saved. [2025-10-20T17:56:37.570673] [INFO ] [MAIN ] Update Progress: 449/2367 symbols completed. CIG took 36.9s. Estimated remaining: 4h 1m 35s. Rate: 7.9 symbols/min [2025-10-20T17:56:37.758325] [INFO ] [MAIN ] CIMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:37.758488] [DEBUG] [LOCAL ] CIMO: No existing data file found in primary storage. [2025-10-20T17:56:37.758529] [WARN ] [MAIN ] CIMO: No existing data found, performing full refresh instead. [2025-10-20T17:56:37.758666] [INFO ] [MAIN ] CIMO: Wiping existing primary data before full refresh. [2025-10-20T17:56:37.758726] [INFO ] [LOCAL ] CIMO: Starting deletion of primary data. [2025-10-20T17:56:37.758773] [DEBUG] [LOCAL ] CIMO: No primary data file found to delete. [2025-10-20T17:56:37.758812] [INFO ] [MAIN ] CIMO: Starting full primary refresh. [2025-10-20T17:56:37.856523] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-03. [2025-10-20T17:56:38.147189] [INFO ] [REMOTE ] CIMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:38.192368] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T17:56:38.224613] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-10-20T17:56:38.622394] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T17:56:38.651412] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-01. [2025-10-20T17:56:39.451402] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-05. [2025-10-20T17:56:39.572754] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:56:40.251428] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-10. [2025-10-20T17:56:41.051367] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-04. [2025-10-20T17:56:41.637820] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T17:56:41.851381] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-05. [2025-10-20T17:56:41.893770] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-05 (2.44 seconds). [2025-10-20T17:56:42.459051] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T17:56:42.651369] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-06. [2025-10-20T17:56:42.984594] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-10 (2.73 seconds). [2025-10-20T17:56:42.993522] [INFO ] [REMOTE ] CIEN: Full fetch complete (75839 records). [2025-10-20T17:56:42.993964] [DEBUG] [VALIDATOR ] CIEN: Validating primary data. [2025-10-20T17:56:43.432160] [DEBUG] [VALIDATOR ] CIEN: All expected trading days are present. [2025-10-20T17:56:43.432283] [DEBUG] [VALIDATOR ] CIEN: Primary data validation passed. [2025-10-20T17:56:43.433482] [INFO ] [MAIN ] CIEN: Fetched primary data is valid. [2025-10-20T17:56:43.433552] [DEBUG] [LOCAL ] CIEN: Saving primary data. [2025-10-20T17:56:43.435858] [DEBUG] [LOCAL ] CIEN: Trimmed 75839 records to 52129 (keeping 200 days of data) [2025-10-20T17:56:43.487282] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-02. [2025-10-20T17:56:43.802255] [DEBUG] [LOCAL ] CIEN: Primary data saved (52129 records). [2025-10-20T17:56:43.802481] [INFO ] [MAIN ] CIEN: Full history saved. [2025-10-20T17:56:43.802591] [INFO ] [MAIN ] Update Progress: 450/2367 symbols completed. CIEN took 48.6s. Estimated remaining: 4h 1m 21s. Rate: 7.9 symbols/min [2025-10-20T17:56:43.901337] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T17:56:44.023251] [INFO ] [MAIN ] CIMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:44.042889] [DEBUG] [LOCAL ] CIMP: No existing data file found in primary storage. [2025-10-20T17:56:44.044268] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T17:56:44.044398] [WARN ] [MAIN ] CIMP: No existing data found, performing full refresh instead. [2025-10-20T17:56:44.052421] [INFO ] [MAIN ] CIMP: Wiping existing primary data before full refresh. [2025-10-20T17:56:44.052896] [INFO ] [LOCAL ] CIMP: Starting deletion of primary data. [2025-10-20T17:56:44.052973] [DEBUG] [LOCAL ] CIMP: No primary data file found to delete. [2025-10-20T17:56:44.053014] [INFO ] [MAIN ] CIMP: Starting full primary refresh. [2025-10-20T17:56:44.260051] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-06. [2025-10-20T17:56:44.476430] [INFO ] [REMOTE ] CIMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:44.879173] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T17:56:45.051393] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-03. [2025-10-20T17:56:45.522481] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T17:56:45.851386] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-07. [2025-10-20T17:56:46.651396] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-04. [2025-10-20T17:56:47.230276] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:56:47.262044] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T17:56:47.451388] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-05. [2025-10-20T17:56:47.772760] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-10-20T17:56:48.251391] [INFO ] [REMOTE ] CIMP: Fetching primary data for 2025-01. [2025-10-20T17:56:48.469734] [DEBUG] [REMOTE ] CIMP: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:56:48.469987] [WARN ] [REMOTE ] CIMP: No data returned for month 2025-01 [2025-10-20T17:56:48.470074] [FATAL] [REMOTE ] CIMP: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:56:48.470149] [FATAL] [MAIN ] CIMP: Failed to fetch primary data. [2025-10-20T17:56:48.470213] [WARN ] [MAIN ] CIMP: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:56:48.470285] [INFO ] [LOCAL ] CIMP: Starting deletion of primary data. [2025-10-20T17:56:48.470366] [DEBUG] [LOCAL ] CIMP: No primary data file found to delete. [2025-10-20T17:56:48.470437] [INFO ] [LOCAL ] CIMP: Starting deletion of derivative data. [2025-10-20T17:56:48.470481] [DEBUG] [LOCAL ] CIMP: No derivative data file found to delete. [2025-10-20T17:56:48.470526] [INFO ] [MAIN ] Update Progress: 451/2367 symbols completed. CIMP took 4.7s. Estimated remaining: 4h 1m 2s. Rate: 7.9 symbols/min [2025-10-20T17:56:48.637911] [INFO ] [MAIN ] CINT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:48.657576] [DEBUG] [LOCAL ] CINT: Loaded 20304 existing records from primary storage. [2025-10-20T17:56:48.657883] [INFO ] [MAIN ] CINT: Loaded 20304 existing records. [2025-10-20T17:56:48.836342] [WARN ] [MAIN ] CINT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:48.836542] [INFO ] [MAIN ] CINT: Wiping existing primary data before full refresh. [2025-10-20T17:56:48.836591] [INFO ] [LOCAL ] CINT: Starting deletion of primary data. [2025-10-20T17:56:48.836818] [DEBUG] [LOCAL ] CINT: Deleted primary data file. [2025-10-20T17:56:48.836876] [INFO ] [MAIN ] CINT: Starting full primary refresh. [2025-10-20T17:56:49.056514] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-07. [2025-10-20T17:56:49.201031] [INFO ] [REMOTE ] CINT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:49.544451] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-10-20T17:56:49.851370] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-06. [2025-10-20T17:56:50.433794] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T17:56:50.651422] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-07. [2025-10-20T17:56:51.115666] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T17:56:51.451381] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-08. [2025-10-20T17:56:51.935145] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T17:56:52.251370] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-09. [2025-10-20T17:56:52.887082] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T17:56:53.051389] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-01. [2025-10-20T17:56:53.851443] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-08. [2025-10-20T17:56:54.028162] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T17:56:54.651416] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-10. [2025-10-20T17:56:55.321518] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:56:55.323082] [INFO ] [REMOTE ] CIMO: Full fetch complete (4633 records). [2025-10-20T17:56:55.323175] [DEBUG] [VALIDATOR ] CIMO: Validating primary data. [2025-10-20T17:56:55.433701] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:56:55.460266] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-08. [2025-10-20T17:56:55.547525] [WARN ] [VALIDATOR ] CIMO: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T17:56:55.548135] [FATAL] [MAIN ] CIMO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:56:55.548205] [WARN ] [MAIN ] CIMO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:56:55.548278] [INFO ] [LOCAL ] CIMO: Starting deletion of primary data. [2025-10-20T17:56:55.548372] [DEBUG] [LOCAL ] CIMO: No primary data file found to delete. [2025-10-20T17:56:55.548429] [INFO ] [LOCAL ] CIMO: Starting deletion of derivative data. [2025-10-20T17:56:55.548473] [DEBUG] [LOCAL ] CIMO: No derivative data file found to delete. [2025-10-20T17:56:55.548531] [INFO ] [MAIN ] Update Progress: 452/2367 symbols completed. CIMO took 18.0s. Estimated remaining: 4h 0m 52s. Rate: 8.0 symbols/min [2025-10-20T17:56:55.722760] [INFO ] [MAIN ] CIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:55.742499] [DEBUG] [LOCAL ] CIO: Loaded 21338 existing records from primary storage. [2025-10-20T17:56:55.742789] [INFO ] [MAIN ] CIO: Loaded 21338 existing records. [2025-10-20T17:56:55.919690] [WARN ] [MAIN ] CIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:55.919895] [INFO ] [MAIN ] CIO: Wiping existing primary data before full refresh. [2025-10-20T17:56:55.919941] [INFO ] [LOCAL ] CIO: Starting deletion of primary data. [2025-10-20T17:56:55.920207] [DEBUG] [LOCAL ] CIO: Deleted primary data file. [2025-10-20T17:56:55.920264] [INFO ] [MAIN ] CIO: Starting full primary refresh. [2025-10-20T17:56:56.105577] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T17:56:56.254871] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-02. [2025-10-20T17:56:56.291427] [INFO ] [REMOTE ] CIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:57.051372] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-01. [2025-10-20T17:56:57.099664] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T17:56:57.851362] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-09. [2025-10-20T17:56:58.078579] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T17:56:58.651400] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-02. [2025-10-20T17:56:59.191149] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T17:56:59.451424] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-10. [2025-10-20T17:56:59.583440] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T17:57:00.251371] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-03. [2025-10-20T17:57:00.940216] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:57:00.946139] [INFO ] [REMOTE ] CIM: Full fetch complete (54902 records). [2025-10-20T17:57:00.946238] [DEBUG] [VALIDATOR ] CIM: Validating primary data. [2025-10-20T17:57:01.053770] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-03. [2025-10-20T17:57:01.356339] [DEBUG] [VALIDATOR ] CIM: All expected trading days are present. [2025-10-20T17:57:01.357083] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T17:57:01.357179] [DEBUG] [VALIDATOR ] CIM: Primary data validation passed. [2025-10-20T17:57:01.361674] [INFO ] [MAIN ] CIM: Fetched primary data is valid. [2025-10-20T17:57:01.362012] [DEBUG] [LOCAL ] CIM: Saving primary data. [2025-10-20T17:57:01.363006] [DEBUG] [LOCAL ] CIM: Trimmed 54902 records to 37626 (keeping 200 days of data) [2025-10-20T17:57:01.593260] [DEBUG] [LOCAL ] CIM: Primary data saved (37626 records). [2025-10-20T17:57:01.593405] [INFO ] [MAIN ] CIM: Full history saved. [2025-10-20T17:57:01.593484] [INFO ] [MAIN ] Update Progress: 453/2367 symbols completed. CIM took 41.2s. Estimated remaining: 4h 0m 38s. Rate: 8.0 symbols/min [2025-10-20T17:57:01.769957] [INFO ] [MAIN ] CION: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:01.795110] [DEBUG] [LOCAL ] CION: Loaded 28822 existing records from primary storage. [2025-10-20T17:57:01.795398] [INFO ] [MAIN ] CION: Loaded 28822 existing records. [2025-10-20T17:57:01.861150] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-04. [2025-10-20T17:57:02.015934] [WARN ] [MAIN ] CION: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:02.016453] [INFO ] [MAIN ] CION: Wiping existing primary data before full refresh. [2025-10-20T17:57:02.016530] [INFO ] [LOCAL ] CION: Starting deletion of primary data. [2025-10-20T17:57:02.016927] [DEBUG] [LOCAL ] CION: Deleted primary data file. [2025-10-20T17:57:02.016987] [INFO ] [MAIN ] CION: Starting full primary refresh. [2025-10-20T17:57:02.147159] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:57:02.412608] [INFO ] [REMOTE ] CION: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:02.651379] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-01. [2025-10-20T17:57:02.775580] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T17:57:03.451386] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-05. [2025-10-20T17:57:03.854328] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:57:04.251399] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-04. [2025-10-20T17:57:04.362138] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T17:57:04.976055] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T17:57:05.051385] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-09. [2025-10-20T17:57:05.537862] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T17:57:05.851427] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-10. [2025-10-20T17:57:06.501802] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T17:57:06.503631] [INFO ] [REMOTE ] CIMN: Full fetch complete (3934 records). [2025-10-20T17:57:06.504035] [DEBUG] [VALIDATOR ] CIMN: Validating primary data. [2025-10-20T17:57:06.656627] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-06. [2025-10-20T17:57:06.899936] [DEBUG] [VALIDATOR ] CIMN: All expected trading days are present. [2025-10-20T17:57:06.900038] [DEBUG] [VALIDATOR ] CIMN: Primary data validation passed. [2025-10-20T17:57:06.900201] [INFO ] [MAIN ] CIMN: Fetched primary data is valid. [2025-10-20T17:57:06.900242] [DEBUG] [LOCAL ] CIMN: Saving primary data. [2025-10-20T17:57:06.900636] [DEBUG] [LOCAL ] CIMN: Trimmed 3934 records to 2756 (keeping 200 days of data) [2025-10-20T17:57:06.918437] [DEBUG] [LOCAL ] CIMN: Primary data saved (2756 records). [2025-10-20T17:57:06.918573] [INFO ] [MAIN ] CIMN: Full history saved. [2025-10-20T17:57:06.918676] [INFO ] [MAIN ] Update Progress: 454/2367 symbols completed. CIMN took 34.7s. Estimated remaining: 4h 0m 21s. Rate: 8.0 symbols/min [2025-10-20T17:57:07.113051] [INFO ] [MAIN ] CIVI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:07.165606] [DEBUG] [LOCAL ] CIVI: Loaded 53478 existing records from primary storage. [2025-10-20T17:57:07.166010] [INFO ] [MAIN ] CIVI: Loaded 53478 existing records. [2025-10-20T17:57:07.403145] [WARN ] [MAIN ] CIVI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:07.403370] [INFO ] [MAIN ] CIVI: Wiping existing primary data before full refresh. [2025-10-20T17:57:07.403421] [INFO ] [LOCAL ] CIVI: Starting deletion of primary data. [2025-10-20T17:57:07.404050] [DEBUG] [LOCAL ] CIVI: Deleted primary data file. [2025-10-20T17:57:07.404106] [INFO ] [MAIN ] CIVI: Starting full primary refresh. [2025-10-20T17:57:07.461261] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-02. [2025-10-20T17:57:07.745496] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T17:57:07.815799] [INFO ] [REMOTE ] CIVI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:08.251419] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-01. [2025-10-20T17:57:08.618254] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:57:09.051433] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-03. [2025-10-20T17:57:09.853438] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-07. [2025-10-20T17:57:09.857219] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:57:10.049828] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T17:57:10.651393] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-04. [2025-10-20T17:57:10.831361] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:57:11.451385] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-05. [2025-10-20T17:57:12.064845] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T17:57:12.251428] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-08. [2025-10-20T17:57:12.362925] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T17:57:13.057598] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-02. [2025-10-20T17:57:13.060151] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T17:57:13.851397] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-09. [2025-10-20T17:57:14.499349] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T17:57:14.651386] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-03. [2025-10-20T17:57:15.063114] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T17:57:15.451406] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-05. [2025-10-20T17:57:16.251424] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-06. [2025-10-20T17:57:16.541690] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T17:57:16.733152] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T17:57:16.971352] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T17:57:17.051407] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-06. [2025-10-20T17:57:17.851581] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-10. [2025-10-20T17:57:18.336534] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:57:18.651457] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-04. [2025-10-20T17:57:18.886809] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T17:57:18.893830] [INFO ] [REMOTE ] CINT: Full fetch complete (28264 records). [2025-10-20T17:57:18.893996] [DEBUG] [VALIDATOR ] CINT: Validating primary data. [2025-10-20T17:57:19.265862] [DEBUG] [VALIDATOR ] CINT: All expected trading days are present. [2025-10-20T17:57:19.265935] [DEBUG] [VALIDATOR ] CINT: Primary data validation passed. [2025-10-20T17:57:19.266375] [INFO ] [MAIN ] CINT: Fetched primary data is valid. [2025-10-20T17:57:19.266413] [DEBUG] [LOCAL ] CINT: Saving primary data. [2025-10-20T17:57:19.267090] [DEBUG] [LOCAL ] CINT: Trimmed 28264 records to 20229 (keeping 200 days of data) [2025-10-20T17:57:19.374742] [DEBUG] [LOCAL ] CINT: Primary data saved (20229 records). [2025-10-20T17:57:19.374861] [INFO ] [MAIN ] CINT: Full history saved. [2025-10-20T17:57:19.374938] [INFO ] [MAIN ] Update Progress: 455/2367 symbols completed. CINT took 30.9s. Estimated remaining: 4h 0m 34s. Rate: 7.9 symbols/min [2025-10-20T17:57:19.456547] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-07. [2025-10-20T17:57:19.547827] [INFO ] [MAIN ] CL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:19.595930] [DEBUG] [LOCAL ] CL: Loaded 54429 existing records from primary storage. [2025-10-20T17:57:19.596254] [INFO ] [MAIN ] CL: Loaded 54429 existing records. [2025-10-20T17:57:19.772861] [WARN ] [MAIN ] CL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:19.773017] [INFO ] [MAIN ] CL: Wiping existing primary data before full refresh. [2025-10-20T17:57:19.773064] [INFO ] [LOCAL ] CL: Starting deletion of primary data. [2025-10-20T17:57:19.773351] [DEBUG] [LOCAL ] CL: Deleted primary data file. [2025-10-20T17:57:19.773410] [INFO ] [MAIN ] CL: Starting full primary refresh. [2025-10-20T17:57:20.131109] [INFO ] [REMOTE ] CL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:20.230823] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:57:20.251360] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-07. [2025-10-20T17:57:20.672882] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T17:57:21.051350] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-01. [2025-10-20T17:57:21.263741] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T17:57:21.851392] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-05. [2025-10-20T17:57:22.542220] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T17:57:22.651372] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-02. [2025-10-20T17:57:23.451408] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-08. [2025-10-20T17:57:23.561778] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:57:24.251389] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-06. [2025-10-20T17:57:24.404064] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T17:57:24.683447] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T17:57:25.051394] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-03. [2025-10-20T17:57:25.851420] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-09. [2025-10-20T17:57:25.947958] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T17:57:26.651475] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-07. [2025-10-20T17:57:26.992062] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T17:57:27.043886] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T17:57:27.451488] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-08. [2025-10-20T17:57:28.106266] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T17:57:28.251447] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-04. [2025-10-20T17:57:28.683973] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T17:57:29.051436] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-08. [2025-10-20T17:57:29.851407] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-09. [2025-10-20T17:57:29.965988] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:57:30.484673] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T17:57:30.651363] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-05. [2025-10-20T17:57:30.869017] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:57:31.451450] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-10. [2025-10-20T17:57:32.251364] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-10. [2025-10-20T17:57:32.500144] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:57:32.505327] [INFO ] [REMOTE ] CIO: Full fetch complete (29666 records). [2025-10-20T17:57:32.505453] [DEBUG] [VALIDATOR ] CIO: Validating primary data. [2025-10-20T17:57:32.860448] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-05 (2.21 seconds). [2025-10-20T17:57:32.959985] [DEBUG] [VALIDATOR ] CIO: All expected trading days are present. [2025-10-20T17:57:32.960070] [DEBUG] [VALIDATOR ] CIO: Primary data validation passed. [2025-10-20T17:57:32.960527] [INFO ] [MAIN ] CIO: Fetched primary data is valid. [2025-10-20T17:57:32.960566] [DEBUG] [LOCAL ] CIO: Saving primary data. [2025-10-20T17:57:32.961347] [DEBUG] [LOCAL ] CIO: Trimmed 29666 records to 21090 (keeping 200 days of data) [2025-10-20T17:57:33.060961] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-06. [2025-10-20T17:57:33.077947] [DEBUG] [LOCAL ] CIO: Primary data saved (21090 records). [2025-10-20T17:57:33.078091] [INFO ] [MAIN ] CIO: Full history saved. [2025-10-20T17:57:33.078170] [INFO ] [MAIN ] Update Progress: 456/2367 symbols completed. CIO took 37.5s. Estimated remaining: 4h 0m 53s. Rate: 7.9 symbols/min [2025-10-20T17:57:33.245662] [INFO ] [MAIN ] CLB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:33.276324] [DEBUG] [LOCAL ] CLB: Loaded 33985 existing records from primary storage. [2025-10-20T17:57:33.276599] [INFO ] [MAIN ] CLB: Loaded 33985 existing records. [2025-10-20T17:57:33.455773] [WARN ] [MAIN ] CLB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:33.456031] [INFO ] [MAIN ] CLB: Wiping existing primary data before full refresh. [2025-10-20T17:57:33.456210] [INFO ] [LOCAL ] CLB: Starting deletion of primary data. [2025-10-20T17:57:33.456622] [DEBUG] [LOCAL ] CLB: Deleted primary data file. [2025-10-20T17:57:33.456713] [INFO ] [MAIN ] CLB: Starting full primary refresh. [2025-10-20T17:57:33.581901] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:57:33.633965] [INFO ] [REMOTE ] CION: Full fetch complete (40693 records). [2025-10-20T17:57:33.649375] [DEBUG] [VALIDATOR ] CION: Validating primary data. [2025-10-20T17:57:33.852248] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-09. [2025-10-20T17:57:33.969040] [INFO ] [REMOTE ] CLB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:34.235976] [DEBUG] [VALIDATOR ] CION: All expected trading days are present. [2025-10-20T17:57:34.236083] [DEBUG] [VALIDATOR ] CION: Primary data validation passed. [2025-10-20T17:57:34.236794] [INFO ] [MAIN ] CION: Fetched primary data is valid. [2025-10-20T17:57:34.236857] [DEBUG] [LOCAL ] CION: Saving primary data. [2025-10-20T17:57:34.238131] [DEBUG] [LOCAL ] CION: Trimmed 40693 records to 28316 (keeping 200 days of data) [2025-10-20T17:57:34.413743] [DEBUG] [LOCAL ] CION: Primary data saved (28316 records). [2025-10-20T17:57:34.413931] [INFO ] [MAIN ] CION: Full history saved. [2025-10-20T17:57:34.414028] [INFO ] [MAIN ] Update Progress: 457/2367 symbols completed. CION took 32.8s. Estimated remaining: 4h 0m 19s. Rate: 7.9 symbols/min [2025-10-20T17:57:34.599145] [INFO ] [MAIN ] CLCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:34.615125] [DEBUG] [LOCAL ] CLCO: Loaded 13484 existing records from primary storage. [2025-10-20T17:57:34.615464] [INFO ] [MAIN ] CLCO: Loaded 13484 existing records. [2025-10-20T17:57:34.681723] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-01. [2025-10-20T17:57:34.836246] [WARN ] [MAIN ] CLCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:34.842535] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:57:34.853041] [INFO ] [MAIN ] CLCO: Wiping existing primary data before full refresh. [2025-10-20T17:57:34.853719] [INFO ] [LOCAL ] CLCO: Starting deletion of primary data. [2025-10-20T17:57:34.854356] [DEBUG] [LOCAL ] CLCO: Deleted primary data file. [2025-10-20T17:57:34.854437] [INFO ] [MAIN ] CLCO: Starting full primary refresh. [2025-10-20T17:57:35.251882] [INFO ] [REMOTE ] CLCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:35.451386] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-01. [2025-10-20T17:57:35.742582] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T17:57:36.072084] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:57:36.251399] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-02. [2025-10-20T17:57:36.639622] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:57:37.051388] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-02. [2025-10-20T17:57:37.261444] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T17:57:37.851358] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-03. [2025-10-20T17:57:38.002003] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T17:57:38.651399] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-03. [2025-10-20T17:57:38.801828] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:57:39.451447] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-10. [2025-10-20T17:57:39.661825] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:57:40.251403] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-04. [2025-10-20T17:57:41.051415] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-04. [2025-10-20T17:57:41.320036] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T17:57:41.329059] [INFO ] [REMOTE ] CIVI: Full fetch complete (76040 records). [2025-10-20T17:57:41.329318] [DEBUG] [VALIDATOR ] CIVI: Validating primary data. [2025-10-20T17:57:41.766400] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T17:57:41.841096] [DEBUG] [VALIDATOR ] CIVI: All expected trading days are present. [2025-10-20T17:57:41.841209] [DEBUG] [VALIDATOR ] CIVI: Primary data validation passed. [2025-10-20T17:57:41.842430] [INFO ] [MAIN ] CIVI: Fetched primary data is valid. [2025-10-20T17:57:41.842501] [DEBUG] [LOCAL ] CIVI: Saving primary data. [2025-10-20T17:57:41.844980] [DEBUG] [LOCAL ] CIVI: Trimmed 76040 records to 52736 (keeping 200 days of data) [2025-10-20T17:57:41.851340] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-05. [2025-10-20T17:57:42.098300] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T17:57:42.219123] [DEBUG] [LOCAL ] CIVI: Primary data saved (52736 records). [2025-10-20T17:57:42.219339] [INFO ] [MAIN ] CIVI: Full history saved. [2025-10-20T17:57:42.219451] [INFO ] [MAIN ] Update Progress: 458/2367 symbols completed. CIVI took 35.3s. Estimated remaining: 4h 0m 13s. Rate: 7.9 symbols/min [2025-10-20T17:57:42.397626] [INFO ] [MAIN ] CLDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:42.427685] [DEBUG] [LOCAL ] CLDT: Loaded 25824 existing records from primary storage. [2025-10-20T17:57:42.428191] [INFO ] [MAIN ] CLDT: Loaded 25824 existing records. [2025-10-20T17:57:42.623200] [WARN ] [MAIN ] CLDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:42.623532] [INFO ] [MAIN ] CLDT: Wiping existing primary data before full refresh. [2025-10-20T17:57:42.623701] [INFO ] [LOCAL ] CLDT: Starting deletion of primary data. [2025-10-20T17:57:42.624021] [DEBUG] [LOCAL ] CLDT: Deleted primary data file. [2025-10-20T17:57:42.624103] [INFO ] [MAIN ] CLDT: Starting full primary refresh. [2025-10-20T17:57:42.657827] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-05. [2025-10-20T17:57:43.009567] [INFO ] [REMOTE ] CLDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:43.233314] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T17:57:43.451351] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-06. [2025-10-20T17:57:43.523747] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T17:57:44.251334] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-06. [2025-10-20T17:57:44.664266] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:57:45.051369] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-07. [2025-10-20T17:57:45.115818] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T17:57:45.851482] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-07. [2025-10-20T17:57:46.651476] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-01. [2025-10-20T17:57:46.971159] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T17:57:47.427456] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T17:57:47.451881] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-07. [2025-10-20T17:57:47.454169] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T17:57:48.251407] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-08. [2025-10-20T17:57:48.325702] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T17:57:49.051461] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-08. [2025-10-20T17:57:49.378530] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:57:49.672212] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T17:57:49.851439] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-02. [2025-10-20T17:57:50.651426] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-09. [2025-10-20T17:57:50.678855] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T17:57:51.451424] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-03. [2025-10-20T17:57:51.995584] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T17:57:52.251370] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-10. [2025-10-20T17:57:52.707471] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:57:53.051422] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-08. [2025-10-20T17:57:53.653695] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T17:57:53.659596] [INFO ] [REMOTE ] CLB: Full fetch complete (47275 records). [2025-10-20T17:57:53.659783] [DEBUG] [VALIDATOR ] CLB: Validating primary data. [2025-10-20T17:57:53.853128] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-09. [2025-10-20T17:57:54.085836] [DEBUG] [VALIDATOR ] CLB: All expected trading days are present. [2025-10-20T17:57:54.085957] [DEBUG] [VALIDATOR ] CLB: Primary data validation passed. [2025-10-20T17:57:54.086920] [INFO ] [MAIN ] CLB: Fetched primary data is valid. [2025-10-20T17:57:54.087031] [DEBUG] [LOCAL ] CLB: Saving primary data. [2025-10-20T17:57:54.088614] [DEBUG] [LOCAL ] CLB: Trimmed 47275 records to 33606 (keeping 200 days of data) [2025-10-20T17:57:54.319820] [DEBUG] [LOCAL ] CLB: Primary data saved (33606 records). [2025-10-20T17:57:54.320017] [INFO ] [MAIN ] CLB: Full history saved. [2025-10-20T17:57:54.320411] [INFO ] [MAIN ] Update Progress: 459/2367 symbols completed. CLB took 21.2s. Estimated remaining: 4h 0m 24s. Rate: 7.9 symbols/min [2025-10-20T17:57:54.506789] [INFO ] [MAIN ] CLF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:54.568743] [DEBUG] [LOCAL ] CLF: Loaded 54521 existing records from primary storage. [2025-10-20T17:57:54.569199] [INFO ] [MAIN ] CLF: Loaded 54521 existing records. [2025-10-20T17:57:54.666324] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-04. [2025-10-20T17:57:54.769864] [WARN ] [MAIN ] CLF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:54.770435] [INFO ] [MAIN ] CLF: Wiping existing primary data before full refresh. [2025-10-20T17:57:54.770522] [INFO ] [LOCAL ] CLF: Starting deletion of primary data. [2025-10-20T17:57:54.770930] [DEBUG] [LOCAL ] CLF: Deleted primary data file. [2025-10-20T17:57:54.770992] [INFO ] [MAIN ] CLF: Starting full primary refresh. [2025-10-20T17:57:55.114388] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T17:57:55.204680] [INFO ] [REMOTE ] CLF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:55.240061] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-08 (2.19 seconds). [2025-10-20T17:57:55.451399] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-01. [2025-10-20T17:57:56.049314] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:57:56.251428] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-09. [2025-10-20T17:57:57.051406] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-10. [2025-10-20T17:57:57.436990] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T17:57:57.851443] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-02. [2025-10-20T17:57:58.212076] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:57:58.217184] [INFO ] [REMOTE ] CLCO: Full fetch complete (22860 records). [2025-10-20T17:57:58.217452] [DEBUG] [VALIDATOR ] CLCO: Validating primary data. [2025-10-20T17:57:58.639677] [DEBUG] [VALIDATOR ] CLCO: All expected trading days are present. [2025-10-20T17:57:58.639847] [DEBUG] [VALIDATOR ] CLCO: Primary data validation passed. [2025-10-20T17:57:58.640622] [INFO ] [MAIN ] CLCO: Fetched primary data is valid. [2025-10-20T17:57:58.640876] [DEBUG] [LOCAL ] CLCO: Saving primary data. [2025-10-20T17:57:58.642066] [DEBUG] [LOCAL ] CLCO: Trimmed 22860 records to 13189 (keeping 200 days of data) [2025-10-20T17:57:58.660428] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-05. [2025-10-20T17:57:58.727958] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-09 (2.48 seconds). [2025-10-20T17:57:58.783063] [DEBUG] [LOCAL ] CLCO: Primary data saved (13189 records). [2025-10-20T17:57:58.783282] [INFO ] [MAIN ] CLCO: Full history saved. [2025-10-20T17:57:58.783394] [INFO ] [MAIN ] Update Progress: 460/2367 symbols completed. CLCO took 24.4s. Estimated remaining: 4h 0m 4s. Rate: 7.9 symbols/min [2025-10-20T17:57:58.970234] [INFO ] [MAIN ] CLH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:59.003053] [DEBUG] [LOCAL ] CLH: Loaded 28288 existing records from primary storage. [2025-10-20T17:57:59.003488] [INFO ] [MAIN ] CLH: Loaded 28288 existing records. [2025-10-20T17:57:59.193595] [WARN ] [MAIN ] CLH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:59.193883] [INFO ] [MAIN ] CLH: Wiping existing primary data before full refresh. [2025-10-20T17:57:59.193932] [INFO ] [LOCAL ] CLH: Starting deletion of primary data. [2025-10-20T17:57:59.194182] [DEBUG] [LOCAL ] CLH: Deleted primary data file. [2025-10-20T17:57:59.194239] [INFO ] [MAIN ] CLH: Starting full primary refresh. [2025-10-20T17:57:59.460697] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-10. [2025-10-20T17:57:59.573815] [INFO ] [REMOTE ] CLH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:59.886366] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T17:58:00.064784] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-10-20T17:58:00.251375] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-06. [2025-10-20T17:58:01.051439] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-01. [2025-10-20T17:58:01.290535] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:58:01.613542] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T17:58:01.625626] [INFO ] [REMOTE ] CL: Full fetch complete (77817 records). [2025-10-20T17:58:01.625855] [DEBUG] [VALIDATOR ] CL: Validating primary data. [2025-10-20T17:58:01.859400] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-07. [2025-10-20T17:58:02.052737] [DEBUG] [VALIDATOR ] CL: All expected trading days are present. [2025-10-20T17:58:02.053054] [DEBUG] [VALIDATOR ] CL: Primary data validation passed. [2025-10-20T17:58:02.053987] [INFO ] [MAIN ] CL: Fetched primary data is valid. [2025-10-20T17:58:02.054055] [DEBUG] [LOCAL ] CL: Saving primary data. [2025-10-20T17:58:02.055702] [DEBUG] [LOCAL ] CL: Trimmed 77817 records to 53649 (keeping 200 days of data) [2025-10-20T17:58:02.414689] [DEBUG] [LOCAL ] CL: Primary data saved (53649 records). [2025-10-20T17:58:02.414853] [INFO ] [MAIN ] CL: Full history saved. [2025-10-20T17:58:02.414939] [INFO ] [MAIN ] Update Progress: 461/2367 symbols completed. CL took 43.0s. Estimated remaining: 3h 59m 40s. Rate: 8.0 symbols/min [2025-10-20T17:58:02.614398] [INFO ] [MAIN ] CLPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:02.614798] [DEBUG] [LOCAL ] CLPR: No existing data file found in primary storage. [2025-10-20T17:58:02.614928] [WARN ] [MAIN ] CLPR: No existing data found, performing full refresh instead. [2025-10-20T17:58:02.615170] [INFO ] [MAIN ] CLPR: Wiping existing primary data before full refresh. [2025-10-20T17:58:02.622376] [INFO ] [LOCAL ] CLPR: Starting deletion of primary data. [2025-10-20T17:58:02.622470] [DEBUG] [LOCAL ] CLPR: No primary data file found to delete. [2025-10-20T17:58:02.622542] [INFO ] [MAIN ] CLPR: Starting full primary refresh. [2025-10-20T17:58:02.657066] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-03. [2025-10-20T17:58:02.701610] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:58:03.037095] [INFO ] [REMOTE ] CLPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:03.050396] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:58:03.451384] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-01. [2025-10-20T17:58:04.253697] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-08. [2025-10-20T17:58:04.257492] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:58:04.365683] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:58:05.051343] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-02. [2025-10-20T17:58:05.335112] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T17:58:05.851429] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-09. [2025-10-20T17:58:06.474864] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:58:06.570210] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T17:58:06.651380] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-03. [2025-10-20T17:58:07.451368] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-02. [2025-10-20T17:58:08.028773] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T17:58:08.202127] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T17:58:08.251376] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-10. [2025-10-20T17:58:09.051421] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-03. [2025-10-20T17:58:09.530852] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T17:58:09.535213] [INFO ] [REMOTE ] CLDT: Full fetch complete (37558 records). [2025-10-20T17:58:09.535293] [DEBUG] [VALIDATOR ] CLDT: Validating primary data. [2025-10-20T17:58:09.856509] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-04. [2025-10-20T17:58:09.904117] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T17:58:09.980228] [DEBUG] [VALIDATOR ] CLDT: All expected trading days are present. [2025-10-20T17:58:09.980338] [DEBUG] [VALIDATOR ] CLDT: Primary data validation passed. [2025-10-20T17:58:09.980990] [INFO ] [MAIN ] CLDT: Fetched primary data is valid. [2025-10-20T17:58:09.981056] [DEBUG] [LOCAL ] CLDT: Saving primary data. [2025-10-20T17:58:09.982071] [DEBUG] [LOCAL ] CLDT: Trimmed 37558 records to 25365 (keeping 200 days of data) [2025-10-20T17:58:10.135699] [DEBUG] [LOCAL ] CLDT: Primary data saved (25365 records). [2025-10-20T17:58:10.135831] [INFO ] [MAIN ] CLDT: Full history saved. [2025-10-20T17:58:10.135908] [INFO ] [MAIN ] Update Progress: 462/2367 symbols completed. CLDT took 27.9s. Estimated remaining: 3h 59m 33s. Rate: 8.0 symbols/min [2025-10-20T17:58:10.302754] [INFO ] [MAIN ] CLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:10.302913] [DEBUG] [LOCAL ] CLS: No existing data file found in primary storage. [2025-10-20T17:58:10.302949] [WARN ] [MAIN ] CLS: No existing data found, performing full refresh instead. [2025-10-20T17:58:10.303056] [INFO ] [MAIN ] CLS: Wiping existing primary data before full refresh. [2025-10-20T17:58:10.303095] [INFO ] [LOCAL ] CLS: Starting deletion of primary data. [2025-10-20T17:58:10.303136] [DEBUG] [LOCAL ] CLS: No primary data file found to delete. [2025-10-20T17:58:10.303166] [INFO ] [MAIN ] CLS: Starting full primary refresh. [2025-10-20T17:58:10.653931] [INFO ] [REMOTE ] CLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:10.654040] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-04. [2025-10-20T17:58:11.451387] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-04. [2025-10-20T17:58:11.634858] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T17:58:12.256070] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-05. [2025-10-20T17:58:12.261925] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-04 (2.41 seconds). [2025-10-20T17:58:12.889705] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T17:58:12.998600] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T17:58:13.051334] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-06. [2025-10-20T17:58:13.761752] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T17:58:13.851412] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-05. [2025-10-20T17:58:14.651411] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-05. [2025-10-20T17:58:14.983812] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:58:15.451443] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-01. [2025-10-20T17:58:16.242234] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T17:58:16.253482] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-06. [2025-10-20T17:58:17.051456] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-07. [2025-10-20T17:58:17.321230] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T17:58:17.857247] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-06. [2025-10-20T17:58:17.871324] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-01 (2.42 seconds). [2025-10-20T17:58:17.994829] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:58:18.651383] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-07. [2025-10-20T17:58:19.451348] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-02. [2025-10-20T17:58:19.904126] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:58:19.955569] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-10-20T17:58:20.251447] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-08. [2025-10-20T17:58:21.051527] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-07. [2025-10-20T17:58:21.065841] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T17:58:21.582296] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-10-20T17:58:21.851415] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-03. [2025-10-20T17:58:22.651379] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-08. [2025-10-20T17:58:23.409335] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-10-20T17:58:23.451411] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-09. [2025-10-20T17:58:23.831512] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T17:58:23.919529] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T17:58:23.975510] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-10-20T17:58:24.251388] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-08. [2025-10-20T17:58:25.051408] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-04. [2025-10-20T17:58:25.851450] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-09. [2025-10-20T17:58:26.205425] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-10-20T17:58:26.651359] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-09. [2025-10-20T17:58:26.812062] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T17:58:27.451354] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-10. [2025-10-20T17:58:27.600239] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-04 (2.55 seconds). [2025-10-20T17:58:28.251363] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-05. [2025-10-20T17:58:28.588529] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T17:58:28.784958] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:58:28.789508] [INFO ] [REMOTE ] CLH: Full fetch complete (42231 records). [2025-10-20T17:58:28.789588] [DEBUG] [VALIDATOR ] CLH: Validating primary data. [2025-10-20T17:58:29.056545] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-10. [2025-10-20T17:58:29.188199] [DEBUG] [VALIDATOR ] CLH: All expected trading days are present. [2025-10-20T17:58:29.188280] [DEBUG] [VALIDATOR ] CLH: Primary data validation passed. [2025-10-20T17:58:29.188964] [INFO ] [MAIN ] CLH: Fetched primary data is valid. [2025-10-20T17:58:29.189026] [DEBUG] [LOCAL ] CLH: Saving primary data. [2025-10-20T17:58:29.190008] [DEBUG] [LOCAL ] CLH: Trimmed 42231 records to 27808 (keeping 200 days of data) [2025-10-20T17:58:29.353905] [DEBUG] [LOCAL ] CLH: Primary data saved (27808 records). [2025-10-20T17:58:29.354045] [INFO ] [MAIN ] CLH: Full history saved. [2025-10-20T17:58:29.354158] [INFO ] [MAIN ] Update Progress: 463/2367 symbols completed. CLH took 30.6s. Estimated remaining: 4h 0m 13s. Rate: 7.9 symbols/min [2025-10-20T17:58:29.519289] [INFO ] [MAIN ] CLVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:29.519451] [DEBUG] [LOCAL ] CLVT: No existing data file found in primary storage. [2025-10-20T17:58:29.519488] [WARN ] [MAIN ] CLVT: No existing data found, performing full refresh instead. [2025-10-20T17:58:29.519599] [INFO ] [MAIN ] CLVT: Wiping existing primary data before full refresh. [2025-10-20T17:58:29.519659] [INFO ] [LOCAL ] CLVT: Starting deletion of primary data. [2025-10-20T17:58:29.519727] [DEBUG] [LOCAL ] CLVT: No primary data file found to delete. [2025-10-20T17:58:29.519760] [INFO ] [MAIN ] CLVT: Starting full primary refresh. [2025-10-20T17:58:29.853127] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-10. [2025-10-20T17:58:29.874586] [INFO ] [REMOTE ] CLVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:30.064317] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T17:58:30.065972] [INFO ] [REMOTE ] CLPR: Full fetch complete (16046 records). [2025-10-20T17:58:30.066042] [DEBUG] [VALIDATOR ] CLPR: Validating primary data. [2025-10-20T17:58:30.427914] [DEBUG] [VALIDATOR ] CLPR: All expected trading days are present. [2025-10-20T17:58:30.427995] [DEBUG] [VALIDATOR ] CLPR: Primary data validation passed. [2025-10-20T17:58:30.428372] [INFO ] [MAIN ] CLPR: Fetched primary data is valid. [2025-10-20T17:58:30.428439] [DEBUG] [LOCAL ] CLPR: Saving primary data. [2025-10-20T17:58:30.429037] [DEBUG] [LOCAL ] CLPR: Trimmed 16046 records to 10162 (keeping 200 days of data) [2025-10-20T17:58:30.487959] [DEBUG] [LOCAL ] CLPR: Primary data saved (10162 records). [2025-10-20T17:58:30.488085] [INFO ] [MAIN ] CLPR: Full history saved. [2025-10-20T17:58:30.488155] [INFO ] [MAIN ] Update Progress: 464/2367 symbols completed. CLPR took 28.1s. Estimated remaining: 3h 59m 39s. Rate: 7.9 symbols/min [2025-10-20T17:58:30.657181] [INFO ] [MAIN ] CLW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:30.657431] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-01. [2025-10-20T17:58:30.678272] [DEBUG] [LOCAL ] CLW: Loaded 21809 existing records from primary storage. [2025-10-20T17:58:30.678569] [INFO ] [MAIN ] CLW: Loaded 21809 existing records. [2025-10-20T17:58:30.867058] [WARN ] [MAIN ] CLW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:58:30.867481] [INFO ] [MAIN ] CLW: Wiping existing primary data before full refresh. [2025-10-20T17:58:30.867558] [INFO ] [LOCAL ] CLW: Starting deletion of primary data. [2025-10-20T17:58:30.867865] [DEBUG] [LOCAL ] CLW: Deleted primary data file. [2025-10-20T17:58:30.867927] [INFO ] [MAIN ] CLW: Starting full primary refresh. [2025-10-20T17:58:31.084037] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-05 (2.83 seconds). [2025-10-20T17:58:31.283852] [INFO ] [REMOTE ] CLW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:31.451431] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-06. [2025-10-20T17:58:32.148007] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-10 (2.29 seconds). [2025-10-20T17:58:32.156962] [INFO ] [REMOTE ] CLF: Full fetch complete (77918 records). [2025-10-20T17:58:32.157145] [DEBUG] [VALIDATOR ] CLF: Validating primary data. [2025-10-20T17:58:32.258817] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-01. [2025-10-20T17:58:32.362854] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T17:58:32.609881] [DEBUG] [VALIDATOR ] CLF: All expected trading days are present. [2025-10-20T17:58:32.610039] [DEBUG] [VALIDATOR ] CLF: Primary data validation passed. [2025-10-20T17:58:32.610926] [INFO ] [MAIN ] CLF: Fetched primary data is valid. [2025-10-20T17:58:32.610990] [DEBUG] [LOCAL ] CLF: Saving primary data. [2025-10-20T17:58:32.612442] [DEBUG] [LOCAL ] CLF: Trimmed 77918 records to 53741 (keeping 200 days of data) [2025-10-20T17:58:32.915091] [DEBUG] [LOCAL ] CLF: Primary data saved (53741 records). [2025-10-20T17:58:32.915224] [INFO ] [MAIN ] CLF: Full history saved. [2025-10-20T17:58:32.915571] [INFO ] [MAIN ] Update Progress: 465/2367 symbols completed. CLF took 38.6s. Estimated remaining: 3h 59m 11s. Rate: 8.0 symbols/min [2025-10-20T17:58:33.061328] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-02. [2025-10-20T17:58:33.087898] [INFO ] [MAIN ] CLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:33.138559] [DEBUG] [LOCAL ] CLX: Loaded 52408 existing records from primary storage. [2025-10-20T17:58:33.138897] [INFO ] [MAIN ] CLX: Loaded 52408 existing records. [2025-10-20T17:58:33.321363] [WARN ] [MAIN ] CLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:58:33.322090] [INFO ] [MAIN ] CLX: Wiping existing primary data before full refresh. [2025-10-20T17:58:33.322414] [INFO ] [LOCAL ] CLX: Starting deletion of primary data. [2025-10-20T17:58:33.322734] [DEBUG] [LOCAL ] CLX: Deleted primary data file. [2025-10-20T17:58:33.322794] [INFO ] [MAIN ] CLX: Starting full primary refresh. [2025-10-20T17:58:33.676260] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:58:33.742840] [INFO ] [REMOTE ] CLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:33.777603] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-06 (2.33 seconds). [2025-10-20T17:58:33.851362] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-07. [2025-10-20T17:58:34.288087] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:58:34.651436] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-02. [2025-10-20T17:58:35.451392] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-01. [2025-10-20T17:58:35.780384] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T17:58:36.229050] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-07 (2.38 seconds). [2025-10-20T17:58:36.251342] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-08. [2025-10-20T17:58:37.051415] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-03. [2025-10-20T17:58:37.183379] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T17:58:37.851437] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-02. [2025-10-20T17:58:38.287552] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-10-20T17:58:38.660742] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-09. [2025-10-20T17:58:38.686088] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T17:58:39.451412] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-03. [2025-10-20T17:58:39.621187] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T17:58:40.251399] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-04. [2025-10-20T17:58:40.551215] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T17:58:40.830718] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T17:58:41.051382] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-04. [2025-10-20T17:58:41.511503] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T17:58:41.851409] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-10. [2025-10-20T17:58:41.888906] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T17:58:42.651371] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-05. [2025-10-20T17:58:43.451442] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-05. [2025-10-20T17:58:44.251366] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-03. [2025-10-20T17:58:44.399590] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-10 (2.55 seconds). [2025-10-20T17:58:44.408297] [INFO ] [REMOTE ] CLS: Full fetch complete (77697 records). [2025-10-20T17:58:44.408556] [DEBUG] [VALIDATOR ] CLS: Validating primary data. [2025-10-20T17:58:44.532876] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T17:58:44.824502] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T17:58:44.875666] [DEBUG] [VALIDATOR ] CLS: All expected trading days are present. [2025-10-20T17:58:44.875775] [DEBUG] [VALIDATOR ] CLS: Primary data validation passed. [2025-10-20T17:58:44.877494] [INFO ] [MAIN ] CLS: Fetched primary data is valid. [2025-10-20T17:58:44.877561] [DEBUG] [LOCAL ] CLS: Saving primary data. [2025-10-20T17:58:44.879097] [DEBUG] [LOCAL ] CLS: Trimmed 77697 records to 53569 (keeping 200 days of data) [2025-10-20T17:58:45.053798] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-06. [2025-10-20T17:58:45.208263] [DEBUG] [LOCAL ] CLS: Primary data saved (53569 records). [2025-10-20T17:58:45.208463] [INFO ] [MAIN ] CLS: Full history saved. [2025-10-20T17:58:45.208561] [INFO ] [MAIN ] Update Progress: 466/2367 symbols completed. CLS took 35.1s. Estimated remaining: 3h 59m 23s. Rate: 7.9 symbols/min [2025-10-20T17:58:45.385448] [INFO ] [MAIN ] CM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:45.442573] [DEBUG] [LOCAL ] CM: Loaded 49434 existing records from primary storage. [2025-10-20T17:58:45.443221] [INFO ] [MAIN ] CM: Loaded 49434 existing records. [2025-10-20T17:58:45.631094] [WARN ] [MAIN ] CM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:58:45.631349] [INFO ] [MAIN ] CM: Wiping existing primary data before full refresh. [2025-10-20T17:58:45.631556] [INFO ] [LOCAL ] CM: Starting deletion of primary data. [2025-10-20T17:58:45.632077] [DEBUG] [LOCAL ] CM: Deleted primary data file. [2025-10-20T17:58:45.632138] [INFO ] [MAIN ] CM: Starting full primary refresh. [2025-10-20T17:58:45.856616] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-06. [2025-10-20T17:58:45.992583] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:58:46.087434] [INFO ] [REMOTE ] CM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:46.100117] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T17:58:46.651380] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-04. [2025-10-20T17:58:47.344076] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T17:58:47.451398] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-07. [2025-10-20T17:58:48.251358] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-01. [2025-10-20T17:58:48.548908] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T17:58:48.573262] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T17:58:49.051387] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-07. [2025-10-20T17:58:49.612596] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T17:58:49.851391] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-05. [2025-10-20T17:58:50.651404] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-02. [2025-10-20T17:58:50.786019] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:58:51.328530] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T17:58:51.451379] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-06. [2025-10-20T17:58:52.183810] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T17:58:52.251363] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-03. [2025-10-20T17:58:53.001396] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:58:53.051399] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-08. [2025-10-20T17:58:53.851348] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-08. [2025-10-20T17:58:53.881970] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T17:58:54.120773] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T17:58:54.651427] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-09. [2025-10-20T17:58:55.451356] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-04. [2025-10-20T17:58:55.656544] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T17:58:55.839896] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T17:58:56.251404] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-07. [2025-10-20T17:58:57.044851] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:58:57.054339] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-09. [2025-10-20T17:58:57.851352] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-10. [2025-10-20T17:58:58.214440] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T17:58:58.285538] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T17:58:58.520891] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:58:58.524147] [INFO ] [REMOTE ] CLW: Full fetch complete (33796 records). [2025-10-20T17:58:58.524223] [DEBUG] [VALIDATOR ] CLW: Validating primary data. [2025-10-20T17:58:58.659734] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-05. [2025-10-20T17:58:58.888754] [DEBUG] [VALIDATOR ] CLW: All expected trading days are present. [2025-10-20T17:58:58.888836] [DEBUG] [VALIDATOR ] CLW: Primary data validation passed. [2025-10-20T17:58:58.889336] [INFO ] [MAIN ] CLW: Fetched primary data is valid. [2025-10-20T17:58:58.889375] [DEBUG] [LOCAL ] CLW: Saving primary data. [2025-10-20T17:58:58.890139] [DEBUG] [LOCAL ] CLW: Trimmed 33796 records to 21460 (keeping 200 days of data) [2025-10-20T17:58:59.011709] [DEBUG] [LOCAL ] CLW: Primary data saved (21460 records). [2025-10-20T17:58:59.011834] [INFO ] [MAIN ] CLW: Full history saved. [2025-10-20T17:58:59.011908] [INFO ] [MAIN ] Update Progress: 467/2367 symbols completed. CLW took 28.5s. Estimated remaining: 3h 59m 41s. Rate: 7.9 symbols/min [2025-10-20T17:58:59.177518] [INFO ] [MAIN ] CMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:59.220431] [DEBUG] [LOCAL ] CMA: Loaded 50902 existing records from primary storage. [2025-10-20T17:58:59.220754] [INFO ] [MAIN ] CMA: Loaded 50902 existing records. [2025-10-20T17:58:59.391612] [WARN ] [MAIN ] CMA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:58:59.391843] [INFO ] [MAIN ] CMA: Wiping existing primary data before full refresh. [2025-10-20T17:58:59.391889] [INFO ] [LOCAL ] CMA: Starting deletion of primary data. [2025-10-20T17:58:59.392185] [DEBUG] [LOCAL ] CMA: Deleted primary data file. [2025-10-20T17:58:59.392223] [INFO ] [MAIN ] CMA: Starting full primary refresh. [2025-10-20T17:58:59.456618] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-10. [2025-10-20T17:58:59.741070] [INFO ] [REMOTE ] CMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:00.251395] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-08. [2025-10-20T17:59:00.521402] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:59:01.051350] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-01. [2025-10-20T17:59:01.435093] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T17:59:01.447257] [INFO ] [REMOTE ] CLVT: Full fetch complete (75393 records). [2025-10-20T17:59:01.447490] [DEBUG] [VALIDATOR ] CLVT: Validating primary data. [2025-10-20T17:59:01.852693] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-06. [2025-10-20T17:59:01.871295] [DEBUG] [VALIDATOR ] CLVT: All expected trading days are present. [2025-10-20T17:59:01.872587] [DEBUG] [VALIDATOR ] CLVT: Primary data validation passed. [2025-10-20T17:59:01.873507] [INFO ] [MAIN ] CLVT: Fetched primary data is valid. [2025-10-20T17:59:01.873710] [DEBUG] [LOCAL ] CLVT: Saving primary data. [2025-10-20T17:59:01.875270] [DEBUG] [LOCAL ] CLVT: Trimmed 75393 records to 51923 (keeping 200 days of data) [2025-10-20T17:59:02.171076] [DEBUG] [LOCAL ] CLVT: Primary data saved (51923 records). [2025-10-20T17:59:02.171408] [INFO ] [MAIN ] CLVT: Full history saved. [2025-10-20T17:59:02.171563] [INFO ] [MAIN ] Update Progress: 468/2367 symbols completed. CLVT took 32.8s. Estimated remaining: 3h 59m 15s. Rate: 7.9 symbols/min [2025-10-20T17:59:02.345031] [INFO ] [MAIN ] CMBT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:02.387243] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T17:59:02.439016] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:59:02.436833] [DEBUG] [LOCAL ] CMBT: Loaded 26862 existing records from primary storage. [2025-10-20T17:59:02.451043] [INFO ] [MAIN ] CMBT: Loaded 26862 existing records. [2025-10-20T17:59:02.622180] [WARN ] [MAIN ] CMBT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:02.622328] [INFO ] [MAIN ] CMBT: Wiping existing primary data before full refresh. [2025-10-20T17:59:02.622373] [INFO ] [LOCAL ] CMBT: Starting deletion of primary data. [2025-10-20T17:59:02.623109] [DEBUG] [LOCAL ] CMBT: Deleted primary data file. [2025-10-20T17:59:02.623165] [INFO ] [MAIN ] CMBT: Starting full primary refresh. [2025-10-20T17:59:02.656577] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-09. [2025-10-20T17:59:02.965365] [INFO ] [REMOTE ] CMBT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:03.456493] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-02. [2025-10-20T17:59:03.473727] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:59:04.217677] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T17:59:04.251356] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-07. [2025-10-20T17:59:04.684300] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:59:05.051442] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-10. [2025-10-20T17:59:05.851344] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-01. [2025-10-20T17:59:05.984994] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:59:06.651484] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-03. [2025-10-20T17:59:06.855997] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T17:59:06.863459] [INFO ] [REMOTE ] CLX: Full fetch complete (73725 records). [2025-10-20T17:59:06.863624] [DEBUG] [VALIDATOR ] CLX: Validating primary data. [2025-10-20T17:59:06.974541] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:59:07.264255] [DEBUG] [VALIDATOR ] CLX: All expected trading days are present. [2025-10-20T17:59:07.264331] [DEBUG] [VALIDATOR ] CLX: Primary data validation passed. [2025-10-20T17:59:07.265142] [INFO ] [MAIN ] CLX: Fetched primary data is valid. [2025-10-20T17:59:07.265205] [DEBUG] [LOCAL ] CLX: Saving primary data. [2025-10-20T17:59:07.266491] [DEBUG] [LOCAL ] CLX: Trimmed 73725 records to 51778 (keeping 200 days of data) [2025-10-20T17:59:07.473874] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-02. [2025-10-20T17:59:07.580577] [DEBUG] [LOCAL ] CLX: Primary data saved (51778 records). [2025-10-20T17:59:07.580751] [INFO ] [MAIN ] CLX: Full history saved. [2025-10-20T17:59:07.580842] [INFO ] [MAIN ] Update Progress: 469/2367 symbols completed. CLX took 34.7s. Estimated remaining: 3h 58m 59s. Rate: 7.9 symbols/min [2025-10-20T17:59:07.762977] [INFO ] [MAIN ] CMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:07.801821] [DEBUG] [LOCAL ] CMC: Loaded 46259 existing records from primary storage. [2025-10-20T17:59:07.802107] [INFO ] [MAIN ] CMC: Loaded 46259 existing records. [2025-10-20T17:59:07.982527] [WARN ] [MAIN ] CMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:07.982809] [INFO ] [MAIN ] CMC: Wiping existing primary data before full refresh. [2025-10-20T17:59:07.982962] [INFO ] [LOCAL ] CMC: Starting deletion of primary data. [2025-10-20T17:59:07.983180] [DEBUG] [LOCAL ] CMC: Deleted primary data file. [2025-10-20T17:59:07.983237] [INFO ] [MAIN ] CMC: Starting full primary refresh. [2025-10-20T17:59:08.251544] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-08. [2025-10-20T17:59:08.365393] [INFO ] [REMOTE ] CMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:08.366855] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T17:59:08.425459] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:59:09.051411] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-03. [2025-10-20T17:59:09.851384] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-01. [2025-10-20T17:59:09.904882] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:59:10.060600] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:59:10.651412] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-04. [2025-10-20T17:59:11.451435] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-04. [2025-10-20T17:59:11.545506] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:59:12.052367] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:59:12.251390] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-02. [2025-10-20T17:59:12.495982] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T17:59:13.051398] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-09. [2025-10-20T17:59:13.805303] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:59:13.851347] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-03. [2025-10-20T17:59:14.651408] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-05. [2025-10-20T17:59:14.736089] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:59:15.320240] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T17:59:15.451388] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-05. [2025-10-20T17:59:15.695996] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T17:59:16.251442] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-04. [2025-10-20T17:59:16.811364] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T17:59:17.051379] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-06. [2025-10-20T17:59:17.851365] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-06. [2025-10-20T17:59:18.040816] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:59:18.651430] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-10. [2025-10-20T17:59:18.768697] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T17:59:18.940692] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T17:59:19.451439] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-07. [2025-10-20T17:59:20.234742] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:59:20.245048] [INFO ] [REMOTE ] CM: Full fetch complete (69866 records). [2025-10-20T17:59:20.245271] [DEBUG] [VALIDATOR ] CM: Validating primary data. [2025-10-20T17:59:20.258501] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-05. [2025-10-20T17:59:20.649110] [DEBUG] [VALIDATOR ] CM: All expected trading days are present. [2025-10-20T17:59:20.649228] [DEBUG] [VALIDATOR ] CM: Primary data validation passed. [2025-10-20T17:59:20.650035] [INFO ] [MAIN ] CM: Fetched primary data is valid. [2025-10-20T17:59:20.650097] [DEBUG] [LOCAL ] CM: Saving primary data. [2025-10-20T17:59:20.651497] [DEBUG] [LOCAL ] CM: Trimmed 69866 records to 48575 (keeping 200 days of data) [2025-10-20T17:59:20.947281] [DEBUG] [LOCAL ] CM: Primary data saved (48575 records). [2025-10-20T17:59:20.947423] [INFO ] [MAIN ] CM: Full history saved. [2025-10-20T17:59:20.947501] [INFO ] [MAIN ] Update Progress: 470/2367 symbols completed. CM took 35.7s. Estimated remaining: 3h 59m 15s. Rate: 7.9 symbols/min [2025-10-20T17:59:21.053600] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-07. [2025-10-20T17:59:21.128746] [INFO ] [MAIN ] CMCM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:21.135426] [DEBUG] [LOCAL ] CMCM: Loaded 4894 existing records from primary storage. [2025-10-20T17:59:21.135767] [INFO ] [MAIN ] CMCM: Loaded 4894 existing records. [2025-10-20T17:59:21.275020] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T17:59:21.389971] [WARN ] [MAIN ] CMCM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:21.390236] [INFO ] [MAIN ] CMCM: Wiping existing primary data before full refresh. [2025-10-20T17:59:21.390293] [INFO ] [LOCAL ] CMCM: Starting deletion of primary data. [2025-10-20T17:59:21.390442] [DEBUG] [LOCAL ] CMCM: Deleted primary data file. [2025-10-20T17:59:21.390500] [INFO ] [MAIN ] CMCM: Starting full primary refresh. [2025-10-20T17:59:21.761677] [INFO ] [REMOTE ] CMCM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:21.851368] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-01. [2025-10-20T17:59:22.017281] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T17:59:22.167848] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T17:59:22.170984] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-01 (0.32 seconds). [2025-10-20T17:59:22.651451] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-06. [2025-10-20T17:59:23.451424] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-08. [2025-10-20T17:59:24.257635] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-02. [2025-10-20T17:59:24.271593] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:59:24.860942] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T17:59:25.051439] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-07. [2025-10-20T17:59:25.340780] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T17:59:25.851398] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-03. [2025-10-20T17:59:26.517073] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T17:59:26.585768] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T17:59:26.651343] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-04. [2025-10-20T17:59:27.012480] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-10-20T17:59:27.451371] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-09. [2025-10-20T17:59:28.251379] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-08. [2025-10-20T17:59:29.051427] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-05. [2025-10-20T17:59:29.114867] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T17:59:29.397940] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T17:59:29.846061] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T17:59:29.854512] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-06. [2025-10-20T17:59:30.450602] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T17:59:30.651429] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-07. [2025-10-20T17:59:31.084941] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T17:59:31.451461] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-08. [2025-10-20T17:59:32.251460] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-10. [2025-10-20T17:59:32.796882] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:59:33.051382] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-09. [2025-10-20T17:59:33.851399] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-08. [2025-10-20T17:59:33.898979] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T17:59:33.906939] [INFO ] [REMOTE ] CMA: Full fetch complete (73483 records). [2025-10-20T17:59:33.907103] [DEBUG] [VALIDATOR ] CMA: Validating primary data. [2025-10-20T17:59:34.299592] [DEBUG] [VALIDATOR ] CMA: All expected trading days are present. [2025-10-20T17:59:34.299695] [DEBUG] [VALIDATOR ] CMA: Primary data validation passed. [2025-10-20T17:59:34.300524] [INFO ] [MAIN ] CMA: Fetched primary data is valid. [2025-10-20T17:59:34.300568] [DEBUG] [LOCAL ] CMA: Saving primary data. [2025-10-20T17:59:34.301979] [DEBUG] [LOCAL ] CMA: Trimmed 73483 records to 50177 (keeping 200 days of data) [2025-10-20T17:59:34.538864] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:59:34.641581] [DEBUG] [LOCAL ] CMA: Primary data saved (50177 records). [2025-10-20T17:59:34.641868] [INFO ] [MAIN ] CMA: Full history saved. [2025-10-20T17:59:34.642028] [INFO ] [MAIN ] Update Progress: 471/2367 symbols completed. CMA took 35.6s. Estimated remaining: 3h 59m 32s. Rate: 7.9 symbols/min [2025-10-20T17:59:34.656555] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-09. [2025-10-20T17:59:34.850683] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T17:59:34.853608] [INFO ] [MAIN ] CMDB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:34.863436] [DEBUG] [LOCAL ] CMDB: No existing data file found in primary storage. [2025-10-20T17:59:34.864180] [WARN ] [MAIN ] CMDB: No existing data found, performing full refresh instead. [2025-10-20T17:59:34.864751] [INFO ] [MAIN ] CMDB: Wiping existing primary data before full refresh. [2025-10-20T17:59:34.864839] [INFO ] [LOCAL ] CMDB: Starting deletion of primary data. [2025-10-20T17:59:34.864929] [DEBUG] [LOCAL ] CMDB: No primary data file found to delete. [2025-10-20T17:59:34.865416] [INFO ] [MAIN ] CMDB: Starting full primary refresh. [2025-10-20T17:59:35.210510] [INFO ] [REMOTE ] CMDB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:35.451434] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-10. [2025-10-20T17:59:35.578739] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T17:59:36.251412] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-09. [2025-10-20T17:59:37.051370] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-10. [2025-10-20T17:59:37.214710] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:59:37.221664] [INFO ] [REMOTE ] CMC: Full fetch complete (67272 records). [2025-10-20T17:59:37.221783] [DEBUG] [VALIDATOR ] CMC: Validating primary data. [2025-10-20T17:59:37.623113] [DEBUG] [VALIDATOR ] CMC: All expected trading days are present. [2025-10-20T17:59:37.623501] [DEBUG] [VALIDATOR ] CMC: Primary data validation passed. [2025-10-20T17:59:37.624661] [INFO ] [MAIN ] CMC: Fetched primary data is valid. [2025-10-20T17:59:37.624731] [DEBUG] [LOCAL ] CMC: Saving primary data. [2025-10-20T17:59:37.626972] [DEBUG] [LOCAL ] CMC: Trimmed 67272 records to 45569 (keeping 200 days of data) [2025-10-20T17:59:37.730119] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T17:59:37.741458] [INFO ] [REMOTE ] CMCM: Full fetch complete (7026 records). [2025-10-20T17:59:37.741683] [DEBUG] [VALIDATOR ] CMCM: Validating primary data. [2025-10-20T17:59:37.911428] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-01. [2025-10-20T17:59:38.332466] [DEBUG] [REMOTE ] CMDB: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:59:38.334608] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-10-20T17:59:38.383213] [DEBUG] [VALIDATOR ] CMCM: All expected trading days are present. [2025-10-20T17:59:38.386065] [DEBUG] [VALIDATOR ] CMCM: Primary data validation passed. [2025-10-20T17:59:38.386486] [INFO ] [MAIN ] CMCM: Fetched primary data is valid. [2025-10-20T17:59:38.386625] [WARN ] [REMOTE ] CMDB: No data returned for month 2025-01 [2025-10-20T17:59:38.390412] [DEBUG] [LOCAL ] CMCM: Saving primary data. [2025-10-20T17:59:38.393115] [FATAL] [REMOTE ] CMDB: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:59:38.395295] [FATAL] [MAIN ] CMDB: Failed to fetch primary data. [2025-10-20T17:59:38.401915] [WARN ] [MAIN ] CMDB: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:59:38.403953] [DEBUG] [LOCAL ] CMC: Primary data saved (45569 records). [2025-10-20T17:59:38.401436] [DEBUG] [LOCAL ] CMCM: Trimmed 7026 records to 4827 (keeping 200 days of data) [2025-10-20T17:59:38.404152] [INFO ] [LOCAL ] CMDB: Starting deletion of primary data. [2025-10-20T17:59:38.404321] [INFO ] [MAIN ] CMC: Full history saved. [2025-10-20T17:59:38.405065] [DEBUG] [LOCAL ] CMDB: No primary data file found to delete. [2025-10-20T17:59:38.405210] [INFO ] [LOCAL ] CMDB: Starting deletion of derivative data. [2025-10-20T17:59:38.405669] [INFO ] [MAIN ] Update Progress: 472/2367 symbols completed. CMC took 30.8s. Estimated remaining: 3h 59m 9s. Rate: 7.9 symbols/min [2025-10-20T17:59:38.406231] [DEBUG] [LOCAL ] CMDB: No derivative data file found to delete. [2025-10-20T17:59:38.406485] [INFO ] [MAIN ] Update Progress: 473/2367 symbols completed. CMDB took 3.8s. Estimated remaining: 3h 58m 31s. Rate: 7.9 symbols/min [2025-10-20T17:59:38.583042] [DEBUG] [LOCAL ] CMCM: Primary data saved (4827 records). [2025-10-20T17:59:38.630990] [INFO ] [MAIN ] CMCM: Full history saved. [2025-10-20T17:59:38.657902] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-10. [2025-10-20T17:59:38.679361] [INFO ] [MAIN ] Update Progress: 474/2367 symbols completed. CMCM took 17.7s. Estimated remaining: 3h 57m 54s. Rate: 8.0 symbols/min [2025-10-20T17:59:38.904071] [INFO ] [MAIN ] CMG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:38.959989] [INFO ] [MAIN ] CMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:39.038010] [INFO ] [MAIN ] CMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:39.101623] [DEBUG] [LOCAL ] CMI: Loaded 42340 existing records from primary storage. [2025-10-20T17:59:39.102049] [INFO ] [MAIN ] CMI: Loaded 42340 existing records. [2025-10-20T17:59:39.235013] [DEBUG] [LOCAL ] CMG: Loaded 54501 existing records from primary storage. [2025-10-20T17:59:39.246701] [DEBUG] [LOCAL ] CMP: Loaded 36229 existing records from primary storage. [2025-10-20T17:59:39.280215] [INFO ] [MAIN ] CMG: Loaded 54501 existing records. [2025-10-20T17:59:39.286069] [INFO ] [MAIN ] CMP: Loaded 36229 existing records. [2025-10-20T17:59:39.317631] [WARN ] [MAIN ] CMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:39.430629] [INFO ] [MAIN ] CMI: Wiping existing primary data before full refresh. [2025-10-20T17:59:39.449001] [INFO ] [LOCAL ] CMI: Starting deletion of primary data. [2025-10-20T17:59:39.458806] [DEBUG] [LOCAL ] CMI: Deleted primary data file. [2025-10-20T17:59:39.495139] [INFO ] [MAIN ] CMI: Starting full primary refresh. [2025-10-20T17:59:39.908002] [WARN ] [MAIN ] CMG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:39.941395] [WARN ] [MAIN ] CMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:39.995167] [INFO ] [MAIN ] CMP: Wiping existing primary data before full refresh. [2025-10-20T17:59:39.995526] [INFO ] [LOCAL ] CMP: Starting deletion of primary data. [2025-10-20T17:59:39.995390] [INFO ] [MAIN ] CMG: Wiping existing primary data before full refresh. [2025-10-20T17:59:40.006063] [INFO ] [LOCAL ] CMG: Starting deletion of primary data. [2025-10-20T17:59:40.017883] [DEBUG] [LOCAL ] CMP: Deleted primary data file. [2025-10-20T17:59:40.023193] [INFO ] [MAIN ] CMP: Starting full primary refresh. [2025-10-20T17:59:40.160424] [DEBUG] [LOCAL ] CMG: Deleted primary data file. [2025-10-20T17:59:40.197160] [INFO ] [MAIN ] CMG: Starting full primary refresh. [2025-10-20T17:59:40.506084] [INFO ] [REMOTE ] CMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:40.541873] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-01. [2025-10-20T17:59:40.894021] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-10 (2.24 seconds). [2025-10-20T17:59:40.897615] [INFO ] [REMOTE ] CMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:40.931625] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-01. [2025-10-20T17:59:40.943335] [INFO ] [REMOTE ] CMBT: Full fetch complete (33365 records). [2025-10-20T17:59:40.959171] [DEBUG] [VALIDATOR ] CMBT: Validating primary data. [2025-10-20T17:59:41.107056] [INFO ] [REMOTE ] CMG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:41.121615] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-01. [2025-10-20T17:59:41.420846] [DEBUG] [VALIDATOR ] CMBT: All expected trading days are present. [2025-10-20T17:59:41.420973] [DEBUG] [VALIDATOR ] CMBT: Primary data validation passed. [2025-10-20T17:59:41.422328] [INFO ] [MAIN ] CMBT: Fetched primary data is valid. [2025-10-20T17:59:41.422401] [DEBUG] [LOCAL ] CMBT: Saving primary data. [2025-10-20T17:59:41.423804] [DEBUG] [LOCAL ] CMBT: Trimmed 33365 records to 27012 (keeping 200 days of data) [2025-10-20T17:59:41.612250] [DEBUG] [LOCAL ] CMBT: Primary data saved (27012 records). [2025-10-20T17:59:41.612503] [INFO ] [MAIN ] CMBT: Full history saved. [2025-10-20T17:59:41.612630] [INFO ] [MAIN ] Update Progress: 475/2367 symbols completed. CMBT took 39.4s. Estimated remaining: 3h 57m 29s. Rate: 8.0 symbols/min [2025-10-20T17:59:41.796896] [INFO ] [MAIN ] CMPO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:41.797174] [DEBUG] [LOCAL ] CMPO: No existing data file found in primary storage. [2025-10-20T17:59:41.797218] [WARN ] [MAIN ] CMPO: No existing data found, performing full refresh instead. [2025-10-20T17:59:41.797351] [INFO ] [MAIN ] CMPO: Wiping existing primary data before full refresh. [2025-10-20T17:59:41.797401] [INFO ] [LOCAL ] CMPO: Starting deletion of primary data. [2025-10-20T17:59:41.797446] [DEBUG] [LOCAL ] CMPO: No primary data file found to delete. [2025-10-20T17:59:41.797481] [INFO ] [MAIN ] CMPO: Starting full primary refresh. [2025-10-20T17:59:42.201273] [INFO ] [REMOTE ] CMPO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:42.201917] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-01. [2025-10-20T17:59:42.226966] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:59:42.439355] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T17:59:42.566376] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:59:42.651346] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-02. [2025-10-20T17:59:43.451337] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-02. [2025-10-20T17:59:43.640092] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:59:44.230272] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:59:44.251339] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-03. [2025-10-20T17:59:45.051341] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-02. [2025-10-20T17:59:45.211845] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T17:59:45.851436] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-02. [2025-10-20T17:59:45.977161] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T17:59:46.515004] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:59:46.651371] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-03. [2025-10-20T17:59:47.005787] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T17:59:47.451440] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-04. [2025-10-20T17:59:48.256538] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-03. [2025-10-20T17:59:48.269547] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:59:49.051375] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-04. [2025-10-20T17:59:49.142977] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:59:49.673965] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:59:49.851400] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-04. [2025-10-20T17:59:50.258571] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:59:50.651386] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-03. [2025-10-20T17:59:51.227543] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:59:51.451407] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-05. [2025-10-20T17:59:52.251478] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-05. [2025-10-20T17:59:52.427952] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T17:59:53.051351] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-04. [2025-10-20T17:59:53.166347] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:59:53.824114] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:59:53.851365] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-06. [2025-10-20T17:59:54.651397] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-05. [2025-10-20T17:59:54.841260] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:59:55.159502] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:59:55.451388] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-07. [2025-10-20T17:59:56.251387] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-06. [2025-10-20T17:59:56.334797] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:59:56.971409] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T17:59:57.051369] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-08. [2025-10-20T17:59:57.579303] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:59:57.851387] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-05. [2025-10-20T17:59:58.141456] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T17:59:58.651372] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-06. [2025-10-20T17:59:59.451352] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-09. [2025-10-20T17:59:59.637083] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T17:59:59.704997] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:00:00.251381] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-07. [2025-10-20T18:00:00.946882] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:00:01.051374] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-06. [2025-10-20T18:00:01.851361] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-07. [2025-10-20T18:00:01.927802] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T18:00:02.651357] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-08. [2025-10-20T18:00:02.966303] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T18:00:03.076722] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T18:00:03.451419] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-07. [2025-10-20T18:00:04.251400] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-10. [2025-10-20T18:00:04.369743] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:00:05.051444] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-08. [2025-10-20T18:00:05.636818] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T18:00:05.801103] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T18:00:05.806334] [INFO ] [REMOTE ] CMPO: Full fetch complete (58376 records). [2025-10-20T18:00:05.806487] [DEBUG] [VALIDATOR ] CMPO: Validating primary data. [2025-10-20T18:00:05.861534] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-09. [2025-10-20T18:00:06.191121] [DEBUG] [VALIDATOR ] CMPO: All expected trading days are present. [2025-10-20T18:00:06.191300] [DEBUG] [VALIDATOR ] CMPO: Primary data validation passed. [2025-10-20T18:00:06.192052] [INFO ] [MAIN ] CMPO: Fetched primary data is valid. [2025-10-20T18:00:06.192125] [DEBUG] [LOCAL ] CMPO: Saving primary data. [2025-10-20T18:00:06.193428] [DEBUG] [LOCAL ] CMPO: Trimmed 58376 records to 38711 (keeping 200 days of data) [2025-10-20T18:00:06.294507] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T18:00:06.438200] [DEBUG] [LOCAL ] CMPO: Primary data saved (38711 records). [2025-10-20T18:00:06.438342] [INFO ] [MAIN ] CMPO: Full history saved. [2025-10-20T18:00:06.438414] [INFO ] [MAIN ] Update Progress: 476/2367 symbols completed. CMPO took 24.8s. Estimated remaining: 3h 58m 30s. Rate: 7.9 symbols/min [2025-10-20T18:00:06.603196] [INFO ] [MAIN ] CMRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:06.634968] [DEBUG] [LOCAL ] CMRE: Loaded 36418 existing records from primary storage. [2025-10-20T18:00:06.635256] [INFO ] [MAIN ] CMRE: Loaded 36418 existing records. [2025-10-20T18:00:06.656788] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-08. [2025-10-20T18:00:06.815326] [WARN ] [MAIN ] CMRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:06.815555] [INFO ] [MAIN ] CMRE: Wiping existing primary data before full refresh. [2025-10-20T18:00:06.815609] [INFO ] [LOCAL ] CMRE: Starting deletion of primary data. [2025-10-20T18:00:06.816021] [DEBUG] [LOCAL ] CMRE: Deleted primary data file. [2025-10-20T18:00:06.816076] [INFO ] [MAIN ] CMRE: Starting full primary refresh. [2025-10-20T18:00:07.161204] [INFO ] [REMOTE ] CMRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:07.427226] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T18:00:07.451349] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-10. [2025-10-20T18:00:08.251441] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-01. [2025-10-20T18:00:08.992078] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T18:00:08.998398] [INFO ] [REMOTE ] CMI: Full fetch complete (60719 records). [2025-10-20T18:00:08.998547] [DEBUG] [VALIDATOR ] CMI: Validating primary data. [2025-10-20T18:00:08.999376] [FATAL] [VALIDATOR ] CMI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:00:08.999461] [FATAL] [MAIN ] CMI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:00:08.999502] [WARN ] [MAIN ] CMI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:00:08.999571] [INFO ] [LOCAL ] CMI: Starting deletion of primary data. [2025-10-20T18:00:08.999696] [DEBUG] [LOCAL ] CMI: No primary data file found to delete. [2025-10-20T18:00:08.999748] [INFO ] [LOCAL ] CMI: Starting deletion of derivative data. [2025-10-20T18:00:09.000138] [DEBUG] [LOCAL ] CMI: Deleted derivative data file. [2025-10-20T18:00:09.000227] [INFO ] [MAIN ] Update Progress: 477/2367 symbols completed. CMI took 30.6s. Estimated remaining: 3h 58m 2s. Rate: 7.9 symbols/min [2025-10-20T18:00:09.048533] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-08 (2.39 seconds). [2025-10-20T18:00:09.059266] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-09. [2025-10-20T18:00:09.305147] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T18:00:09.851478] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-02. [2025-10-20T18:00:10.499069] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T18:00:10.651395] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-10. [2025-10-20T18:00:10.984481] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:00:11.451373] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-03. [2025-10-20T18:00:11.709938] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:00:11.716094] [INFO ] [REMOTE ] CMP: Full fetch complete (51646 records). [2025-10-20T18:00:11.716233] [DEBUG] [VALIDATOR ] CMP: Validating primary data. [2025-10-20T18:00:12.129396] [DEBUG] [VALIDATOR ] CMP: All expected trading days are present. [2025-10-20T18:00:12.129484] [DEBUG] [VALIDATOR ] CMP: Primary data validation passed. [2025-10-20T18:00:12.130201] [INFO ] [MAIN ] CMP: Fetched primary data is valid. [2025-10-20T18:00:12.130270] [DEBUG] [LOCAL ] CMP: Saving primary data. [2025-10-20T18:00:12.131470] [DEBUG] [LOCAL ] CMP: Trimmed 51646 records to 35799 (keeping 200 days of data) [2025-10-20T18:00:12.252814] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-09. [2025-10-20T18:00:12.349015] [DEBUG] [LOCAL ] CMP: Primary data saved (35799 records). [2025-10-20T18:00:12.349130] [INFO ] [MAIN ] CMP: Full history saved. [2025-10-20T18:00:12.349220] [INFO ] [MAIN ] Update Progress: 478/2367 symbols completed. CMP took 33.7s. Estimated remaining: 3h 57m 38s. Rate: 7.9 symbols/min [2025-10-20T18:00:12.795384] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:00:13.051497] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-04. [2025-10-20T18:00:14.473103] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T18:00:14.482838] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-05. [2025-10-20T18:00:14.574117] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-09 (2.32 seconds). [2025-10-20T18:00:14.651418] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-10. [2025-10-20T18:00:15.494854] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:00:15.502341] [INFO ] [REMOTE ] CMG: Full fetch complete (77503 records). [2025-10-20T18:00:15.502493] [DEBUG] [VALIDATOR ] CMG: Validating primary data. [2025-10-20T18:00:15.503471] [FATAL] [VALIDATOR ] CMG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:00:15.503562] [FATAL] [MAIN ] CMG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:00:15.503604] [WARN ] [MAIN ] CMG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:00:15.503675] [INFO ] [LOCAL ] CMG: Starting deletion of primary data. [2025-10-20T18:00:15.503757] [DEBUG] [LOCAL ] CMG: No primary data file found to delete. [2025-10-20T18:00:15.503794] [INFO ] [LOCAL ] CMG: Starting deletion of derivative data. [2025-10-20T18:00:15.503947] [DEBUG] [LOCAL ] CMG: Deleted derivative data file. [2025-10-20T18:00:15.504034] [INFO ] [MAIN ] Update Progress: 479/2367 symbols completed. CMG took 37.1s. Estimated remaining: 3h 57m 13s. Rate: 8.0 symbols/min [2025-10-20T18:00:15.920707] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:00:15.928521] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-06. [2025-10-20T18:00:17.340606] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:00:17.348696] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-07. [2025-10-20T18:00:18.879214] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T18:00:18.887438] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-08. [2025-10-20T18:00:20.468882] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T18:00:20.477052] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-09. [2025-10-20T18:00:21.756469] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:00:21.765930] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-10. [2025-10-20T18:00:22.922282] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T18:00:22.930881] [INFO ] [REMOTE ] CMRE: Full fetch complete (51327 records). [2025-10-20T18:00:22.931085] [DEBUG] [VALIDATOR ] CMRE: Validating primary data. [2025-10-20T18:00:23.368818] [DEBUG] [VALIDATOR ] CMRE: All expected trading days are present. [2025-10-20T18:00:23.368926] [DEBUG] [VALIDATOR ] CMRE: Primary data validation passed. [2025-10-20T18:00:23.369582] [INFO ] [MAIN ] CMRE: Fetched primary data is valid. [2025-10-20T18:00:23.369626] [DEBUG] [LOCAL ] CMRE: Saving primary data. [2025-10-20T18:00:23.370752] [DEBUG] [LOCAL ] CMRE: Trimmed 51327 records to 35844 (keeping 200 days of data) [2025-10-20T18:00:23.575635] [DEBUG] [LOCAL ] CMRE: Primary data saved (35844 records). [2025-10-20T18:00:23.575802] [INFO ] [MAIN ] CMRE: Full history saved. [2025-10-20T18:00:23.575880] [INFO ] [MAIN ] Update Progress: 480/2367 symbols completed. CMRE took 17.1s. Estimated remaining: 3h 57m 8s. Rate: 8.0 symbols/min [2025-10-20T18:00:24.331745] [INFO ] [MAIN ] CMSA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:24.336276] [INFO ] [MAIN ] CMSC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:24.363726] [INFO ] [MAIN ] CMSD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:24.412221] [INFO ] [MAIN ] CMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:24.412439] [DEBUG] [LOCAL ] CMS: No existing data file found in primary storage. [2025-10-20T18:00:24.418701] [DEBUG] [LOCAL ] CMSA: Loaded 3570 existing records from primary storage. [2025-10-20T18:00:24.413841] [WARN ] [MAIN ] CMS: No existing data found, performing full refresh instead. [2025-10-20T18:00:24.419021] [INFO ] [MAIN ] CMSA: Loaded 3570 existing records. [2025-10-20T18:00:24.422263] [INFO ] [MAIN ] CMS: Wiping existing primary data before full refresh. [2025-10-20T18:00:24.422637] [DEBUG] [LOCAL ] CMSC: Loaded 4946 existing records from primary storage. [2025-10-20T18:00:24.422452] [INFO ] [LOCAL ] CMS: Starting deletion of primary data. [2025-10-20T18:00:24.443874] [INFO ] [MAIN ] CMSC: Loaded 4946 existing records. [2025-10-20T18:00:24.463877] [DEBUG] [LOCAL ] CMS: No primary data file found to delete. [2025-10-20T18:00:24.502277] [INFO ] [MAIN ] CMS: Starting full primary refresh. [2025-10-20T18:00:24.557332] [DEBUG] [LOCAL ] CMSD: Loaded 7921 existing records from primary storage. [2025-10-20T18:00:24.640469] [INFO ] [MAIN ] CMSD: Loaded 7921 existing records. [2025-10-20T18:00:25.027032] [WARN ] [MAIN ] CMSC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:25.079868] [WARN ] [MAIN ] CMSA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:25.117285] [INFO ] [MAIN ] CMSA: Wiping existing primary data before full refresh. [2025-10-20T18:00:25.127827] [INFO ] [MAIN ] CMSC: Wiping existing primary data before full refresh. [2025-10-20T18:00:25.148384] [WARN ] [MAIN ] CMSD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:25.153774] [INFO ] [MAIN ] CMSD: Wiping existing primary data before full refresh. [2025-10-20T18:00:25.142472] [INFO ] [LOCAL ] CMSA: Starting deletion of primary data. [2025-10-20T18:00:25.154047] [INFO ] [LOCAL ] CMSC: Starting deletion of primary data. [2025-10-20T18:00:25.154011] [INFO ] [LOCAL ] CMSD: Starting deletion of primary data. [2025-10-20T18:00:25.178920] [DEBUG] [LOCAL ] CMSA: Deleted primary data file. [2025-10-20T18:00:25.186133] [INFO ] [MAIN ] CMSA: Starting full primary refresh. [2025-10-20T18:00:25.215674] [DEBUG] [LOCAL ] CMSD: Deleted primary data file. [2025-10-20T18:00:25.195611] [DEBUG] [LOCAL ] CMSC: Deleted primary data file. [2025-10-20T18:00:25.225494] [INFO ] [MAIN ] CMSD: Starting full primary refresh. [2025-10-20T18:00:25.239436] [INFO ] [MAIN ] CMSC: Starting full primary refresh. [2025-10-20T18:00:25.887399] [INFO ] [REMOTE ] CMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:26.045712] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-01. [2025-10-20T18:00:26.372918] [INFO ] [REMOTE ] CMSA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:26.412603] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-01. [2025-10-20T18:00:26.419391] [INFO ] [REMOTE ] CMSD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:26.425793] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-01. [2025-10-20T18:00:26.512927] [INFO ] [REMOTE ] CMSC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:26.513272] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-01. [2025-10-20T18:00:27.141307] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T18:00:27.143173] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-02. [2025-10-20T18:00:27.218277] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T18:00:27.220077] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-02. [2025-10-20T18:00:27.356239] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T18:00:27.358958] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-02. [2025-10-20T18:00:27.725966] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T18:00:27.727538] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-03. [2025-10-20T18:00:27.735058] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T18:00:27.736378] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-03. [2025-10-20T18:00:27.937184] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T18:00:27.939110] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-03. [2025-10-20T18:00:28.054202] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T18:00:28.064293] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-02. [2025-10-20T18:00:28.362788] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:00:28.364543] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-04. [2025-10-20T18:00:28.483626] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T18:00:28.596828] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T18:00:28.719051] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T18:00:29.051431] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-05. [2025-10-20T18:00:29.606910] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T18:00:29.790538] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T18:00:29.851398] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-06. [2025-10-20T18:00:30.190785] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-10-20T18:00:30.651364] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-03. [2025-10-20T18:00:31.451433] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-04. [2025-10-20T18:00:31.863976] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T18:00:32.251428] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-04. [2025-10-20T18:00:32.382847] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T18:00:32.751877] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T18:00:33.051386] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-04. [2025-10-20T18:00:33.851424] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-07. [2025-10-20T18:00:34.457544] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:00:34.651410] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-05. [2025-10-20T18:00:34.994990] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T18:00:35.285064] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T18:00:35.451421] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-05. [2025-10-20T18:00:36.112480] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T18:00:36.251387] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-06. [2025-10-20T18:00:36.604397] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-10-20T18:00:37.051435] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-08. [2025-10-20T18:00:37.744605] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T18:00:37.851393] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-07. [2025-10-20T18:00:38.586162] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:00:38.651353] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-06. [2025-10-20T18:00:39.451354] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-08. [2025-10-20T18:00:39.512638] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T18:00:39.830231] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-10-20T18:00:40.251431] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-05. [2025-10-20T18:00:41.051368] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-09. [2025-10-20T18:00:41.613109] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T18:00:41.838431] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T18:00:41.851340] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-06. [2025-10-20T18:00:42.651348] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-09. [2025-10-20T18:00:43.146457] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T18:00:43.223998] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T18:00:43.451379] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-10. [2025-10-20T18:00:44.096952] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T18:00:44.098037] [INFO ] [REMOTE ] CMSA: Full fetch complete (4873 records). [2025-10-20T18:00:44.098108] [DEBUG] [VALIDATOR ] CMSA: Validating primary data. [2025-10-20T18:00:44.256516] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-10. [2025-10-20T18:00:44.475722] [DEBUG] [VALIDATOR ] CMSA: All expected trading days are present. [2025-10-20T18:00:44.475814] [DEBUG] [VALIDATOR ] CMSA: Primary data validation passed. [2025-10-20T18:00:44.475960] [INFO ] [MAIN ] CMSA: Fetched primary data is valid. [2025-10-20T18:00:44.475999] [DEBUG] [LOCAL ] CMSA: Saving primary data. [2025-10-20T18:00:44.476364] [DEBUG] [LOCAL ] CMSA: Trimmed 4873 records to 3515 (keeping 200 days of data) [2025-10-20T18:00:44.497745] [DEBUG] [LOCAL ] CMSA: Primary data saved (3515 records). [2025-10-20T18:00:44.497882] [INFO ] [MAIN ] CMSA: Full history saved. [2025-10-20T18:00:44.497963] [INFO ] [MAIN ] Update Progress: 481/2367 symbols completed. CMSA took 20.9s. Estimated remaining: 3h 57m 53s. Rate: 7.9 symbols/min [2025-10-20T18:00:44.665057] [INFO ] [MAIN ] CMTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:44.699857] [DEBUG] [LOCAL ] CMTG: Loaded 36246 existing records from primary storage. [2025-10-20T18:00:44.700157] [INFO ] [MAIN ] CMTG: Loaded 36246 existing records. [2025-10-20T18:00:44.877069] [WARN ] [MAIN ] CMTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:44.877296] [INFO ] [MAIN ] CMTG: Wiping existing primary data before full refresh. [2025-10-20T18:00:44.877344] [INFO ] [LOCAL ] CMTG: Starting deletion of primary data. [2025-10-20T18:00:44.877565] [DEBUG] [LOCAL ] CMTG: Deleted primary data file. [2025-10-20T18:00:44.877621] [INFO ] [MAIN ] CMTG: Starting full primary refresh. [2025-10-20T18:00:45.001035] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T18:00:45.036930] [INFO ] [REMOTE ] CMSC: Full fetch complete (6965 records). [2025-10-20T18:00:45.058127] [DEBUG] [VALIDATOR ] CMSC: Validating primary data. [2025-10-20T18:00:45.056126] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-07. [2025-10-20T18:00:45.464046] [INFO ] [REMOTE ] CMTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:45.713794] [DEBUG] [VALIDATOR ] CMSC: All expected trading days are present. [2025-10-20T18:00:45.713922] [DEBUG] [VALIDATOR ] CMSC: Primary data validation passed. [2025-10-20T18:00:45.714301] [INFO ] [MAIN ] CMSC: Fetched primary data is valid. [2025-10-20T18:00:45.714344] [DEBUG] [LOCAL ] CMSC: Saving primary data. [2025-10-20T18:00:45.714914] [DEBUG] [LOCAL ] CMSC: Trimmed 6965 records to 4847 (keeping 200 days of data) [2025-10-20T18:00:45.747013] [DEBUG] [LOCAL ] CMSC: Primary data saved (4847 records). [2025-10-20T18:00:45.747199] [INFO ] [MAIN ] CMSC: Full history saved. [2025-10-20T18:00:45.747305] [INFO ] [MAIN ] Update Progress: 482/2367 symbols completed. CMSC took 22.2s. Estimated remaining: 3h 57m 21s. Rate: 7.9 symbols/min [2025-10-20T18:00:45.853258] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-01. [2025-10-20T18:00:45.927614] [INFO ] [MAIN ] CMU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:45.936897] [DEBUG] [LOCAL ] CMU: Loaded 6381 existing records from primary storage. [2025-10-20T18:00:45.937248] [INFO ] [MAIN ] CMU: Loaded 6381 existing records. [2025-10-20T18:00:46.125745] [WARN ] [MAIN ] CMU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:46.126055] [INFO ] [MAIN ] CMU: Wiping existing primary data before full refresh. [2025-10-20T18:00:46.126112] [INFO ] [LOCAL ] CMU: Starting deletion of primary data. [2025-10-20T18:00:46.126370] [DEBUG] [LOCAL ] CMU: Deleted primary data file. [2025-10-20T18:00:46.126443] [INFO ] [MAIN ] CMU: Starting full primary refresh. [2025-10-20T18:00:46.506636] [INFO ] [REMOTE ] CMU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:46.651453] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-07. [2025-10-20T18:00:46.764177] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T18:00:47.286075] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-10-20T18:00:47.452065] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-08. [2025-10-20T18:00:47.454369] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T18:00:47.929668] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T18:00:48.251355] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-08. [2025-10-20T18:00:49.051420] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-01. [2025-10-20T18:00:49.640802] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T18:00:49.773350] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T18:00:49.851366] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-09. [2025-10-20T18:00:50.651393] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-02. [2025-10-20T18:00:51.237137] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T18:00:51.451377] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-02. [2025-10-20T18:00:51.527190] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:00:52.251404] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-03. [2025-10-20T18:00:52.756495] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T18:00:53.019913] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T18:00:53.051349] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-09. [2025-10-20T18:00:53.685992] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T18:00:53.851418] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-10. [2025-10-20T18:00:54.651374] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-03. [2025-10-20T18:00:55.451362] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-04. [2025-10-20T18:00:55.705320] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:00:55.713300] [INFO ] [REMOTE ] CMS: Full fetch complete (76238 records). [2025-10-20T18:00:55.713494] [DEBUG] [VALIDATOR ] CMS: Validating primary data. [2025-10-20T18:00:56.146486] [DEBUG] [VALIDATOR ] CMS: All expected trading days are present. [2025-10-20T18:00:56.146683] [DEBUG] [VALIDATOR ] CMS: Primary data validation passed. [2025-10-20T18:00:56.147714] [INFO ] [MAIN ] CMS: Fetched primary data is valid. [2025-10-20T18:00:56.147807] [DEBUG] [LOCAL ] CMS: Saving primary data. [2025-10-20T18:00:56.149552] [DEBUG] [LOCAL ] CMS: Trimmed 76238 records to 52585 (keeping 200 days of data) [2025-10-20T18:00:56.252819] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-10. [2025-10-20T18:00:56.268788] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T18:00:56.304526] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:00:56.530843] [DEBUG] [LOCAL ] CMS: Primary data saved (52585 records). [2025-10-20T18:00:56.530983] [INFO ] [MAIN ] CMS: Full history saved. [2025-10-20T18:00:56.531053] [INFO ] [MAIN ] Update Progress: 483/2367 symbols completed. CMS took 33.0s. Estimated remaining: 3h 57m 26s. Rate: 7.9 symbols/min [2025-10-20T18:00:56.703018] [INFO ] [MAIN ] CNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:56.742301] [DEBUG] [LOCAL ] CNA: Loaded 33357 existing records from primary storage. [2025-10-20T18:00:56.742734] [INFO ] [MAIN ] CNA: Loaded 33357 existing records. [2025-10-20T18:00:56.939618] [WARN ] [MAIN ] CNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:56.939949] [INFO ] [MAIN ] CNA: Wiping existing primary data before full refresh. [2025-10-20T18:00:56.940012] [INFO ] [LOCAL ] CNA: Starting deletion of primary data. [2025-10-20T18:00:56.940465] [DEBUG] [LOCAL ] CNA: Deleted primary data file. [2025-10-20T18:00:56.940506] [INFO ] [MAIN ] CNA: Starting full primary refresh. [2025-10-20T18:00:57.032417] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T18:00:57.056944] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-04. [2025-10-20T18:00:57.087935] [INFO ] [REMOTE ] CMSD: Full fetch complete (11510 records). [2025-10-20T18:00:57.150692] [DEBUG] [VALIDATOR ] CMSD: Validating primary data. [2025-10-20T18:00:57.487756] [INFO ] [REMOTE ] CNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:57.708303] [DEBUG] [VALIDATOR ] CMSD: All expected trading days are present. [2025-10-20T18:00:57.708382] [DEBUG] [VALIDATOR ] CMSD: Primary data validation passed. [2025-10-20T18:00:57.708707] [INFO ] [MAIN ] CMSD: Fetched primary data is valid. [2025-10-20T18:00:57.708768] [DEBUG] [LOCAL ] CMSD: Saving primary data. [2025-10-20T18:00:57.709237] [DEBUG] [LOCAL ] CMSD: Trimmed 11510 records to 7682 (keeping 200 days of data) [2025-10-20T18:00:57.754229] [DEBUG] [LOCAL ] CMSD: Primary data saved (7682 records). [2025-10-20T18:00:57.754337] [INFO ] [MAIN ] CMSD: Full history saved. [2025-10-20T18:00:57.754411] [INFO ] [MAIN ] Update Progress: 484/2367 symbols completed. CMSD took 34.2s. Estimated remaining: 3h 56m 53s. Rate: 7.9 symbols/min [2025-10-20T18:00:57.853876] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-05. [2025-10-20T18:00:57.921159] [INFO ] [MAIN ] CNC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:57.968342] [DEBUG] [LOCAL ] CNC: Loaded 54550 existing records from primary storage. [2025-10-20T18:00:57.968692] [INFO ] [MAIN ] CNC: Loaded 54550 existing records. [2025-10-20T18:00:58.144978] [WARN ] [MAIN ] CNC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:58.145168] [INFO ] [MAIN ] CNC: Wiping existing primary data before full refresh. [2025-10-20T18:00:58.145213] [INFO ] [LOCAL ] CNC: Starting deletion of primary data. [2025-10-20T18:00:58.145509] [DEBUG] [LOCAL ] CNC: Deleted primary data file. [2025-10-20T18:00:58.145546] [INFO ] [MAIN ] CNC: Starting full primary refresh. [2025-10-20T18:00:58.250931] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-10-20T18:00:58.503204] [INFO ] [REMOTE ] CNC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:58.651342] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-01. [2025-10-20T18:00:58.709909] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:00:59.451376] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-06. [2025-10-20T18:00:59.868316] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T18:00:59.990216] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:01:00.251365] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-05. [2025-10-20T18:01:01.051382] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-07. [2025-10-20T18:01:01.401903] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T18:01:01.851420] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-06. [2025-10-20T18:01:01.904131] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T18:01:02.651431] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-01. [2025-10-20T18:01:03.247436] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:01:03.451445] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-07. [2025-10-20T18:01:04.155177] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:01:04.251399] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-02. [2025-10-20T18:01:04.867161] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T18:01:05.051361] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-02. [2025-10-20T18:01:05.228177] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:01:05.851411] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-08. [2025-10-20T18:01:06.371955] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:01:06.470007] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T18:01:06.651359] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-08. [2025-10-20T18:01:07.451485] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-09. [2025-10-20T18:01:07.917925] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:01:08.078819] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T18:01:08.251473] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-03. [2025-10-20T18:01:09.051501] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-03. [2025-10-20T18:01:09.724878] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:01:09.851365] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-04. [2025-10-20T18:01:10.589142] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T18:01:10.651417] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-10. [2025-10-20T18:01:11.410863] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T18:01:11.411983] [INFO ] [REMOTE ] CMU: Full fetch complete (9484 records). [2025-10-20T18:01:11.412052] [DEBUG] [VALIDATOR ] CMU: Validating primary data. [2025-10-20T18:01:11.464666] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-04. [2025-10-20T18:01:11.800766] [DEBUG] [VALIDATOR ] CMU: All expected trading days are present. [2025-10-20T18:01:11.800892] [DEBUG] [VALIDATOR ] CMU: Primary data validation passed. [2025-10-20T18:01:11.801319] [INFO ] [MAIN ] CMU: Fetched primary data is valid. [2025-10-20T18:01:11.801444] [DEBUG] [LOCAL ] CMU: Saving primary data. [2025-10-20T18:01:11.802522] [DEBUG] [LOCAL ] CMU: Trimmed 9484 records to 6357 (keeping 200 days of data) [2025-10-20T18:01:11.877795] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T18:01:11.890313] [DEBUG] [LOCAL ] CMU: Primary data saved (6357 records). [2025-10-20T18:01:11.890918] [INFO ] [MAIN ] CMU: Full history saved. [2025-10-20T18:01:11.891150] [INFO ] [MAIN ] Update Progress: 485/2367 symbols completed. CMU took 26.1s. Estimated remaining: 3h 57m 11s. Rate: 7.9 symbols/min [2025-10-20T18:01:12.082598] [INFO ] [MAIN ] CNF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:12.092953] [DEBUG] [LOCAL ] CNF: Loaded 10666 existing records from primary storage. [2025-10-20T18:01:12.093182] [INFO ] [MAIN ] CNF: Loaded 10666 existing records. [2025-10-20T18:01:12.256927] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-09. [2025-10-20T18:01:12.289287] [WARN ] [MAIN ] CNF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:12.289506] [INFO ] [MAIN ] CNF: Wiping existing primary data before full refresh. [2025-10-20T18:01:12.289555] [INFO ] [LOCAL ] CNF: Starting deletion of primary data. [2025-10-20T18:01:12.289734] [DEBUG] [LOCAL ] CNF: Deleted primary data file. [2025-10-20T18:01:12.289791] [INFO ] [MAIN ] CNF: Starting full primary refresh. [2025-10-20T18:01:12.671073] [INFO ] [REMOTE ] CNF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:12.804209] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:01:13.051418] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-05. [2025-10-20T18:01:13.256386] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T18:01:13.851412] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-01. [2025-10-20T18:01:14.328106] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T18:01:14.494317] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T18:01:14.651411] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-05. [2025-10-20T18:01:15.451432] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-02. [2025-10-20T18:01:15.852427] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-10-20T18:01:16.251428] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-10. [2025-10-20T18:01:16.820938] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T18:01:17.051382] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-06. [2025-10-20T18:01:17.515102] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:01:17.519487] [INFO ] [REMOTE ] CMTG: Full fetch complete (53040 records). [2025-10-20T18:01:17.519582] [DEBUG] [VALIDATOR ] CMTG: Validating primary data. [2025-10-20T18:01:17.520251] [FATAL] [VALIDATOR ] CMTG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:01:17.520337] [FATAL] [MAIN ] CMTG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:01:17.520380] [WARN ] [MAIN ] CMTG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:01:17.520448] [INFO ] [LOCAL ] CMTG: Starting deletion of primary data. [2025-10-20T18:01:17.520516] [DEBUG] [LOCAL ] CMTG: No primary data file found to delete. [2025-10-20T18:01:17.520553] [INFO ] [LOCAL ] CMTG: Starting deletion of derivative data. [2025-10-20T18:01:17.520707] [DEBUG] [LOCAL ] CMTG: Deleted derivative data file. [2025-10-20T18:01:17.520795] [INFO ] [MAIN ] Update Progress: 486/2367 symbols completed. CMTG took 33.0s. Estimated remaining: 3h 56m 56s. Rate: 7.9 symbols/min [2025-10-20T18:01:17.685145] [INFO ] [MAIN ] CNH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:17.736918] [DEBUG] [LOCAL ] CNH: Loaded 54332 existing records from primary storage. [2025-10-20T18:01:17.737310] [INFO ] [MAIN ] CNH: Loaded 54332 existing records. [2025-10-20T18:01:17.856547] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-06. [2025-10-20T18:01:17.913571] [WARN ] [MAIN ] CNH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:17.913961] [INFO ] [MAIN ] CNH: Wiping existing primary data before full refresh. [2025-10-20T18:01:17.914027] [INFO ] [LOCAL ] CNH: Starting deletion of primary data. [2025-10-20T18:01:17.914323] [DEBUG] [LOCAL ] CNH: Deleted primary data file. [2025-10-20T18:01:17.914378] [INFO ] [MAIN ] CNH: Starting full primary refresh. [2025-10-20T18:01:18.265143] [INFO ] [REMOTE ] CNH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:18.357944] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:01:18.651421] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-07. [2025-10-20T18:01:19.460981] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-01. [2025-10-20T18:01:19.465236] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:01:20.251397] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-03. [2025-10-20T18:01:20.807664] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T18:01:21.028045] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:01:21.051331] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-02. [2025-10-20T18:01:21.060863] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T18:01:21.851398] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-04. [2025-10-20T18:01:22.430736] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T18:01:22.557495] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T18:01:22.651393] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-03. [2025-10-20T18:01:23.451381] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-07. [2025-10-20T18:01:24.251365] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-05. [2025-10-20T18:01:24.500575] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T18:01:24.907728] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T18:01:24.945482] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:01:25.051413] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-08. [2025-10-20T18:01:25.851383] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-08. [2025-10-20T18:01:26.651391] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-04. [2025-10-20T18:01:26.901386] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T18:01:26.938058] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T18:01:27.451397] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-09. [2025-10-20T18:01:28.251408] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-06. [2025-10-20T18:01:28.519671] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T18:01:28.774578] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T18:01:29.051425] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-10. [2025-10-20T18:01:29.178989] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T18:01:29.851441] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-05. [2025-10-20T18:01:30.169131] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T18:01:30.175098] [INFO ] [REMOTE ] CNA: Full fetch complete (46129 records). [2025-10-20T18:01:30.175291] [DEBUG] [VALIDATOR ] CNA: Validating primary data. [2025-10-20T18:01:30.560828] [DEBUG] [VALIDATOR ] CNA: All expected trading days are present. [2025-10-20T18:01:30.560995] [DEBUG] [VALIDATOR ] CNA: Primary data validation passed. [2025-10-20T18:01:30.561713] [INFO ] [MAIN ] CNA: Fetched primary data is valid. [2025-10-20T18:01:30.561776] [DEBUG] [LOCAL ] CNA: Saving primary data. [2025-10-20T18:01:30.562883] [DEBUG] [LOCAL ] CNA: Trimmed 46129 records to 32763 (keeping 200 days of data) [2025-10-20T18:01:30.655791] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-09. [2025-10-20T18:01:30.762784] [DEBUG] [LOCAL ] CNA: Primary data saved (32763 records). [2025-10-20T18:01:30.762931] [INFO ] [MAIN ] CNA: Full history saved. [2025-10-20T18:01:30.763013] [INFO ] [MAIN ] Update Progress: 487/2367 symbols completed. CNA took 34.2s. Estimated remaining: 3h 57m 11s. Rate: 7.9 symbols/min [2025-10-20T18:01:30.954815] [INFO ] [MAIN ] CNI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:31.005540] [DEBUG] [LOCAL ] CNI: Loaded 52456 existing records from primary storage. [2025-10-20T18:01:31.005989] [INFO ] [MAIN ] CNI: Loaded 52456 existing records. [2025-10-20T18:01:31.196114] [WARN ] [MAIN ] CNI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:31.196339] [INFO ] [MAIN ] CNI: Wiping existing primary data before full refresh. [2025-10-20T18:01:31.196411] [INFO ] [LOCAL ] CNI: Starting deletion of primary data. [2025-10-20T18:01:31.196862] [DEBUG] [LOCAL ] CNI: Deleted primary data file. [2025-10-20T18:01:31.196934] [INFO ] [MAIN ] CNI: Starting full primary refresh. [2025-10-20T18:01:31.456148] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-07. [2025-10-20T18:01:31.613080] [INFO ] [REMOTE ] CNI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:31.751057] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T18:01:32.251414] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-06. [2025-10-20T18:01:32.260451] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:01:32.597046] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T18:01:33.051363] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-08. [2025-10-20T18:01:33.655899] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T18:01:33.851345] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-09. [2025-10-20T18:01:33.930951] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T18:01:34.513065] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-10-20T18:01:34.651343] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-10. [2025-10-20T18:01:35.451344] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-10. [2025-10-20T18:01:35.635528] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T18:01:35.637029] [INFO ] [REMOTE ] CNF: Full fetch complete (13022 records). [2025-10-20T18:01:35.637100] [DEBUG] [VALIDATOR ] CNF: Validating primary data. [2025-10-20T18:01:35.637487] [FATAL] [VALIDATOR ] CNF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:01:35.637570] [FATAL] [MAIN ] CNF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:01:35.637620] [WARN ] [MAIN ] CNF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:01:35.637736] [INFO ] [LOCAL ] CNF: Starting deletion of primary data. [2025-10-20T18:01:35.637816] [DEBUG] [LOCAL ] CNF: No primary data file found to delete. [2025-10-20T18:01:35.637863] [INFO ] [LOCAL ] CNF: Starting deletion of derivative data. [2025-10-20T18:01:35.638058] [DEBUG] [LOCAL ] CNF: Deleted derivative data file. [2025-10-20T18:01:35.638158] [INFO ] [MAIN ] Update Progress: 488/2367 symbols completed. CNF took 23.7s. Estimated remaining: 3h 56m 53s. Rate: 7.9 symbols/min [2025-10-20T18:01:35.833313] [INFO ] [MAIN ] CNK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:35.879948] [DEBUG] [LOCAL ] CNK: Loaded 53677 existing records from primary storage. [2025-10-20T18:01:35.880266] [INFO ] [MAIN ] CNK: Loaded 53677 existing records. [2025-10-20T18:01:36.086958] [WARN ] [MAIN ] CNK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:36.087212] [INFO ] [MAIN ] CNK: Wiping existing primary data before full refresh. [2025-10-20T18:01:36.087269] [INFO ] [LOCAL ] CNK: Starting deletion of primary data. [2025-10-20T18:01:36.087753] [DEBUG] [LOCAL ] CNK: Deleted primary data file. [2025-10-20T18:01:36.087810] [INFO ] [MAIN ] CNK: Starting full primary refresh. [2025-10-20T18:01:36.260541] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-07. [2025-10-20T18:01:36.462361] [INFO ] [REMOTE ] CNK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:36.485717] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:01:36.494162] [INFO ] [REMOTE ] CNC: Full fetch complete (77850 records). [2025-10-20T18:01:36.494327] [DEBUG] [VALIDATOR ] CNC: Validating primary data. [2025-10-20T18:01:36.915958] [DEBUG] [VALIDATOR ] CNC: All expected trading days are present. [2025-10-20T18:01:36.916039] [DEBUG] [VALIDATOR ] CNC: Primary data validation passed. [2025-10-20T18:01:36.916869] [INFO ] [MAIN ] CNC: Fetched primary data is valid. [2025-10-20T18:01:36.916931] [DEBUG] [LOCAL ] CNC: Saving primary data. [2025-10-20T18:01:36.918351] [DEBUG] [LOCAL ] CNC: Trimmed 77850 records to 53778 (keeping 200 days of data) [2025-10-20T18:01:37.084229] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-01. [2025-10-20T18:01:37.276566] [DEBUG] [LOCAL ] CNC: Primary data saved (53778 records). [2025-10-20T18:01:37.276813] [INFO ] [MAIN ] CNC: Full history saved. [2025-10-20T18:01:37.276923] [INFO ] [MAIN ] Update Progress: 489/2367 symbols completed. CNC took 39.5s. Estimated remaining: 3h 56m 22s. Rate: 7.9 symbols/min [2025-10-20T18:01:37.468489] [INFO ] [MAIN ] CNM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:37.523978] [DEBUG] [LOCAL ] CNM: Loaded 51797 existing records from primary storage. [2025-10-20T18:01:37.524405] [INFO ] [MAIN ] CNM: Loaded 51797 existing records. [2025-10-20T18:01:37.715216] [WARN ] [MAIN ] CNM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:37.715547] [INFO ] [MAIN ] CNM: Wiping existing primary data before full refresh. [2025-10-20T18:01:37.715613] [INFO ] [LOCAL ] CNM: Starting deletion of primary data. [2025-10-20T18:01:37.715983] [DEBUG] [LOCAL ] CNM: Deleted primary data file. [2025-10-20T18:01:37.716051] [INFO ] [MAIN ] CNM: Starting full primary refresh. [2025-10-20T18:01:37.866756] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-01. [2025-10-20T18:01:38.108296] [INFO ] [REMOTE ] CNM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:38.212521] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T18:01:38.613183] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T18:01:38.651404] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-08. [2025-10-20T18:01:39.451396] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-02. [2025-10-20T18:01:39.821616] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T18:01:40.251478] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-01. [2025-10-20T18:01:40.382079] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:01:41.058292] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-02. [2025-10-20T18:01:41.065117] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:01:41.851386] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-03. [2025-10-20T18:01:42.004726] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T18:01:42.651493] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-02. [2025-10-20T18:01:42.769294] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T18:01:43.451372] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-03. [2025-10-20T18:01:43.849804] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T18:01:44.193600] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:01:44.251406] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-04. [2025-10-20T18:01:44.919791] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:01:45.051411] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-04. [2025-10-20T18:01:45.851384] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-03. [2025-10-20T18:01:45.994521] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T18:01:46.651380] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-09. [2025-10-20T18:01:46.875424] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T18:01:47.075565] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T18:01:47.451386] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-04. [2025-10-20T18:01:48.251401] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-05. [2025-10-20T18:01:48.574791] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T18:01:49.051392] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-05. [2025-10-20T18:01:49.224049] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:01:49.851385] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-10. [2025-10-20T18:01:50.098098] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:01:50.651386] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-05. [2025-10-20T18:01:50.912518] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T18:01:51.451348] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-06. [2025-10-20T18:01:51.597469] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T18:01:51.605340] [INFO ] [REMOTE ] CNH: Full fetch complete (77704 records). [2025-10-20T18:01:51.605472] [DEBUG] [VALIDATOR ] CNH: Validating primary data. [2025-10-20T18:01:52.000491] [DEBUG] [VALIDATOR ] CNH: All expected trading days are present. [2025-10-20T18:01:52.000588] [DEBUG] [VALIDATOR ] CNH: Primary data validation passed. [2025-10-20T18:01:52.001478] [INFO ] [MAIN ] CNH: Fetched primary data is valid. [2025-10-20T18:01:52.001540] [DEBUG] [LOCAL ] CNH: Saving primary data. [2025-10-20T18:01:52.003058] [DEBUG] [LOCAL ] CNH: Trimmed 77704 records to 53548 (keeping 200 days of data) [2025-10-20T18:01:52.256796] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-06. [2025-10-20T18:01:52.380239] [DEBUG] [LOCAL ] CNH: Primary data saved (53548 records). [2025-10-20T18:01:52.384060] [INFO ] [MAIN ] CNH: Full history saved. [2025-10-20T18:01:52.384216] [INFO ] [MAIN ] Update Progress: 490/2367 symbols completed. CNH took 34.9s. Estimated remaining: 3h 56m 44s. Rate: 7.9 symbols/min [2025-10-20T18:01:52.456631] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T18:01:52.570916] [INFO ] [MAIN ] CNMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:52.601273] [DEBUG] [LOCAL ] CNMD: Loaded 32497 existing records from primary storage. [2025-10-20T18:01:52.601577] [INFO ] [MAIN ] CNMD: Loaded 32497 existing records. [2025-10-20T18:01:52.778832] [WARN ] [MAIN ] CNMD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:52.779098] [INFO ] [MAIN ] CNMD: Wiping existing primary data before full refresh. [2025-10-20T18:01:52.779176] [INFO ] [LOCAL ] CNMD: Starting deletion of primary data. [2025-10-20T18:01:52.779772] [DEBUG] [LOCAL ] CNMD: Deleted primary data file. [2025-10-20T18:01:52.779855] [INFO ] [MAIN ] CNMD: Starting full primary refresh. [2025-10-20T18:01:53.054249] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T18:01:53.052486] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-06. [2025-10-20T18:01:53.192617] [INFO ] [REMOTE ] CNMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:53.851373] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-07. [2025-10-20T18:01:54.059113] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T18:01:54.651389] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-01. [2025-10-20T18:01:54.800259] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T18:01:55.451378] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-07. [2025-10-20T18:01:55.677714] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T18:01:56.057907] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T18:01:56.251378] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-07. [2025-10-20T18:01:57.051387] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-02. [2025-10-20T18:01:57.144264] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T18:01:57.851366] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-08. [2025-10-20T18:01:57.981226] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T18:01:58.154137] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T18:01:58.651435] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-08. [2025-10-20T18:01:59.348417] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T18:01:59.451380] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-09. [2025-10-20T18:02:00.251410] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-03. [2025-10-20T18:02:00.511107] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T18:02:01.051352] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-09. [2025-10-20T18:02:01.160536] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T18:02:01.569487] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:02:01.851362] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-04. [2025-10-20T18:02:02.635952] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:02:02.651351] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-10. [2025-10-20T18:02:03.104052] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T18:02:03.451390] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-05. [2025-10-20T18:02:04.251403] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-08. [2025-10-20T18:02:04.537957] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T18:02:04.611074] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T18:02:04.618632] [INFO ] [REMOTE ] CNM: Full fetch complete (73783 records). [2025-10-20T18:02:04.618803] [DEBUG] [VALIDATOR ] CNM: Validating primary data. [2025-10-20T18:02:04.995359] [DEBUG] [VALIDATOR ] CNM: All expected trading days are present. [2025-10-20T18:02:04.995440] [DEBUG] [VALIDATOR ] CNM: Primary data validation passed. [2025-10-20T18:02:04.996273] [INFO ] [MAIN ] CNM: Fetched primary data is valid. [2025-10-20T18:02:04.996337] [DEBUG] [LOCAL ] CNM: Saving primary data. [2025-10-20T18:02:04.997720] [DEBUG] [LOCAL ] CNM: Trimmed 73783 records to 51031 (keeping 200 days of data) [2025-10-20T18:02:05.053758] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-10. [2025-10-20T18:02:05.290300] [DEBUG] [LOCAL ] CNM: Primary data saved (51031 records). [2025-10-20T18:02:05.290438] [INFO ] [MAIN ] CNM: Full history saved. [2025-10-20T18:02:05.290515] [INFO ] [MAIN ] Update Progress: 491/2367 symbols completed. CNM took 28.0s. Estimated remaining: 3h 56m 57s. Rate: 7.9 symbols/min [2025-10-20T18:02:05.457142] [INFO ] [MAIN ] CNNE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:05.457299] [DEBUG] [LOCAL ] CNNE: No existing data file found in primary storage. [2025-10-20T18:02:05.457357] [WARN ] [MAIN ] CNNE: No existing data found, performing full refresh instead. [2025-10-20T18:02:05.457465] [INFO ] [MAIN ] CNNE: Wiping existing primary data before full refresh. [2025-10-20T18:02:05.457505] [INFO ] [LOCAL ] CNNE: Starting deletion of primary data. [2025-10-20T18:02:05.457545] [DEBUG] [LOCAL ] CNNE: No primary data file found to delete. [2025-10-20T18:02:05.457575] [INFO ] [MAIN ] CNNE: Starting full primary refresh. [2025-10-20T18:02:05.800190] [INFO ] [REMOTE ] CNNE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:05.851334] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-01. [2025-10-20T18:02:06.169085] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T18:02:06.651380] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-06. [2025-10-20T18:02:06.962236] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:02:06.970478] [INFO ] [REMOTE ] CNK: Full fetch complete (76221 records). [2025-10-20T18:02:06.970692] [DEBUG] [VALIDATOR ] CNK: Validating primary data. [2025-10-20T18:02:07.289351] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:02:07.391565] [DEBUG] [VALIDATOR ] CNK: All expected trading days are present. [2025-10-20T18:02:07.391697] [DEBUG] [VALIDATOR ] CNK: Primary data validation passed. [2025-10-20T18:02:07.392600] [INFO ] [MAIN ] CNK: Fetched primary data is valid. [2025-10-20T18:02:07.392664] [DEBUG] [LOCAL ] CNK: Saving primary data. [2025-10-20T18:02:07.394224] [DEBUG] [LOCAL ] CNK: Trimmed 76221 records to 52920 (keeping 200 days of data) [2025-10-20T18:02:07.455433] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-02. [2025-10-20T18:02:07.701228] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:02:07.747996] [DEBUG] [LOCAL ] CNK: Primary data saved (52920 records). [2025-10-20T18:02:07.748161] [INFO ] [MAIN ] CNK: Full history saved. [2025-10-20T18:02:07.748474] [INFO ] [MAIN ] Update Progress: 492/2367 symbols completed. CNK took 32.1s. Estimated remaining: 3h 56m 30s. Rate: 7.9 symbols/min [2025-10-20T18:02:07.919576] [INFO ] [MAIN ] CNO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:07.955333] [DEBUG] [LOCAL ] CNO: Loaded 41237 existing records from primary storage. [2025-10-20T18:02:07.955618] [INFO ] [MAIN ] CNO: Loaded 41237 existing records. [2025-10-20T18:02:08.127798] [WARN ] [MAIN ] CNO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:08.128030] [INFO ] [MAIN ] CNO: Wiping existing primary data before full refresh. [2025-10-20T18:02:08.128077] [INFO ] [LOCAL ] CNO: Starting deletion of primary data. [2025-10-20T18:02:08.128334] [DEBUG] [LOCAL ] CNO: Deleted primary data file. [2025-10-20T18:02:08.128372] [INFO ] [MAIN ] CNO: Starting full primary refresh. [2025-10-20T18:02:08.256632] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-07. [2025-10-20T18:02:08.484895] [INFO ] [REMOTE ] CNO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:08.702588] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T18:02:09.051373] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-03. [2025-10-20T18:02:09.819778] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T18:02:09.851447] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-08. [2025-10-20T18:02:10.230885] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T18:02:10.651507] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-01. [2025-10-20T18:02:10.765907] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T18:02:11.451420] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-09. [2025-10-20T18:02:12.259051] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-04. [2025-10-20T18:02:12.263872] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T18:02:12.873450] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T18:02:13.051379] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-09. [2025-10-20T18:02:13.279315] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T18:02:13.851387] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-05. [2025-10-20T18:02:14.651382] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-10. [2025-10-20T18:02:14.787396] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T18:02:14.867032] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:02:15.451486] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-10. [2025-10-20T18:02:15.664259] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T18:02:15.668448] [INFO ] [REMOTE ] CNMD: Full fetch complete (47029 records). [2025-10-20T18:02:15.668592] [DEBUG] [VALIDATOR ] CNMD: Validating primary data. [2025-10-20T18:02:15.669237] [FATAL] [VALIDATOR ] CNMD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:02:15.669351] [FATAL] [MAIN ] CNMD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:02:15.669403] [WARN ] [MAIN ] CNMD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:02:15.669441] [INFO ] [LOCAL ] CNMD: Starting deletion of primary data. [2025-10-20T18:02:15.669495] [DEBUG] [LOCAL ] CNMD: No primary data file found to delete. [2025-10-20T18:02:15.669528] [INFO ] [LOCAL ] CNMD: Starting deletion of derivative data. [2025-10-20T18:02:15.669722] [DEBUG] [LOCAL ] CNMD: Deleted derivative data file. [2025-10-20T18:02:15.669809] [INFO ] [MAIN ] Update Progress: 493/2367 symbols completed. CNMD took 23.3s. Estimated remaining: 3h 56m 23s. Rate: 7.9 symbols/min [2025-10-20T18:02:15.842629] [INFO ] [MAIN ] CNP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:15.892946] [DEBUG] [LOCAL ] CNP: Loaded 54258 existing records from primary storage. [2025-10-20T18:02:15.893275] [INFO ] [MAIN ] CNP: Loaded 54258 existing records. [2025-10-20T18:02:16.072245] [WARN ] [MAIN ] CNP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:16.072460] [INFO ] [MAIN ] CNP: Wiping existing primary data before full refresh. [2025-10-20T18:02:16.072547] [INFO ] [LOCAL ] CNP: Starting deletion of primary data. [2025-10-20T18:02:16.072781] [DEBUG] [LOCAL ] CNP: Deleted primary data file. [2025-10-20T18:02:16.072835] [INFO ] [MAIN ] CNP: Starting full primary refresh. [2025-10-20T18:02:16.259172] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-02. [2025-10-20T18:02:16.496933] [INFO ] [REMOTE ] CNP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:17.051397] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-06. [2025-10-20T18:02:17.555023] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T18:02:17.563160] [INFO ] [REMOTE ] CNI: Full fetch complete (74850 records). [2025-10-20T18:02:17.563407] [DEBUG] [VALIDATOR ] CNI: Validating primary data. [2025-10-20T18:02:17.861492] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-01. [2025-10-20T18:02:17.983550] [DEBUG] [VALIDATOR ] CNI: All expected trading days are present. [2025-10-20T18:02:17.983706] [DEBUG] [VALIDATOR ] CNI: Primary data validation passed. [2025-10-20T18:02:17.984541] [INFO ] [MAIN ] CNI: Fetched primary data is valid. [2025-10-20T18:02:17.984616] [DEBUG] [LOCAL ] CNI: Saving primary data. [2025-10-20T18:02:18.014292] [DEBUG] [LOCAL ] CNI: Trimmed 74850 records to 51678 (keeping 200 days of data) [2025-10-20T18:02:18.015850] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T18:02:18.274122] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T18:02:18.390617] [DEBUG] [LOCAL ] CNI: Primary data saved (51678 records). [2025-10-20T18:02:18.390805] [INFO ] [MAIN ] CNI: Full history saved. [2025-10-20T18:02:18.390899] [INFO ] [MAIN ] Update Progress: 494/2367 symbols completed. CNI took 47.6s. Estimated remaining: 3h 55m 57s. Rate: 7.9 symbols/min [2025-10-20T18:02:18.572076] [INFO ] [MAIN ] CNQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:18.619162] [DEBUG] [LOCAL ] CNQ: Loaded 54362 existing records from primary storage. [2025-10-20T18:02:18.619493] [INFO ] [MAIN ] CNQ: Loaded 54362 existing records. [2025-10-20T18:02:18.656571] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-07. [2025-10-20T18:02:18.798468] [WARN ] [MAIN ] CNQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:18.798759] [INFO ] [MAIN ] CNQ: Wiping existing primary data before full refresh. [2025-10-20T18:02:18.798808] [INFO ] [LOCAL ] CNQ: Starting deletion of primary data. [2025-10-20T18:02:18.799248] [DEBUG] [LOCAL ] CNQ: Deleted primary data file. [2025-10-20T18:02:18.799304] [INFO ] [MAIN ] CNQ: Starting full primary refresh. [2025-10-20T18:02:19.153041] [INFO ] [REMOTE ] CNQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:19.451344] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-01. [2025-10-20T18:02:19.752798] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:02:19.811492] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T18:02:20.251419] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-02. [2025-10-20T18:02:21.051388] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-08. [2025-10-20T18:02:21.363371] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T18:02:21.851378] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-02. [2025-10-20T18:02:21.880879] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:02:22.324480] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:02:22.651403] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-03. [2025-10-20T18:02:23.293239] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T18:02:23.451455] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-03. [2025-10-20T18:02:24.141909] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:02:24.251375] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-03. [2025-10-20T18:02:25.051407] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-04. [2025-10-20T18:02:25.298381] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T18:02:25.851459] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-04. [2025-10-20T18:02:26.088251] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T18:02:26.651446] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-04. [2025-10-20T18:02:26.750932] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:02:27.451377] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-05. [2025-10-20T18:02:27.809989] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T18:02:28.015510] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T18:02:28.251366] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-05. [2025-10-20T18:02:29.051344] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-05. [2025-10-20T18:02:29.097542] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T18:02:29.749915] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T18:02:29.851375] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-06. [2025-10-20T18:02:30.651389] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-09. [2025-10-20T18:02:30.858939] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T18:02:31.216467] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:02:31.451437] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-07. [2025-10-20T18:02:32.233073] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:02:32.251364] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-10. [2025-10-20T18:02:33.012413] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T18:02:33.051455] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-08. [2025-10-20T18:02:33.584568] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T18:02:33.592922] [INFO ] [REMOTE ] CNNE: Full fetch complete (50340 records). [2025-10-20T18:02:33.593164] [DEBUG] [VALIDATOR ] CNNE: Validating primary data. [2025-10-20T18:02:33.594453] [FATAL] [VALIDATOR ] CNNE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:02:33.594566] [FATAL] [MAIN ] CNNE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:02:33.594618] [WARN ] [MAIN ] CNNE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:02:33.594715] [INFO ] [LOCAL ] CNNE: Starting deletion of primary data. [2025-10-20T18:02:33.594852] [DEBUG] [LOCAL ] CNNE: No primary data file found to delete. [2025-10-20T18:02:33.594922] [INFO ] [LOCAL ] CNNE: Starting deletion of derivative data. [2025-10-20T18:02:33.594976] [DEBUG] [LOCAL ] CNNE: No derivative data file found to delete. [2025-10-20T18:02:33.595057] [INFO ] [MAIN ] Update Progress: 495/2367 symbols completed. CNNE took 28.3s. Estimated remaining: 3h 56m 19s. Rate: 7.9 symbols/min [2025-10-20T18:02:33.839137] [INFO ] [MAIN ] CNR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:33.839320] [DEBUG] [LOCAL ] CNR: No existing data file found in primary storage. [2025-10-20T18:02:33.839410] [WARN ] [MAIN ] CNR: No existing data found, performing full refresh instead. [2025-10-20T18:02:33.839495] [INFO ] [MAIN ] CNR: Wiping existing primary data before full refresh. [2025-10-20T18:02:33.839532] [INFO ] [LOCAL ] CNR: Starting deletion of primary data. [2025-10-20T18:02:33.839572] [DEBUG] [LOCAL ] CNR: No primary data file found to delete. [2025-10-20T18:02:33.839603] [INFO ] [MAIN ] CNR: Starting full primary refresh. [2025-10-20T18:02:33.859857] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-06. [2025-10-20T18:02:34.227224] [INFO ] [REMOTE ] CNR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:34.280884] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T18:02:34.651383] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-06. [2025-10-20T18:02:35.451411] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-09. [2025-10-20T18:02:35.915258] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T18:02:36.251428] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-07. [2025-10-20T18:02:36.640351] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T18:02:37.051452] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-01. [2025-10-20T18:02:37.272137] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:02:37.860100] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-10. [2025-10-20T18:02:37.867798] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T18:02:38.484086] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:02:38.651481] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-08. [2025-10-20T18:02:39.388985] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T18:02:39.395671] [INFO ] [REMOTE ] CNO: Full fetch complete (60822 records). [2025-10-20T18:02:39.395845] [DEBUG] [VALIDATOR ] CNO: Validating primary data. [2025-10-20T18:02:39.463162] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-07. [2025-10-20T18:02:39.814291] [DEBUG] [VALIDATOR ] CNO: All expected trading days are present. [2025-10-20T18:02:39.814380] [DEBUG] [VALIDATOR ] CNO: Primary data validation passed. [2025-10-20T18:02:39.815219] [INFO ] [MAIN ] CNO: Fetched primary data is valid. [2025-10-20T18:02:39.815284] [DEBUG] [LOCAL ] CNO: Saving primary data. [2025-10-20T18:02:39.816484] [DEBUG] [LOCAL ] CNO: Trimmed 60822 records to 40510 (keeping 200 days of data) [2025-10-20T18:02:40.058007] [DEBUG] [LOCAL ] CNO: Primary data saved (40510 records). [2025-10-20T18:02:40.058209] [INFO ] [MAIN ] CNO: Full history saved. [2025-10-20T18:02:40.058673] [INFO ] [MAIN ] Update Progress: 496/2367 symbols completed. CNO took 32.3s. Estimated remaining: 3h 56m 7s. Rate: 7.9 symbols/min [2025-10-20T18:02:40.253495] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-02. [2025-10-20T18:02:40.258295] [INFO ] [MAIN ] CNS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:40.258581] [DEBUG] [LOCAL ] CNS: No existing data file found in primary storage. [2025-10-20T18:02:40.258705] [WARN ] [MAIN ] CNS: No existing data found, performing full refresh instead. [2025-10-20T18:02:40.258821] [INFO ] [MAIN ] CNS: Wiping existing primary data before full refresh. [2025-10-20T18:02:40.258862] [INFO ] [LOCAL ] CNS: Starting deletion of primary data. [2025-10-20T18:02:40.258925] [DEBUG] [LOCAL ] CNS: No primary data file found to delete. [2025-10-20T18:02:40.258984] [INFO ] [MAIN ] CNS: Starting full primary refresh. [2025-10-20T18:02:40.650249] [INFO ] [REMOTE ] CNS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:40.683113] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-08 (2.03 seconds). [2025-10-20T18:02:41.039666] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T18:02:41.051423] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-09. [2025-10-20T18:02:41.861404] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-08. [2025-10-20T18:02:41.868088] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:02:42.528170] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T18:02:42.651427] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-01. [2025-10-20T18:02:43.451405] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-10. [2025-10-20T18:02:43.675836] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:02:43.843176] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T18:02:44.251414] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-03. [2025-10-20T18:02:44.953123] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T18:02:44.961199] [INFO ] [REMOTE ] CNP: Full fetch complete (77161 records). [2025-10-20T18:02:44.961390] [DEBUG] [VALIDATOR ] CNP: Validating primary data. [2025-10-20T18:02:44.962948] [FATAL] [VALIDATOR ] CNP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:02:44.963046] [FATAL] [MAIN ] CNP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:02:44.963091] [WARN ] [MAIN ] CNP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:02:44.963135] [INFO ] [LOCAL ] CNP: Starting deletion of primary data. [2025-10-20T18:02:44.963202] [DEBUG] [LOCAL ] CNP: No primary data file found to delete. [2025-10-20T18:02:44.963242] [INFO ] [LOCAL ] CNP: Starting deletion of derivative data. [2025-10-20T18:02:44.963398] [DEBUG] [LOCAL ] CNP: Deleted derivative data file. [2025-10-20T18:02:44.963487] [INFO ] [MAIN ] Update Progress: 497/2367 symbols completed. CNP took 29.3s. Estimated remaining: 3h 55m 49s. Rate: 7.9 symbols/min [2025-10-20T18:02:45.056481] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-09. [2025-10-20T18:02:45.154900] [INFO ] [MAIN ] CNX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:45.207936] [DEBUG] [LOCAL ] CNX: Loaded 52093 existing records from primary storage. [2025-10-20T18:02:45.208352] [INFO ] [MAIN ] CNX: Loaded 52093 existing records. [2025-10-20T18:02:45.395341] [WARN ] [MAIN ] CNX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:45.395623] [INFO ] [MAIN ] CNX: Wiping existing primary data before full refresh. [2025-10-20T18:02:45.395707] [INFO ] [LOCAL ] CNX: Starting deletion of primary data. [2025-10-20T18:02:45.395958] [DEBUG] [LOCAL ] CNX: Deleted primary data file. [2025-10-20T18:02:45.396001] [INFO ] [MAIN ] CNX: Starting full primary refresh. [2025-10-20T18:02:45.798018] [INFO ] [REMOTE ] CNX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:45.851364] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-02. [2025-10-20T18:02:45.956179] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:02:46.651362] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-01. [2025-10-20T18:02:46.926113] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T18:02:47.032615] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T18:02:47.451424] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-10. [2025-10-20T18:02:48.177185] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T18:02:48.251418] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-03. [2025-10-20T18:02:49.051407] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-02. [2025-10-20T18:02:49.307982] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T18:02:49.415509] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T18:02:49.424976] [INFO ] [REMOTE ] CNQ: Full fetch complete (77740 records). [2025-10-20T18:02:49.425357] [DEBUG] [VALIDATOR ] CNQ: Validating primary data. [2025-10-20T18:02:49.851996] [DEBUG] [VALIDATOR ] CNQ: All expected trading days are present. [2025-10-20T18:02:49.851547] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-04. [2025-10-20T18:02:49.853096] [DEBUG] [VALIDATOR ] CNQ: Primary data validation passed. [2025-10-20T18:02:49.854080] [INFO ] [MAIN ] CNQ: Fetched primary data is valid. [2025-10-20T18:02:49.854255] [DEBUG] [LOCAL ] CNQ: Saving primary data. [2025-10-20T18:02:49.856210] [DEBUG] [LOCAL ] CNQ: Trimmed 77740 records to 53583 (keeping 200 days of data) [2025-10-20T18:02:50.191222] [DEBUG] [LOCAL ] CNQ: Primary data saved (53583 records). [2025-10-20T18:02:50.191400] [INFO ] [MAIN ] CNQ: Full history saved. [2025-10-20T18:02:50.191490] [INFO ] [MAIN ] Update Progress: 498/2367 symbols completed. CNQ took 31.8s. Estimated remaining: 3h 55m 33s. Rate: 7.9 symbols/min [2025-10-20T18:02:50.371373] [INFO ] [MAIN ] CODI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:50.412251] [DEBUG] [LOCAL ] CODI: Loaded 42174 existing records from primary storage. [2025-10-20T18:02:50.412766] [INFO ] [MAIN ] CODI: Loaded 42174 existing records. [2025-10-20T18:02:50.640891] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T18:02:50.662738] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-04. [2025-10-20T18:02:50.667108] [WARN ] [MAIN ] CODI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:50.684802] [INFO ] [MAIN ] CODI: Wiping existing primary data before full refresh. [2025-10-20T18:02:50.685278] [INFO ] [LOCAL ] CODI: Starting deletion of primary data. [2025-10-20T18:02:50.685517] [DEBUG] [LOCAL ] CODI: Deleted primary data file. [2025-10-20T18:02:50.685575] [INFO ] [MAIN ] CODI: Starting full primary refresh. [2025-10-20T18:02:51.088426] [INFO ] [REMOTE ] CODI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:51.105746] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T18:02:51.451377] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-01. [2025-10-20T18:02:52.251390] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-05. [2025-10-20T18:02:52.278954] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T18:02:52.458521] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:02:52.898374] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T18:02:53.051378] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-03. [2025-10-20T18:02:53.851381] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-02. [2025-10-20T18:02:54.399435] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T18:02:54.651383] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-06. [2025-10-20T18:02:54.975686] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T18:02:55.451417] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-04. [2025-10-20T18:02:55.806873] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T18:02:56.251375] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-05. [2025-10-20T18:02:56.818268] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T18:02:57.051450] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-03. [2025-10-20T18:02:57.460823] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T18:02:57.851413] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-07. [2025-10-20T18:02:58.357829] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T18:02:58.622228] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T18:02:58.651405] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-08. [2025-10-20T18:02:59.451443] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-04. [2025-10-20T18:02:59.775570] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T18:03:00.251402] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-06. [2025-10-20T18:03:00.866482] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T18:03:01.051389] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-05. [2025-10-20T18:03:01.777866] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T18:03:01.851386] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-05. [2025-10-20T18:03:02.651347] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-07. [2025-10-20T18:03:02.780487] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T18:03:03.451353] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-09. [2025-10-20T18:03:03.720172] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T18:03:04.230317] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T18:03:04.251339] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-08. [2025-10-20T18:03:04.599261] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T18:03:05.051426] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-06. [2025-10-20T18:03:05.667718] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T18:03:05.851383] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-06. [2025-10-20T18:03:06.651378] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-09. [2025-10-20T18:03:06.765389] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T18:03:07.451347] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-07. [2025-10-20T18:03:07.673826] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T18:03:08.246509] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T18:03:08.255017] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-07. [2025-10-20T18:03:09.051354] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-10. [2025-10-20T18:03:09.296234] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T18:03:09.585577] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T18:03:09.851376] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-08. [2025-10-20T18:03:10.553047] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T18:03:10.559544] [INFO ] [REMOTE ] CNR: Full fetch complete (61918 records). [2025-10-20T18:03:10.559703] [DEBUG] [VALIDATOR ] CNR: Validating primary data. [2025-10-20T18:03:10.560382] [FATAL] [VALIDATOR ] CNR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:03:10.560470] [FATAL] [MAIN ] CNR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:03:10.560538] [WARN ] [MAIN ] CNR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:03:10.560612] [INFO ] [LOCAL ] CNR: Starting deletion of primary data. [2025-10-20T18:03:10.560713] [DEBUG] [LOCAL ] CNR: No primary data file found to delete. [2025-10-20T18:03:10.560798] [INFO ] [LOCAL ] CNR: Starting deletion of derivative data. [2025-10-20T18:03:10.560852] [DEBUG] [LOCAL ] CNR: No derivative data file found to delete. [2025-10-20T18:03:10.560910] [INFO ] [MAIN ] Update Progress: 499/2367 symbols completed. CNR took 37.0s. Estimated remaining: 3h 56m 13s. Rate: 7.9 symbols/min [2025-10-20T18:03:10.658678] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-10. [2025-10-20T18:03:10.744153] [INFO ] [MAIN ] COF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:10.801993] [DEBUG] [LOCAL ] COF: Loaded 54402 existing records from primary storage. [2025-10-20T18:03:10.802329] [INFO ] [MAIN ] COF: Loaded 54402 existing records. [2025-10-20T18:03:10.994183] [WARN ] [MAIN ] COF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:10.994477] [INFO ] [MAIN ] COF: Wiping existing primary data before full refresh. [2025-10-20T18:03:10.994525] [INFO ] [LOCAL ] COF: Starting deletion of primary data. [2025-10-20T18:03:10.994866] [DEBUG] [LOCAL ] COF: Deleted primary data file. [2025-10-20T18:03:10.994922] [INFO ] [MAIN ] COF: Starting full primary refresh. [2025-10-20T18:03:11.377677] [INFO ] [REMOTE ] COF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:11.451366] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-08. [2025-10-20T18:03:11.570341] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:03:11.774185] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T18:03:11.778231] [INFO ] [REMOTE ] CNS: Full fetch complete (30965 records). [2025-10-20T18:03:11.778306] [DEBUG] [VALIDATOR ] CNS: Validating primary data. [2025-10-20T18:03:11.778822] [FATAL] [VALIDATOR ] CNS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:03:11.778901] [FATAL] [MAIN ] CNS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:03:11.778942] [WARN ] [MAIN ] CNS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:03:11.779009] [INFO ] [LOCAL ] CNS: Starting deletion of primary data. [2025-10-20T18:03:11.779070] [DEBUG] [LOCAL ] CNS: No primary data file found to delete. [2025-10-20T18:03:11.779103] [INFO ] [LOCAL ] CNS: Starting deletion of derivative data. [2025-10-20T18:03:11.779140] [DEBUG] [LOCAL ] CNS: No derivative data file found to delete. [2025-10-20T18:03:11.779190] [INFO ] [MAIN ] Update Progress: 500/2367 symbols completed. CNS took 31.7s. Estimated remaining: 3h 55m 42s. Rate: 7.9 symbols/min [2025-10-20T18:03:11.949310] [INFO ] [MAIN ] COHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:11.949450] [DEBUG] [LOCAL ] COHR: No existing data file found in primary storage. [2025-10-20T18:03:11.949486] [WARN ] [MAIN ] COHR: No existing data found, performing full refresh instead. [2025-10-20T18:03:11.949591] [INFO ] [MAIN ] COHR: Wiping existing primary data before full refresh. [2025-10-20T18:03:11.949631] [INFO ] [LOCAL ] COHR: Starting deletion of primary data. [2025-10-20T18:03:11.949715] [DEBUG] [LOCAL ] COHR: No primary data file found to delete. [2025-10-20T18:03:11.949754] [INFO ] [MAIN ] COHR: Starting full primary refresh. [2025-10-20T18:03:12.256514] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-01. [2025-10-20T18:03:12.331939] [INFO ] [REMOTE ] COHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:12.720792] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:03:13.051425] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-01. [2025-10-20T18:03:13.851391] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-09. [2025-10-20T18:03:14.332929] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T18:03:14.651376] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-09. [2025-10-20T18:03:15.136159] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T18:03:15.300857] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T18:03:15.451440] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-10. [2025-10-20T18:03:16.251364] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-02. [2025-10-20T18:03:16.525301] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T18:03:16.877086] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T18:03:16.883786] [INFO ] [REMOTE ] CODI: Full fetch complete (53347 records). [2025-10-20T18:03:16.883884] [DEBUG] [VALIDATOR ] CODI: Validating primary data. [2025-10-20T18:03:17.058046] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-02. [2025-10-20T18:03:17.296101] [DEBUG] [VALIDATOR ] CODI: All expected trading days are present. [2025-10-20T18:03:17.296196] [DEBUG] [VALIDATOR ] CODI: Primary data validation passed. [2025-10-20T18:03:17.296945] [INFO ] [MAIN ] CODI: Fetched primary data is valid. [2025-10-20T18:03:17.297017] [DEBUG] [LOCAL ] CODI: Saving primary data. [2025-10-20T18:03:17.298530] [DEBUG] [LOCAL ] CODI: Trimmed 53347 records to 41816 (keeping 200 days of data) [2025-10-20T18:03:17.566356] [DEBUG] [LOCAL ] CODI: Primary data saved (41816 records). [2025-10-20T18:03:17.566725] [INFO ] [MAIN ] CODI: Full history saved. [2025-10-20T18:03:17.566839] [INFO ] [MAIN ] Update Progress: 501/2367 symbols completed. CODI took 27.4s. Estimated remaining: 3h 55m 28s. Rate: 7.9 symbols/min [2025-10-20T18:03:17.771084] [INFO ] [MAIN ] COLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:17.771362] [DEBUG] [LOCAL ] COLD: No existing data file found in primary storage. [2025-10-20T18:03:17.771704] [WARN ] [MAIN ] COLD: No existing data found, performing full refresh instead. [2025-10-20T18:03:17.772109] [INFO ] [MAIN ] COLD: Wiping existing primary data before full refresh. [2025-10-20T18:03:17.772205] [INFO ] [LOCAL ] COLD: Starting deletion of primary data. [2025-10-20T18:03:17.772346] [DEBUG] [LOCAL ] COLD: No primary data file found to delete. [2025-10-20T18:03:17.772421] [INFO ] [MAIN ] COLD: Starting full primary refresh. [2025-10-20T18:03:17.852541] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-10. [2025-10-20T18:03:18.082730] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T18:03:18.388229] [INFO ] [REMOTE ] COLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:18.651388] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-03. [2025-10-20T18:03:19.036928] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T18:03:19.451490] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-01. [2025-10-20T18:03:19.802461] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T18:03:19.810018] [INFO ] [REMOTE ] CNX: Full fetch complete (74568 records). [2025-10-20T18:03:19.810163] [DEBUG] [VALIDATOR ] CNX: Validating primary data. [2025-10-20T18:03:19.811041] [FATAL] [VALIDATOR ] CNX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:03:19.811131] [FATAL] [MAIN ] CNX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:03:19.811175] [WARN ] [MAIN ] CNX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:03:19.811238] [INFO ] [LOCAL ] CNX: Starting deletion of primary data. [2025-10-20T18:03:19.811303] [DEBUG] [LOCAL ] CNX: No primary data file found to delete. [2025-10-20T18:03:19.811339] [INFO ] [LOCAL ] CNX: Starting deletion of derivative data. [2025-10-20T18:03:19.811500] [DEBUG] [LOCAL ] CNX: Deleted derivative data file. [2025-10-20T18:03:19.811609] [INFO ] [MAIN ] Update Progress: 502/2367 symbols completed. CNX took 34.8s. Estimated remaining: 3h 55m 0s. Rate: 7.9 symbols/min [2025-10-20T18:03:20.017762] [INFO ] [MAIN ] COMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:20.071078] [DEBUG] [LOCAL ] COMP: Loaded 54186 existing records from primary storage. [2025-10-20T18:03:20.071448] [INFO ] [MAIN ] COMP: Loaded 54186 existing records. [2025-10-20T18:03:20.254867] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-03. [2025-10-20T18:03:20.259720] [WARN ] [MAIN ] COMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:20.260865] [INFO ] [MAIN ] COMP: Wiping existing primary data before full refresh. [2025-10-20T18:03:20.260927] [INFO ] [LOCAL ] COMP: Starting deletion of primary data. [2025-10-20T18:03:20.261280] [DEBUG] [LOCAL ] COMP: Deleted primary data file. [2025-10-20T18:03:20.261320] [INFO ] [MAIN ] COMP: Starting full primary refresh. [2025-10-20T18:03:20.668114] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T18:03:20.673583] [INFO ] [REMOTE ] COMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:20.926766] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:03:21.051423] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-01. [2025-10-20T18:03:21.856569] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-04. [2025-10-20T18:03:21.877676] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:03:22.651399] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-04. [2025-10-20T18:03:22.883766] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:03:23.451386] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-02. [2025-10-20T18:03:23.547151] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:03:24.251446] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-02. [2025-10-20T18:03:24.678553] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T18:03:24.994223] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:03:25.051407] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-03. [2025-10-20T18:03:25.630555] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:03:25.851423] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-05. [2025-10-20T18:03:26.652534] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-05. [2025-10-20T18:03:26.657508] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:03:27.451387] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-04. [2025-10-20T18:03:27.809610] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T18:03:28.251461] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-03. [2025-10-20T18:03:28.355996] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:03:28.930527] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:03:29.051473] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-06. [2025-10-20T18:03:29.851394] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-06. [2025-10-20T18:03:30.076000] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T18:03:30.660137] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-05. [2025-10-20T18:03:30.668002] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:03:31.451481] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-04. [2025-10-20T18:03:31.798001] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T18:03:32.106998] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T18:03:32.251397] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-06. [2025-10-20T18:03:33.051450] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-07. [2025-10-20T18:03:33.326493] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T18:03:33.851403] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-05. [2025-10-20T18:03:33.917610] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:03:34.651390] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-07. [2025-10-20T18:03:35.246489] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-07 (2.20 seconds). [2025-10-20T18:03:35.451391] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-08. [2025-10-20T18:03:35.619214] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T18:03:36.251424] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-06. [2025-10-20T18:03:36.461764] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:03:37.051398] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-08. [2025-10-20T18:03:37.458851] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T18:03:37.662838] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:03:37.851397] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-07. [2025-10-20T18:03:38.651357] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-09. [2025-10-20T18:03:38.725061] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T18:03:39.314795] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:03:39.451386] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-07. [2025-10-20T18:03:40.251348] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-08. [2025-10-20T18:03:40.335997] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:03:41.051363] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-10. [2025-10-20T18:03:41.495596] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:03:41.560752] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T18:03:41.693294] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T18:03:41.701627] [INFO ] [REMOTE ] COF: Full fetch complete (77688 records). [2025-10-20T18:03:41.701809] [DEBUG] [VALIDATOR ] COF: Validating primary data. [2025-10-20T18:03:41.856514] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-09. [2025-10-20T18:03:42.106007] [DEBUG] [VALIDATOR ] COF: All expected trading days are present. [2025-10-20T18:03:42.106092] [DEBUG] [VALIDATOR ] COF: Primary data validation passed. [2025-10-20T18:03:42.106987] [INFO ] [MAIN ] COF: Fetched primary data is valid. [2025-10-20T18:03:42.107052] [DEBUG] [LOCAL ] COF: Saving primary data. [2025-10-20T18:03:42.108461] [DEBUG] [LOCAL ] COF: Trimmed 77688 records to 53625 (keeping 200 days of data) [2025-10-20T18:03:42.438205] [DEBUG] [LOCAL ] COF: Primary data saved (53625 records). [2025-10-20T18:03:42.438340] [INFO ] [MAIN ] COF: Full history saved. [2025-10-20T18:03:42.438415] [INFO ] [MAIN ] Update Progress: 503/2367 symbols completed. COF took 31.9s. Estimated remaining: 3h 55m 49s. Rate: 7.9 symbols/min [2025-10-20T18:03:42.605449] [INFO ] [MAIN ] CON: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:42.641052] [DEBUG] [LOCAL ] CON: Loaded 39523 existing records from primary storage. [2025-10-20T18:03:42.641344] [INFO ] [MAIN ] CON: Loaded 39523 existing records. [2025-10-20T18:03:42.662044] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-09. [2025-10-20T18:03:42.857377] [WARN ] [MAIN ] CON: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:42.857609] [INFO ] [MAIN ] CON: Wiping existing primary data before full refresh. [2025-10-20T18:03:42.857685] [INFO ] [LOCAL ] CON: Starting deletion of primary data. [2025-10-20T18:03:42.858035] [DEBUG] [LOCAL ] CON: Deleted primary data file. [2025-10-20T18:03:42.858092] [INFO ] [MAIN ] CON: Starting full primary refresh. [2025-10-20T18:03:43.214285] [INFO ] [REMOTE ] CON: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:43.455508] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-01. [2025-10-20T18:03:43.462074] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:03:44.251338] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-10. [2025-10-20T18:03:44.337701] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:03:44.714427] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:03:45.051392] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-10. [2025-10-20T18:03:45.851415] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-08. [2025-10-20T18:03:46.289544] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T18:03:46.297433] [INFO ] [REMOTE ] COMP: Full fetch complete (77396 records). [2025-10-20T18:03:46.297552] [DEBUG] [VALIDATOR ] COMP: Validating primary data. [2025-10-20T18:03:46.687241] [DEBUG] [VALIDATOR ] COMP: All expected trading days are present. [2025-10-20T18:03:46.687580] [DEBUG] [VALIDATOR ] COMP: Primary data validation passed. [2025-10-20T18:03:46.662761] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-02. [2025-10-20T18:03:46.688809] [INFO ] [MAIN ] COMP: Fetched primary data is valid. [2025-10-20T18:03:46.689036] [DEBUG] [LOCAL ] COMP: Saving primary data. [2025-10-20T18:03:46.691786] [DEBUG] [LOCAL ] COMP: Trimmed 77396 records to 53404 (keeping 200 days of data) [2025-10-20T18:03:46.718932] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T18:03:46.805249] [INFO ] [REMOTE ] COLD: Full fetch complete (74785 records). [2025-10-20T18:03:46.823733] [DEBUG] [VALIDATOR ] COLD: Validating primary data. [2025-10-20T18:03:47.352011] [DEBUG] [VALIDATOR ] COLD: All expected trading days are present. [2025-10-20T18:03:47.352147] [DEBUG] [VALIDATOR ] COLD: Primary data validation passed. [2025-10-20T18:03:47.353090] [INFO ] [MAIN ] COLD: Fetched primary data is valid. [2025-10-20T18:03:47.382432] [DEBUG] [LOCAL ] COLD: Saving primary data. [2025-10-20T18:03:47.408869] [DEBUG] [LOCAL ] COLD: Trimmed 74785 records to 52476 (keeping 200 days of data) [2025-10-20T18:03:47.535541] [DEBUG] [LOCAL ] COMP: Primary data saved (53404 records). [2025-10-20T18:03:47.537515] [INFO ] [MAIN ] COMP: Full history saved. [2025-10-20T18:03:47.537638] [INFO ] [MAIN ] Update Progress: 504/2367 symbols completed. COMP took 27.7s. Estimated remaining: 3h 55m 32s. Rate: 7.9 symbols/min [2025-10-20T18:03:47.849338] [INFO ] [MAIN ] COOK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:47.959503] [DEBUG] [LOCAL ] COOK: Loaded 25238 existing records from primary storage. [2025-10-20T18:03:47.961285] [INFO ] [MAIN ] COOK: Loaded 25238 existing records. [2025-10-20T18:03:48.039021] [DEBUG] [LOCAL ] COLD: Primary data saved (52476 records). [2025-10-20T18:03:48.073629] [INFO ] [MAIN ] COLD: Full history saved. [2025-10-20T18:03:48.090389] [INFO ] [MAIN ] Update Progress: 505/2367 symbols completed. COLD took 30.5s. Estimated remaining: 3h 54m 58s. Rate: 7.9 symbols/min [2025-10-20T18:03:48.223038] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-08 (2.37 seconds). [2025-10-20T18:03:48.270465] [WARN ] [MAIN ] COOK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:48.298490] [INFO ] [MAIN ] COOK: Wiping existing primary data before full refresh. [2025-10-20T18:03:48.302680] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-09. [2025-10-20T18:03:48.302947] [INFO ] [LOCAL ] COOK: Starting deletion of primary data. [2025-10-20T18:03:48.330209] [DEBUG] [LOCAL ] COOK: Deleted primary data file. [2025-10-20T18:03:48.351816] [INFO ] [MAIN ] COOK: Starting full primary refresh. [2025-10-20T18:03:48.393468] [INFO ] [MAIN ] COP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:48.432448] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T18:03:48.478421] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-03. [2025-10-20T18:03:48.795917] [DEBUG] [LOCAL ] COP: Loaded 54507 existing records from primary storage. [2025-10-20T18:03:48.794945] [INFO ] [REMOTE ] COOK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:48.796417] [INFO ] [MAIN ] COP: Loaded 54507 existing records. [2025-10-20T18:03:48.968364] [WARN ] [MAIN ] COP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:48.968541] [INFO ] [MAIN ] COP: Wiping existing primary data before full refresh. [2025-10-20T18:03:48.968589] [INFO ] [LOCAL ] COP: Starting deletion of primary data. [2025-10-20T18:03:48.969175] [DEBUG] [LOCAL ] COP: Deleted primary data file. [2025-10-20T18:03:48.969231] [INFO ] [MAIN ] COP: Starting full primary refresh. [2025-10-20T18:03:49.059294] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-01. [2025-10-20T18:03:49.340537] [INFO ] [REMOTE ] COP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:49.851367] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-01. [2025-10-20T18:03:50.010504] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T18:03:50.202059] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T18:03:50.389555] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-10-20T18:03:50.651444] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-02. [2025-10-20T18:03:51.277600] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T18:03:51.451394] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-04. [2025-10-20T18:03:51.719758] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T18:03:52.251438] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-03. [2025-10-20T18:03:52.894788] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T18:03:53.051384] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-02. [2025-10-20T18:03:53.082915] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:03:53.851422] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-10. [2025-10-20T18:03:54.651358] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-05. [2025-10-20T18:03:55.041063] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-10-20T18:03:55.451386] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-04. [2025-10-20T18:03:55.643113] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T18:03:55.650464] [INFO ] [REMOTE ] COHR: Full fetch complete (77073 records). [2025-10-20T18:03:55.650593] [DEBUG] [VALIDATOR ] COHR: Validating primary data. [2025-10-20T18:03:55.651830] [FATAL] [VALIDATOR ] COHR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:03:55.651915] [FATAL] [MAIN ] COHR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:03:55.651958] [WARN ] [MAIN ] COHR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:03:55.652048] [INFO ] [LOCAL ] COHR: Starting deletion of primary data. [2025-10-20T18:03:55.652114] [DEBUG] [LOCAL ] COHR: No primary data file found to delete. [2025-10-20T18:03:55.652150] [INFO ] [LOCAL ] COHR: Starting deletion of derivative data. [2025-10-20T18:03:55.652190] [DEBUG] [LOCAL ] COHR: No derivative data file found to delete. [2025-10-20T18:03:55.652244] [INFO ] [MAIN ] Update Progress: 506/2367 symbols completed. COHR took 43.9s. Estimated remaining: 3h 54m 51s. Rate: 7.9 symbols/min [2025-10-20T18:03:55.827472] [INFO ] [MAIN ] COR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:55.848835] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T18:03:55.901553] [DEBUG] [LOCAL ] COR: Loaded 47244 existing records from primary storage. [2025-10-20T18:03:55.902019] [INFO ] [MAIN ] COR: Loaded 47244 existing records. [2025-10-20T18:03:56.092331] [WARN ] [MAIN ] COR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:56.092617] [INFO ] [MAIN ] COR: Wiping existing primary data before full refresh. [2025-10-20T18:03:56.092701] [INFO ] [LOCAL ] COR: Starting deletion of primary data. [2025-10-20T18:03:56.092965] [DEBUG] [LOCAL ] COR: Deleted primary data file. [2025-10-20T18:03:56.093004] [INFO ] [MAIN ] COR: Starting full primary refresh. [2025-10-20T18:03:56.254331] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-03. [2025-10-20T18:03:56.391142] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T18:03:56.493835] [INFO ] [REMOTE ] COR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:57.051370] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-05. [2025-10-20T18:03:57.829084] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T18:03:57.851333] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-06. [2025-10-20T18:03:58.503559] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-03 (2.25 seconds). [2025-10-20T18:03:58.651362] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-04. [2025-10-20T18:03:59.187758] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T18:03:59.451371] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-01. [2025-10-20T18:04:00.251426] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-06. [2025-10-20T18:04:00.640149] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T18:04:01.056465] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-05. [2025-10-20T18:04:01.070161] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T18:04:01.632617] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:04:01.851354] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-07. [2025-10-20T18:04:02.651444] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-07. [2025-10-20T18:04:02.892852] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T18:04:03.427997] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-05 (2.37 seconds). [2025-10-20T18:04:03.451344] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-06. [2025-10-20T18:04:04.115047] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:04:04.251418] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-08. [2025-10-20T18:04:05.051376] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-08. [2025-10-20T18:04:05.250582] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T18:04:05.549128] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-10-20T18:04:05.851378] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-09. [2025-10-20T18:04:06.640967] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:04:06.651345] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-09. [2025-10-20T18:04:07.026801] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:04:07.451395] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-10. [2025-10-20T18:04:07.921959] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T18:04:08.251373] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-02. [2025-10-20T18:04:08.436755] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T18:04:08.441475] [INFO ] [REMOTE ] COOK: Full fetch complete (33115 records). [2025-10-20T18:04:08.441584] [DEBUG] [VALIDATOR ] COOK: Validating primary data. [2025-10-20T18:04:08.860724] [DEBUG] [VALIDATOR ] COOK: All expected trading days are present. [2025-10-20T18:04:08.860823] [DEBUG] [VALIDATOR ] COOK: Primary data validation passed. [2025-10-20T18:04:08.861330] [INFO ] [MAIN ] COOK: Fetched primary data is valid. [2025-10-20T18:04:08.861374] [DEBUG] [LOCAL ] COOK: Saving primary data. [2025-10-20T18:04:08.862336] [DEBUG] [LOCAL ] COOK: Trimmed 33115 records to 24923 (keeping 200 days of data) [2025-10-20T18:04:09.006798] [DEBUG] [LOCAL ] COOK: Primary data saved (24923 records). [2025-10-20T18:04:09.006937] [INFO ] [MAIN ] COOK: Full history saved. [2025-10-20T18:04:09.007027] [INFO ] [MAIN ] Update Progress: 507/2367 symbols completed. COOK took 21.5s. Estimated remaining: 3h 55m 4s. Rate: 7.9 symbols/min [2025-10-20T18:04:09.061218] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-07. [2025-10-20T18:04:09.179866] [INFO ] [MAIN ] COTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:09.180036] [DEBUG] [LOCAL ] COTY: No existing data file found in primary storage. [2025-10-20T18:04:09.180074] [WARN ] [MAIN ] COTY: No existing data found, performing full refresh instead. [2025-10-20T18:04:09.180203] [INFO ] [MAIN ] COTY: Wiping existing primary data before full refresh. [2025-10-20T18:04:09.180243] [INFO ] [LOCAL ] COTY: Starting deletion of primary data. [2025-10-20T18:04:09.180285] [DEBUG] [LOCAL ] COTY: No primary data file found to delete. [2025-10-20T18:04:09.180316] [INFO ] [MAIN ] COTY: Starting full primary refresh. [2025-10-20T18:04:09.535482] [INFO ] [REMOTE ] COTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:09.851356] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-10. [2025-10-20T18:04:10.114575] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T18:04:10.651360] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-01. [2025-10-20T18:04:10.808051] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:04:11.050469] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:04:11.057029] [INFO ] [REMOTE ] CON: Full fetch complete (56813 records). [2025-10-20T18:04:11.057211] [DEBUG] [VALIDATOR ] CON: Validating primary data. [2025-10-20T18:04:11.460829] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-08. [2025-10-20T18:04:11.586080] [DEBUG] [VALIDATOR ] CON: All expected trading days are present. [2025-10-20T18:04:11.586178] [DEBUG] [VALIDATOR ] CON: Primary data validation passed. [2025-10-20T18:04:11.586959] [INFO ] [MAIN ] CON: Fetched primary data is valid. [2025-10-20T18:04:11.587030] [DEBUG] [LOCAL ] CON: Saving primary data. [2025-10-20T18:04:11.588332] [DEBUG] [LOCAL ] CON: Trimmed 56813 records to 38877 (keeping 200 days of data) [2025-10-20T18:04:11.867558] [DEBUG] [LOCAL ] CON: Primary data saved (38877 records). [2025-10-20T18:04:11.867741] [INFO ] [MAIN ] CON: Full history saved. [2025-10-20T18:04:11.867830] [INFO ] [MAIN ] Update Progress: 508/2367 symbols completed. CON took 29.4s. Estimated remaining: 3h 54m 40s. Rate: 7.9 symbols/min [2025-10-20T18:04:12.055209] [INFO ] [MAIN ] COUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:12.101632] [DEBUG] [LOCAL ] COUR: Loaded 50057 existing records from primary storage. [2025-10-20T18:04:12.101971] [INFO ] [MAIN ] COUR: Loaded 50057 existing records. [2025-10-20T18:04:12.252396] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-03. [2025-10-20T18:04:12.285182] [WARN ] [MAIN ] COUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:12.285610] [INFO ] [MAIN ] COUR: Wiping existing primary data before full refresh. [2025-10-20T18:04:12.285808] [INFO ] [LOCAL ] COUR: Starting deletion of primary data. [2025-10-20T18:04:12.286210] [DEBUG] [LOCAL ] COUR: Deleted primary data file. [2025-10-20T18:04:12.286269] [INFO ] [MAIN ] COUR: Starting full primary refresh. [2025-10-20T18:04:12.702283] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T18:04:12.707074] [INFO ] [REMOTE ] COUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:13.051389] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-01. [2025-10-20T18:04:13.519545] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T18:04:13.631588] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:04:13.851379] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-02. [2025-10-20T18:04:14.651400] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-04. [2025-10-20T18:04:14.774516] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T18:04:15.273546] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T18:04:15.451354] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-03. [2025-10-20T18:04:16.251435] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-02. [2025-10-20T18:04:16.412346] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:04:16.784700] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T18:04:17.051388] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-04. [2025-10-20T18:04:17.547459] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T18:04:17.851442] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-09. [2025-10-20T18:04:18.651360] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-03. [2025-10-20T18:04:18.852290] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:04:19.451384] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-05. [2025-10-20T18:04:19.514690] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:04:20.056967] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T18:04:20.251433] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-10. [2025-10-20T18:04:21.051464] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-04. [2025-10-20T18:04:21.121839] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T18:04:21.130725] [INFO ] [REMOTE ] COP: Full fetch complete (77906 records). [2025-10-20T18:04:21.130930] [DEBUG] [VALIDATOR ] COP: Validating primary data. [2025-10-20T18:04:21.605191] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-10-20T18:04:21.635204] [DEBUG] [VALIDATOR ] COP: All expected trading days are present. [2025-10-20T18:04:21.635762] [DEBUG] [VALIDATOR ] COP: Primary data validation passed. [2025-10-20T18:04:21.636744] [INFO ] [MAIN ] COP: Fetched primary data is valid. [2025-10-20T18:04:21.636869] [DEBUG] [LOCAL ] COP: Saving primary data. [2025-10-20T18:04:21.638602] [DEBUG] [LOCAL ] COP: Trimmed 77906 records to 53727 (keeping 200 days of data) [2025-10-20T18:04:21.861033] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-05. [2025-10-20T18:04:21.980327] [DEBUG] [LOCAL ] COP: Primary data saved (53727 records). [2025-10-20T18:04:21.980481] [INFO ] [MAIN ] COP: Full history saved. [2025-10-20T18:04:21.980561] [INFO ] [MAIN ] Update Progress: 509/2367 symbols completed. COP took 33.9s. Estimated remaining: 3h 54m 41s. Rate: 7.9 symbols/min [2025-10-20T18:04:22.165572] [INFO ] [MAIN ] CP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:22.229144] [DEBUG] [LOCAL ] CP: Loaded 54017 existing records from primary storage. [2025-10-20T18:04:22.229469] [INFO ] [MAIN ] CP: Loaded 54017 existing records. [2025-10-20T18:04:22.406819] [WARN ] [MAIN ] CP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:22.407005] [INFO ] [MAIN ] CP: Wiping existing primary data before full refresh. [2025-10-20T18:04:22.407051] [INFO ] [LOCAL ] CP: Starting deletion of primary data. [2025-10-20T18:04:22.407227] [DEBUG] [LOCAL ] CP: Deleted primary data file. [2025-10-20T18:04:22.407265] [INFO ] [MAIN ] CP: Starting full primary refresh. [2025-10-20T18:04:22.653092] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-06. [2025-10-20T18:04:22.775018] [INFO ] [REMOTE ] CP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:22.825842] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:04:23.249127] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T18:04:23.451393] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-01. [2025-10-20T18:04:24.256522] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-05. [2025-10-20T18:04:24.272497] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:04:24.856035] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T18:04:25.051401] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-02. [2025-10-20T18:04:25.851426] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-06. [2025-10-20T18:04:26.020003] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T18:04:26.318003] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T18:04:26.651422] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-03. [2025-10-20T18:04:27.451389] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-06. [2025-10-20T18:04:27.723660] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T18:04:28.251420] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-07. [2025-10-20T18:04:28.366776] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T18:04:28.893200] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T18:04:29.051342] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-07. [2025-10-20T18:04:29.424241] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T18:04:29.851386] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-08. [2025-10-20T18:04:30.651396] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-04. [2025-10-20T18:04:30.850371] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:04:31.327841] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:04:31.451383] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-08. [2025-10-20T18:04:31.989541] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:04:32.251383] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-07. [2025-10-20T18:04:33.051467] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-09. [2025-10-20T18:04:33.230372] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:04:33.759606] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T18:04:33.851381] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-08. [2025-10-20T18:04:34.367796] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T18:04:34.651372] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-10. [2025-10-20T18:04:35.451366] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-05. [2025-10-20T18:04:35.737508] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T18:04:36.251389] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-09. [2025-10-20T18:04:36.697853] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T18:04:36.703955] [INFO ] [REMOTE ] COR: Full fetch complete (68252 records). [2025-10-20T18:04:36.704074] [DEBUG] [VALIDATOR ] COR: Validating primary data. [2025-10-20T18:04:36.704764] [FATAL] [VALIDATOR ] COR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:04:36.704850] [FATAL] [MAIN ] COR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:04:36.704892] [WARN ] [MAIN ] COR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:04:36.704959] [INFO ] [LOCAL ] COR: Starting deletion of primary data. [2025-10-20T18:04:36.705021] [DEBUG] [LOCAL ] COR: No primary data file found to delete. [2025-10-20T18:04:36.705055] [INFO ] [LOCAL ] COR: Starting deletion of derivative data. [2025-10-20T18:04:36.705223] [DEBUG] [LOCAL ] COR: Deleted derivative data file. [2025-10-20T18:04:36.705607] [INFO ] [MAIN ] Update Progress: 510/2367 symbols completed. COR took 41.1s. Estimated remaining: 3h 54m 60s. Rate: 7.9 symbols/min [2025-10-20T18:04:36.882979] [INFO ] [MAIN ] CPA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:36.912853] [DEBUG] [LOCAL ] CPA: Loaded 28704 existing records from primary storage. [2025-10-20T18:04:36.913154] [INFO ] [MAIN ] CPA: Loaded 28704 existing records. [2025-10-20T18:04:37.060750] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-09. [2025-10-20T18:04:37.105243] [WARN ] [MAIN ] CPA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:37.105768] [INFO ] [MAIN ] CPA: Wiping existing primary data before full refresh. [2025-10-20T18:04:37.105918] [INFO ] [LOCAL ] CPA: Starting deletion of primary data. [2025-10-20T18:04:37.106168] [DEBUG] [LOCAL ] CPA: Deleted primary data file. [2025-10-20T18:04:37.106209] [INFO ] [MAIN ] CPA: Starting full primary refresh. [2025-10-20T18:04:37.270969] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T18:04:37.509275] [INFO ] [REMOTE ] CPA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:37.851386] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-01. [2025-10-20T18:04:37.950144] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T18:04:38.644881] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:04:38.655258] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-10. [2025-10-20T18:04:39.136319] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T18:04:39.451376] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-02. [2025-10-20T18:04:40.133845] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T18:04:40.141257] [INFO ] [REMOTE ] COUR: Full fetch complete (71100 records). [2025-10-20T18:04:40.141397] [DEBUG] [VALIDATOR ] COUR: Validating primary data. [2025-10-20T18:04:40.256438] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-06. [2025-10-20T18:04:40.540090] [DEBUG] [VALIDATOR ] COUR: All expected trading days are present. [2025-10-20T18:04:40.540169] [DEBUG] [VALIDATOR ] COUR: Primary data validation passed. [2025-10-20T18:04:40.540974] [INFO ] [MAIN ] COUR: Fetched primary data is valid. [2025-10-20T18:04:40.541054] [DEBUG] [LOCAL ] COUR: Saving primary data. [2025-10-20T18:04:40.542368] [DEBUG] [LOCAL ] COUR: Trimmed 71100 records to 49404 (keeping 200 days of data) [2025-10-20T18:04:40.781148] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:04:40.849724] [DEBUG] [LOCAL ] COUR: Primary data saved (49404 records). [2025-10-20T18:04:40.849854] [INFO ] [MAIN ] COUR: Full history saved. [2025-10-20T18:04:40.849928] [INFO ] [MAIN ] Update Progress: 511/2367 symbols completed. COUR took 29.0s. Estimated remaining: 3h 54m 40s. Rate: 7.9 symbols/min [2025-10-20T18:04:41.019758] [INFO ] [MAIN ] CPAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:41.019913] [DEBUG] [LOCAL ] CPAC: No existing data file found in primary storage. [2025-10-20T18:04:41.019948] [WARN ] [MAIN ] CPAC: No existing data found, performing full refresh instead. [2025-10-20T18:04:41.020056] [INFO ] [MAIN ] CPAC: Wiping existing primary data before full refresh. [2025-10-20T18:04:41.020095] [INFO ] [LOCAL ] CPAC: Starting deletion of primary data. [2025-10-20T18:04:41.020135] [DEBUG] [LOCAL ] CPAC: No primary data file found to delete. [2025-10-20T18:04:41.020165] [INFO ] [MAIN ] CPAC: Starting full primary refresh. [2025-10-20T18:04:41.056717] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-03. [2025-10-20T18:04:41.422366] [INFO ] [REMOTE ] CPAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:41.537767] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T18:04:41.851369] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-01. [2025-10-20T18:04:42.146547] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T18:04:42.393748] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-10-20T18:04:42.651377] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-02. [2025-10-20T18:04:43.006733] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:04:43.451380] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-07. [2025-10-20T18:04:44.251377] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-10. [2025-10-20T18:04:45.051380] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-03. [2025-10-20T18:04:45.333451] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T18:04:45.468130] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T18:04:45.851404] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-04. [2025-10-20T18:04:46.217610] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-10-20T18:04:46.259457] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T18:04:46.267436] [INFO ] [REMOTE ] COTY: Full fetch complete (77297 records). [2025-10-20T18:04:46.267583] [DEBUG] [VALIDATOR ] COTY: Validating primary data. [2025-10-20T18:04:46.650197] [DEBUG] [VALIDATOR ] COTY: All expected trading days are present. [2025-10-20T18:04:46.650315] [DEBUG] [VALIDATOR ] COTY: Primary data validation passed. [2025-10-20T18:04:46.651205] [INFO ] [MAIN ] COTY: Fetched primary data is valid. [2025-10-20T18:04:46.651273] [DEBUG] [LOCAL ] COTY: Saving primary data. [2025-10-20T18:04:46.651498] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-05. [2025-10-20T18:04:46.653553] [DEBUG] [LOCAL ] COTY: Trimmed 77297 records to 53275 (keeping 200 days of data) [2025-10-20T18:04:46.944626] [DEBUG] [LOCAL ] COTY: Primary data saved (53275 records). [2025-10-20T18:04:46.944795] [INFO ] [MAIN ] COTY: Full history saved. [2025-10-20T18:04:46.944868] [INFO ] [MAIN ] Update Progress: 512/2367 symbols completed. COTY took 37.9s. Estimated remaining: 3h 54m 27s. Rate: 7.9 symbols/min [2025-10-20T18:04:47.031594] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-10-20T18:04:47.121013] [INFO ] [MAIN ] CPAY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:47.151271] [DEBUG] [LOCAL ] CPAY: Loaded 30966 existing records from primary storage. [2025-10-20T18:04:47.151570] [INFO ] [MAIN ] CPAY: Loaded 30966 existing records. [2025-10-20T18:04:47.344593] [WARN ] [MAIN ] CPAY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:47.344955] [INFO ] [MAIN ] CPAY: Wiping existing primary data before full refresh. [2025-10-20T18:04:47.345008] [INFO ] [LOCAL ] CPAY: Starting deletion of primary data. [2025-10-20T18:04:47.345237] [DEBUG] [LOCAL ] CPAY: Deleted primary data file. [2025-10-20T18:04:47.345276] [INFO ] [MAIN ] CPAY: Starting full primary refresh. [2025-10-20T18:04:47.456577] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-06. [2025-10-20T18:04:47.737915] [INFO ] [REMOTE ] CPAY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:47.904312] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T18:04:48.251402] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-01. [2025-10-20T18:04:49.051374] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-04. [2025-10-20T18:04:49.363634] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:04:49.851366] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-02. [2025-10-20T18:04:50.385681] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T18:04:50.651364] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-08. [2025-10-20T18:04:50.894369] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:04:51.451422] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-03. [2025-10-20T18:04:52.251385] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-05. [2025-10-20T18:04:52.501227] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-10-20T18:04:52.877986] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T18:04:53.051421] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-07. [2025-10-20T18:04:53.115283] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T18:04:53.567042] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T18:04:53.851487] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-09. [2025-10-20T18:04:54.651429] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-08. [2025-10-20T18:04:55.013858] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T18:04:55.451389] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-09. [2025-10-20T18:04:55.668790] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:04:55.892393] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T18:04:56.251347] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-10. [2025-10-20T18:04:56.786574] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T18:04:56.787585] [INFO ] [REMOTE ] CPAC: Full fetch complete (1771 records). [2025-10-20T18:04:56.787679] [DEBUG] [VALIDATOR ] CPAC: Validating primary data. [2025-10-20T18:04:56.788041] [FATAL] [VALIDATOR ] CPAC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:04:56.788122] [FATAL] [MAIN ] CPAC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:04:56.788165] [WARN ] [MAIN ] CPAC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:04:56.788203] [INFO ] [LOCAL ] CPAC: Starting deletion of primary data. [2025-10-20T18:04:56.788257] [DEBUG] [LOCAL ] CPAC: No primary data file found to delete. [2025-10-20T18:04:56.788291] [INFO ] [LOCAL ] CPAC: Starting deletion of derivative data. [2025-10-20T18:04:56.788330] [DEBUG] [LOCAL ] CPAC: No derivative data file found to delete. [2025-10-20T18:04:56.788383] [INFO ] [MAIN ] Update Progress: 513/2367 symbols completed. CPAC took 15.9s. Estimated remaining: 3h 54m 27s. Rate: 7.9 symbols/min [2025-10-20T18:04:56.962020] [INFO ] [MAIN ] CPF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:56.976539] [DEBUG] [LOCAL ] CPF: Loaded 15519 existing records from primary storage. [2025-10-20T18:04:56.976842] [INFO ] [MAIN ] CPF: Loaded 15519 existing records. [2025-10-20T18:04:57.056499] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-04. [2025-10-20T18:04:57.172026] [WARN ] [MAIN ] CPF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:57.172233] [INFO ] [MAIN ] CPF: Wiping existing primary data before full refresh. [2025-10-20T18:04:57.172280] [INFO ] [LOCAL ] CPF: Starting deletion of primary data. [2025-10-20T18:04:57.172437] [DEBUG] [LOCAL ] CPF: Deleted primary data file. [2025-10-20T18:04:57.172475] [INFO ] [MAIN ] CPF: Starting full primary refresh. [2025-10-20T18:04:57.550563] [INFO ] [REMOTE ] CPF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:57.851393] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-01. [2025-10-20T18:04:58.543556] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:04:58.651391] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-06. [2025-10-20T18:04:58.735860] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T18:04:59.451384] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-05. [2025-10-20T18:04:59.935179] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T18:05:00.251440] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-02. [2025-10-20T18:05:00.853699] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T18:05:01.051358] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-07. [2025-10-20T18:05:01.283806] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T18:05:01.851428] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-06. [2025-10-20T18:05:02.651389] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-03. [2025-10-20T18:05:02.702254] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:05:03.202889] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T18:05:03.451416] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-10. [2025-10-20T18:05:03.766816] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:05:04.251438] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-07. [2025-10-20T18:05:05.051428] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-08. [2025-10-20T18:05:05.572511] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T18:05:05.579773] [INFO ] [REMOTE ] CP: Full fetch complete (76919 records). [2025-10-20T18:05:05.579911] [DEBUG] [VALIDATOR ] CP: Validating primary data. [2025-10-20T18:05:05.580556] [FATAL] [VALIDATOR ] CP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:05:05.580684] [FATAL] [MAIN ] CP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:05:05.580757] [WARN ] [MAIN ] CP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:05:05.580850] [INFO ] [LOCAL ] CP: Starting deletion of primary data. [2025-10-20T18:05:05.580938] [DEBUG] [LOCAL ] CP: No primary data file found to delete. [2025-10-20T18:05:05.580978] [INFO ] [LOCAL ] CP: Starting deletion of derivative data. [2025-10-20T18:05:05.581100] [DEBUG] [LOCAL ] CP: Deleted derivative data file. [2025-10-20T18:05:05.581174] [INFO ] [MAIN ] Update Progress: 514/2367 symbols completed. CP took 43.6s. Estimated remaining: 3h 54m 24s. Rate: 7.9 symbols/min [2025-10-20T18:05:05.706539] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:05:05.789614] [INFO ] [MAIN ] CPK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:05.804824] [DEBUG] [LOCAL ] CPK: Loaded 14207 existing records from primary storage. [2025-10-20T18:05:05.805057] [INFO ] [MAIN ] CPK: Loaded 14207 existing records. [2025-10-20T18:05:05.860671] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-04. [2025-10-20T18:05:05.991679] [WARN ] [MAIN ] CPK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:05.991894] [INFO ] [MAIN ] CPK: Wiping existing primary data before full refresh. [2025-10-20T18:05:05.991946] [INFO ] [LOCAL ] CPK: Starting deletion of primary data. [2025-10-20T18:05:05.992112] [DEBUG] [LOCAL ] CPK: Deleted primary data file. [2025-10-20T18:05:05.992148] [INFO ] [MAIN ] CPK: Starting full primary refresh. [2025-10-20T18:05:06.351891] [INFO ] [REMOTE ] CPK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:06.368936] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T18:05:06.651402] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-08. [2025-10-20T18:05:06.802047] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T18:05:07.451437] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-01. [2025-10-20T18:05:07.749610] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:05:08.024732] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T18:05:08.251425] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-09. [2025-10-20T18:05:09.051445] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-09. [2025-10-20T18:05:09.417979] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:05:09.851437] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-10. [2025-10-20T18:05:10.364030] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T18:05:10.651426] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-10. [2025-10-20T18:05:11.106380] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T18:05:11.111133] [INFO ] [REMOTE ] CPA: Full fetch complete (41312 records). [2025-10-20T18:05:11.111351] [DEBUG] [VALIDATOR ] CPA: Validating primary data. [2025-10-20T18:05:11.466428] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-02. [2025-10-20T18:05:11.524256] [DEBUG] [VALIDATOR ] CPA: All expected trading days are present. [2025-10-20T18:05:11.524350] [DEBUG] [VALIDATOR ] CPA: Primary data validation passed. [2025-10-20T18:05:11.525087] [INFO ] [MAIN ] CPA: Fetched primary data is valid. [2025-10-20T18:05:11.525158] [DEBUG] [LOCAL ] CPA: Saving primary data. [2025-10-20T18:05:11.526262] [DEBUG] [LOCAL ] CPA: Trimmed 41312 records to 28158 (keeping 200 days of data) [2025-10-20T18:05:11.721107] [DEBUG] [LOCAL ] CPA: Primary data saved (28158 records). [2025-10-20T18:05:11.721266] [INFO ] [MAIN ] CPA: Full history saved. [2025-10-20T18:05:11.721351] [INFO ] [MAIN ] Update Progress: 515/2367 symbols completed. CPA took 35.0s. Estimated remaining: 3h 54m 11s. Rate: 7.9 symbols/min [2025-10-20T18:05:11.923465] [INFO ] [MAIN ] CPNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:11.938120] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T18:05:11.943175] [INFO ] [REMOTE ] CPAY: Full fetch complete (44916 records). [2025-10-20T18:05:11.943337] [DEBUG] [VALIDATOR ] CPAY: Validating primary data. [2025-10-20T18:05:12.247403] [DEBUG] [LOCAL ] CPNG: Loaded 54459 existing records from primary storage. [2025-10-20T18:05:12.261854] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-05. [2025-10-20T18:05:12.275377] [INFO ] [MAIN ] CPNG: Loaded 54459 existing records. [2025-10-20T18:05:12.556097] [DEBUG] [VALIDATOR ] CPAY: All expected trading days are present. [2025-10-20T18:05:12.550797] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T18:05:12.586790] [DEBUG] [VALIDATOR ] CPAY: Primary data validation passed. [2025-10-20T18:05:12.599328] [INFO ] [MAIN ] CPAY: Fetched primary data is valid. [2025-10-20T18:05:12.608974] [DEBUG] [LOCAL ] CPAY: Saving primary data. [2025-10-20T18:05:12.634678] [WARN ] [MAIN ] CPNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:12.634921] [INFO ] [MAIN ] CPNG: Wiping existing primary data before full refresh. [2025-10-20T18:05:12.635054] [INFO ] [LOCAL ] CPNG: Starting deletion of primary data. [2025-10-20T18:05:12.635374] [DEBUG] [LOCAL ] CPNG: Deleted primary data file. [2025-10-20T18:05:12.635438] [INFO ] [MAIN ] CPNG: Starting full primary refresh. [2025-10-20T18:05:12.667925] [DEBUG] [LOCAL ] CPAY: Trimmed 44916 records to 30345 (keeping 200 days of data) [2025-10-20T18:05:13.057135] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-03. [2025-10-20T18:05:13.219054] [INFO ] [REMOTE ] CPNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:13.242342] [DEBUG] [LOCAL ] CPAY: Primary data saved (30345 records). [2025-10-20T18:05:13.242576] [INFO ] [MAIN ] CPAY: Full history saved. [2025-10-20T18:05:13.242709] [INFO ] [MAIN ] Update Progress: 516/2367 symbols completed. CPAY took 26.3s. Estimated remaining: 3h 53m 42s. Rate: 7.9 symbols/min [2025-10-20T18:05:13.378893] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T18:05:13.457451] [INFO ] [MAIN ] CPRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:13.510754] [DEBUG] [LOCAL ] CPRI: Loaded 53228 existing records from primary storage. [2025-10-20T18:05:13.511093] [INFO ] [MAIN ] CPRI: Loaded 53228 existing records. [2025-10-20T18:05:13.702005] [WARN ] [MAIN ] CPRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:13.702274] [INFO ] [MAIN ] CPRI: Wiping existing primary data before full refresh. [2025-10-20T18:05:13.702326] [INFO ] [LOCAL ] CPRI: Starting deletion of primary data. [2025-10-20T18:05:13.702749] [DEBUG] [LOCAL ] CPRI: Deleted primary data file. [2025-10-20T18:05:13.702806] [INFO ] [MAIN ] CPRI: Starting full primary refresh. [2025-10-20T18:05:13.853358] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-06. [2025-10-20T18:05:14.056687] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T18:05:14.083221] [INFO ] [REMOTE ] CPRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:14.538152] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T18:05:14.651432] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-01. [2025-10-20T18:05:15.451422] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-07. [2025-10-20T18:05:16.251360] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-01. [2025-10-20T18:05:16.452179] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T18:05:16.543043] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:05:17.051436] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-08. [2025-10-20T18:05:17.685945] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T18:05:17.851427] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-09. [2025-10-20T18:05:18.167938] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T18:05:18.625566] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T18:05:18.651393] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-02. [2025-10-20T18:05:19.451450] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-10. [2025-10-20T18:05:19.956457] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T18:05:20.251438] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-03. [2025-10-20T18:05:20.494472] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T18:05:20.497282] [INFO ] [REMOTE ] CPF: Full fetch complete (23557 records). [2025-10-20T18:05:20.497356] [DEBUG] [VALIDATOR ] CPF: Validating primary data. [2025-10-20T18:05:20.905880] [DEBUG] [VALIDATOR ] CPF: All expected trading days are present. [2025-10-20T18:05:20.905966] [DEBUG] [VALIDATOR ] CPF: Primary data validation passed. [2025-10-20T18:05:20.906405] [INFO ] [MAIN ] CPF: Fetched primary data is valid. [2025-10-20T18:05:20.906445] [DEBUG] [LOCAL ] CPF: Saving primary data. [2025-10-20T18:05:20.907165] [DEBUG] [LOCAL ] CPF: Trimmed 23557 records to 15307 (keeping 200 days of data) [2025-10-20T18:05:21.002026] [DEBUG] [LOCAL ] CPF: Primary data saved (15307 records). [2025-10-20T18:05:21.002180] [INFO ] [MAIN ] CPF: Full history saved. [2025-10-20T18:05:21.002262] [INFO ] [MAIN ] Update Progress: 517/2367 symbols completed. CPF took 24.2s. Estimated remaining: 3h 53m 35s. Rate: 7.9 symbols/min [2025-10-20T18:05:21.051422] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-04. [2025-10-20T18:05:21.851457] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-02. [2025-10-20T18:05:22.022098] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T18:05:22.399564] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T18:05:22.651407] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-05. [2025-10-20T18:05:23.451394] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-04. [2025-10-20T18:05:23.510404] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:05:23.535503] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T18:05:24.251423] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-06. [2025-10-20T18:05:25.051414] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-03. [2025-10-20T18:05:25.082503] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T18:05:25.290729] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T18:05:25.851368] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-07. [2025-10-20T18:05:26.651438] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-05. [2025-10-20T18:05:26.807935] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T18:05:27.264708] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-03 (2.21 seconds). [2025-10-20T18:05:27.451415] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-04. [2025-10-20T18:05:28.238127] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T18:05:28.251338] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-06. [2025-10-20T18:05:29.051474] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-08. [2025-10-20T18:05:29.563680] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-10-20T18:05:29.600299] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T18:05:29.851351] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-09. [2025-10-20T18:05:29.975178] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:05:30.401233] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-10-20T18:05:30.651431] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-05. [2025-10-20T18:05:31.451397] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-10. [2025-10-20T18:05:32.251419] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-07. [2025-10-20T18:05:32.309989] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T18:05:32.312942] [INFO ] [REMOTE ] CPK: Full fetch complete (18664 records). [2025-10-20T18:05:32.313025] [DEBUG] [VALIDATOR ] CPK: Validating primary data. [2025-10-20T18:05:32.702010] [DEBUG] [VALIDATOR ] CPK: All expected trading days are present. [2025-10-20T18:05:32.702090] [DEBUG] [VALIDATOR ] CPK: Primary data validation passed. [2025-10-20T18:05:32.702477] [INFO ] [MAIN ] CPK: Fetched primary data is valid. [2025-10-20T18:05:32.702516] [DEBUG] [LOCAL ] CPK: Saving primary data. [2025-10-20T18:05:32.703241] [DEBUG] [LOCAL ] CPK: Trimmed 18664 records to 14051 (keeping 200 days of data) [2025-10-20T18:05:32.792414] [DEBUG] [LOCAL ] CPK: Primary data saved (14051 records). [2025-10-20T18:05:32.792572] [INFO ] [MAIN ] CPK: Full history saved. [2025-10-20T18:05:32.792679] [INFO ] [MAIN ] Update Progress: 518/2367 symbols completed. CPK took 27.2s. Estimated remaining: 3h 53m 42s. Rate: 7.9 symbols/min [2025-10-20T18:05:33.017218] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-05 (2.37 seconds). [2025-10-20T18:05:33.051341] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-06. [2025-10-20T18:05:33.971938] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T18:05:33.983496] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-08. [2025-10-20T18:05:35.163678] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-06 (2.11 seconds). [2025-10-20T18:05:35.174577] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-07. [2025-10-20T18:05:35.723063] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:05:35.734114] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-09. [2025-10-20T18:05:37.486842] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T18:05:37.498144] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-10. [2025-10-20T18:05:37.717965] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-07 (2.54 seconds). [2025-10-20T18:05:37.730013] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-08. [2025-10-20T18:05:39.345156] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:05:39.356625] [INFO ] [REMOTE ] CPRI: Full fetch complete (76183 records). [2025-10-20T18:05:39.356898] [DEBUG] [VALIDATOR ] CPRI: Validating primary data. [2025-10-20T18:05:39.784682] [DEBUG] [VALIDATOR ] CPRI: All expected trading days are present. [2025-10-20T18:05:39.784785] [DEBUG] [VALIDATOR ] CPRI: Primary data validation passed. [2025-10-20T18:05:39.785618] [INFO ] [MAIN ] CPRI: Fetched primary data is valid. [2025-10-20T18:05:39.785700] [DEBUG] [LOCAL ] CPRI: Saving primary data. [2025-10-20T18:05:39.787826] [DEBUG] [LOCAL ] CPRI: Trimmed 76183 records to 52442 (keeping 200 days of data) [2025-10-20T18:05:40.102856] [DEBUG] [LOCAL ] CPRI: Primary data saved (52442 records). [2025-10-20T18:05:40.103005] [INFO ] [MAIN ] CPRI: Full history saved. [2025-10-20T18:05:40.103087] [INFO ] [MAIN ] Update Progress: 519/2367 symbols completed. CPRI took 26.9s. Estimated remaining: 3h 53m 34s. Rate: 7.9 symbols/min [2025-10-20T18:05:40.169395] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-08 (2.44 seconds). [2025-10-20T18:05:40.181807] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-09. [2025-10-20T18:05:42.257661] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-10-20T18:05:42.269598] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-10. [2025-10-20T18:05:44.694528] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-10 (2.42 seconds). [2025-10-20T18:05:44.702301] [INFO ] [REMOTE ] CPNG: Full fetch complete (77789 records). [2025-10-20T18:05:44.702417] [DEBUG] [VALIDATOR ] CPNG: Validating primary data. [2025-10-20T18:05:45.100664] [DEBUG] [VALIDATOR ] CPNG: All expected trading days are present. [2025-10-20T18:05:45.100749] [DEBUG] [VALIDATOR ] CPNG: Primary data validation passed. [2025-10-20T18:05:45.101571] [INFO ] [MAIN ] CPNG: Fetched primary data is valid. [2025-10-20T18:05:45.101612] [DEBUG] [LOCAL ] CPNG: Saving primary data. [2025-10-20T18:05:45.102988] [DEBUG] [LOCAL ] CPNG: Trimmed 77789 records to 53680 (keeping 200 days of data) [2025-10-20T18:05:45.420313] [DEBUG] [LOCAL ] CPNG: Primary data saved (53680 records). [2025-10-20T18:05:45.420452] [INFO ] [MAIN ] CPNG: Full history saved. [2025-10-20T18:05:45.420744] [INFO ] [MAIN ] Update Progress: 520/2367 symbols completed. CPNG took 33.7s. Estimated remaining: 3h 53m 18s. Rate: 7.9 symbols/min [2025-10-20T18:05:45.951600] [INFO ] [MAIN ] CPS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:46.111016] [INFO ] [MAIN ] CR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:46.114916] [INFO ] [MAIN ] CQP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:46.169074] [INFO ] [MAIN ] CPT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:46.198501] [DEBUG] [LOCAL ] CPS: Loaded 25505 existing records from primary storage. [2025-10-20T18:05:46.199075] [INFO ] [MAIN ] CPS: Loaded 25505 existing records. [2025-10-20T18:05:46.277178] [DEBUG] [LOCAL ] CQP: Loaded 19156 existing records from primary storage. [2025-10-20T18:05:46.293633] [INFO ] [MAIN ] CQP: Loaded 19156 existing records. [2025-10-20T18:05:46.314700] [DEBUG] [LOCAL ] CR: Loaded 24488 existing records from primary storage. [2025-10-20T18:05:46.407871] [INFO ] [MAIN ] CR: Loaded 24488 existing records. [2025-10-20T18:05:46.585637] [WARN ] [MAIN ] CQP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:46.713421] [WARN ] [MAIN ] CPS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:46.705203] [INFO ] [MAIN ] CQP: Wiping existing primary data before full refresh. [2025-10-20T18:05:46.728119] [DEBUG] [LOCAL ] CPT: Loaded 45243 existing records from primary storage. [2025-10-20T18:05:46.728303] [INFO ] [MAIN ] CPS: Wiping existing primary data before full refresh. [2025-10-20T18:05:46.728416] [INFO ] [LOCAL ] CQP: Starting deletion of primary data. [2025-10-20T18:05:46.731848] [INFO ] [MAIN ] CPT: Loaded 45243 existing records. [2025-10-20T18:05:46.731948] [INFO ] [LOCAL ] CPS: Starting deletion of primary data. [2025-10-20T18:05:46.737370] [DEBUG] [LOCAL ] CQP: Deleted primary data file. [2025-10-20T18:05:46.761601] [DEBUG] [LOCAL ] CPS: Deleted primary data file. [2025-10-20T18:05:46.761561] [INFO ] [MAIN ] CQP: Starting full primary refresh. [2025-10-20T18:05:46.804768] [INFO ] [MAIN ] CPS: Starting full primary refresh. [2025-10-20T18:05:47.007736] [WARN ] [MAIN ] CR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:47.129699] [INFO ] [MAIN ] CR: Wiping existing primary data before full refresh. [2025-10-20T18:05:47.154364] [INFO ] [LOCAL ] CR: Starting deletion of primary data. [2025-10-20T18:05:47.314629] [DEBUG] [LOCAL ] CR: Deleted primary data file. [2025-10-20T18:05:47.342302] [WARN ] [MAIN ] CPT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:47.350064] [INFO ] [MAIN ] CPT: Wiping existing primary data before full refresh. [2025-10-20T18:05:47.366956] [INFO ] [MAIN ] CR: Starting full primary refresh. [2025-10-20T18:05:47.408039] [INFO ] [LOCAL ] CPT: Starting deletion of primary data. [2025-10-20T18:05:47.618577] [DEBUG] [LOCAL ] CPT: Deleted primary data file. [2025-10-20T18:05:47.628949] [INFO ] [MAIN ] CPT: Starting full primary refresh. [2025-10-20T18:05:48.204753] [INFO ] [REMOTE ] CQP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:48.234290] [INFO ] [REMOTE ] CPS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:48.248937] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-01. [2025-10-20T18:05:48.266376] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-01. [2025-10-20T18:05:48.604282] [INFO ] [REMOTE ] CR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:48.611325] [INFO ] [REMOTE ] CPT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:48.611829] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-01. [2025-10-20T18:05:48.612158] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-01. [2025-10-20T18:05:49.269876] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:05:49.276250] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-02. [2025-10-20T18:05:49.563167] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T18:05:49.567871] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-02. [2025-10-20T18:05:49.877184] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T18:05:49.882633] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-02. [2025-10-20T18:05:50.329134] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T18:05:50.337947] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-02. [2025-10-20T18:05:50.390244] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T18:05:50.395911] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-03. [2025-10-20T18:05:50.473245] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T18:05:50.478591] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-03. [2025-10-20T18:05:50.714547] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T18:05:50.719070] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-03. [2025-10-20T18:05:51.125807] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T18:05:51.130529] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-04. [2025-10-20T18:05:51.712372] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T18:05:51.721564] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-03. [2025-10-20T18:05:51.759762] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T18:05:51.766242] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-04. [2025-10-20T18:05:51.838199] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:05:51.844014] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-04. [2025-10-20T18:05:52.199197] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:05:52.204051] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-05. [2025-10-20T18:05:52.856371] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T18:05:52.864218] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-05. [2025-10-20T18:05:53.111213] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:05:53.118968] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-05. [2025-10-20T18:05:53.299020] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T18:05:53.332762] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:05:53.628018] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T18:05:53.851425] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-04. [2025-10-20T18:05:54.455321] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:05:54.651414] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-06. [2025-10-20T18:05:55.308184] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:05:55.392611] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T18:05:55.451391] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-07. [2025-10-20T18:05:56.251683] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-05. [2025-10-20T18:05:56.260477] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T18:05:57.051389] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-06. [2025-10-20T18:05:57.479091] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T18:05:57.851364] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-08. [2025-10-20T18:05:58.072844] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T18:05:58.651395] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-06. [2025-10-20T18:05:58.833435] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:05:59.451396] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-06. [2025-10-20T18:06:00.204484] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T18:06:00.251336] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-07. [2025-10-20T18:06:00.876066] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:06:01.051414] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-09. [2025-10-20T18:06:01.851396] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-07. [2025-10-20T18:06:02.170847] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T18:06:02.554763] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-07 (2.30 seconds). [2025-10-20T18:06:02.651420] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-08. [2025-10-20T18:06:03.022955] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T18:06:03.451421] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-08. [2025-10-20T18:06:04.251402] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-10. [2025-10-20T18:06:04.349042] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T18:06:04.394037] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T18:06:05.051411] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-09. [2025-10-20T18:06:05.470919] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T18:06:05.474559] [INFO ] [REMOTE ] CR: Full fetch complete (34798 records). [2025-10-20T18:06:05.474699] [DEBUG] [VALIDATOR ] CR: Validating primary data. [2025-10-20T18:06:05.827616] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T18:06:05.874685] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-07. [2025-10-20T18:06:05.902978] [DEBUG] [VALIDATOR ] CR: All expected trading days are present. [2025-10-20T18:06:05.903446] [DEBUG] [VALIDATOR ] CR: Primary data validation passed. [2025-10-20T18:06:05.904101] [INFO ] [MAIN ] CR: Fetched primary data is valid. [2025-10-20T18:06:05.904165] [DEBUG] [LOCAL ] CR: Saving primary data. [2025-10-20T18:06:05.905109] [DEBUG] [LOCAL ] CR: Trimmed 34798 records to 24066 (keeping 200 days of data) [2025-10-20T18:06:06.056104] [DEBUG] [LOCAL ] CR: Primary data saved (24066 records). [2025-10-20T18:06:06.056225] [INFO ] [MAIN ] CR: Full history saved. [2025-10-20T18:06:06.056298] [INFO ] [MAIN ] Update Progress: 521/2367 symbols completed. CR took 20.6s. Estimated remaining: 3h 53m 57s. Rate: 7.9 symbols/min [2025-10-20T18:06:06.231522] [INFO ] [MAIN ] CRBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:06.244733] [DEBUG] [LOCAL ] CRBD: Loaded 9015 existing records from primary storage. [2025-10-20T18:06:06.244918] [INFO ] [MAIN ] CRBD: Loaded 9015 existing records. [2025-10-20T18:06:06.418512] [WARN ] [MAIN ] CRBD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:06.418737] [INFO ] [MAIN ] CRBD: Wiping existing primary data before full refresh. [2025-10-20T18:06:06.418821] [INFO ] [LOCAL ] CRBD: Starting deletion of primary data. [2025-10-20T18:06:06.419180] [DEBUG] [LOCAL ] CRBD: Deleted primary data file. [2025-10-20T18:06:06.419265] [INFO ] [MAIN ] CRBD: Starting full primary refresh. [2025-10-20T18:06:06.657890] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-09. [2025-10-20T18:06:06.782854] [INFO ] [REMOTE ] CRBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:06.796311] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T18:06:07.451374] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-08. [2025-10-20T18:06:08.251359] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-10. [2025-10-20T18:06:08.468332] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T18:06:08.571500] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T18:06:08.905307] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T18:06:08.908420] [INFO ] [REMOTE ] CQP: Full fetch complete (29770 records). [2025-10-20T18:06:08.908511] [DEBUG] [VALIDATOR ] CQP: Validating primary data. [2025-10-20T18:06:08.909114] [FATAL] [VALIDATOR ] CQP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:06:08.909260] [FATAL] [MAIN ] CQP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:06:08.909336] [WARN ] [MAIN ] CQP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:06:08.909427] [INFO ] [LOCAL ] CQP: Starting deletion of primary data. [2025-10-20T18:06:08.909577] [DEBUG] [LOCAL ] CQP: No primary data file found to delete. [2025-10-20T18:06:08.909730] [INFO ] [LOCAL ] CQP: Starting deletion of derivative data. [2025-10-20T18:06:08.910133] [DEBUG] [LOCAL ] CQP: Deleted derivative data file. [2025-10-20T18:06:08.910386] [INFO ] [MAIN ] Update Progress: 522/2367 symbols completed. CQP took 23.5s. Estimated remaining: 3h 53m 32s. Rate: 7.9 symbols/min [2025-10-20T18:06:09.056491] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-10. [2025-10-20T18:06:09.088095] [INFO ] [MAIN ] CRBG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:09.089100] [DEBUG] [LOCAL ] CRBG: No existing data file found in primary storage. [2025-10-20T18:06:09.089146] [WARN ] [MAIN ] CRBG: No existing data found, performing full refresh instead. [2025-10-20T18:06:09.089257] [INFO ] [MAIN ] CRBG: Wiping existing primary data before full refresh. [2025-10-20T18:06:09.089299] [INFO ] [LOCAL ] CRBG: Starting deletion of primary data. [2025-10-20T18:06:09.089339] [DEBUG] [LOCAL ] CRBG: No primary data file found to delete. [2025-10-20T18:06:09.089370] [INFO ] [MAIN ] CRBG: Starting full primary refresh. [2025-10-20T18:06:09.455456] [INFO ] [REMOTE ] CRBG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:09.851442] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-09. [2025-10-20T18:06:10.651365] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-01. [2025-10-20T18:06:10.962883] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:06:10.985575] [INFO ] [REMOTE ] CPT: Full fetch complete (64633 records). [2025-10-20T18:06:10.986487] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T18:06:10.991718] [DEBUG] [VALIDATOR ] CPT: Validating primary data. [2025-10-20T18:06:11.420061] [DEBUG] [VALIDATOR ] CPT: All expected trading days are present. [2025-10-20T18:06:11.420218] [DEBUG] [VALIDATOR ] CPT: Primary data validation passed. [2025-10-20T18:06:11.421601] [INFO ] [MAIN ] CPT: Fetched primary data is valid. [2025-10-20T18:06:11.421695] [DEBUG] [LOCAL ] CPT: Saving primary data. [2025-10-20T18:06:11.423816] [DEBUG] [LOCAL ] CPT: Trimmed 64633 records to 44551 (keeping 200 days of data) [2025-10-20T18:06:11.485988] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-01. [2025-10-20T18:06:11.560163] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T18:06:11.745324] [DEBUG] [LOCAL ] CPT: Primary data saved (44551 records). [2025-10-20T18:06:11.745481] [INFO ] [MAIN ] CPT: Full history saved. [2025-10-20T18:06:11.745569] [INFO ] [MAIN ] Update Progress: 523/2367 symbols completed. CPT took 26.3s. Estimated remaining: 3h 53m 8s. Rate: 7.9 symbols/min [2025-10-20T18:06:11.944560] [INFO ] [MAIN ] CRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:11.986737] [DEBUG] [LOCAL ] CRC: Loaded 44717 existing records from primary storage. [2025-10-20T18:06:11.987050] [INFO ] [MAIN ] CRC: Loaded 44717 existing records. [2025-10-20T18:06:12.188872] [WARN ] [MAIN ] CRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:12.189120] [INFO ] [MAIN ] CRC: Wiping existing primary data before full refresh. [2025-10-20T18:06:12.189170] [INFO ] [LOCAL ] CRC: Starting deletion of primary data. [2025-10-20T18:06:12.189724] [DEBUG] [LOCAL ] CRC: Deleted primary data file. [2025-10-20T18:06:12.189811] [INFO ] [MAIN ] CRC: Starting full primary refresh. [2025-10-20T18:06:12.264229] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-02. [2025-10-20T18:06:12.620961] [INFO ] [REMOTE ] CRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:13.019910] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T18:06:13.051340] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-03. [2025-10-20T18:06:13.306623] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T18:06:13.810965] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T18:06:13.851353] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-04. [2025-10-20T18:06:14.517339] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T18:06:14.651447] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-02. [2025-10-20T18:06:15.451431] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-10. [2025-10-20T18:06:16.251395] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-01. [2025-10-20T18:06:16.310490] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:06:16.526718] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T18:06:16.530171] [INFO ] [REMOTE ] CPS: Full fetch complete (36419 records). [2025-10-20T18:06:16.530265] [DEBUG] [VALIDATOR ] CPS: Validating primary data. [2025-10-20T18:06:16.933678] [DEBUG] [VALIDATOR ] CPS: All expected trading days are present. [2025-10-20T18:06:16.933772] [DEBUG] [VALIDATOR ] CPS: Primary data validation passed. [2025-10-20T18:06:16.934298] [INFO ] [MAIN ] CPS: Fetched primary data is valid. [2025-10-20T18:06:16.934339] [DEBUG] [LOCAL ] CPS: Saving primary data. [2025-10-20T18:06:16.935147] [DEBUG] [LOCAL ] CPS: Trimmed 36419 records to 25047 (keeping 200 days of data) [2025-10-20T18:06:17.056133] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-05. [2025-10-20T18:06:17.096713] [DEBUG] [LOCAL ] CPS: Primary data saved (25047 records). [2025-10-20T18:06:17.096857] [INFO ] [MAIN ] CPS: Full history saved. [2025-10-20T18:06:17.096944] [INFO ] [MAIN ] Update Progress: 524/2367 symbols completed. CPS took 31.7s. Estimated remaining: 3h 52m 53s. Rate: 7.9 symbols/min [2025-10-20T18:06:17.274330] [INFO ] [MAIN ] CRCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:17.274522] [DEBUG] [LOCAL ] CRCL: No existing data file found in primary storage. [2025-10-20T18:06:17.274562] [WARN ] [MAIN ] CRCL: No existing data found, performing full refresh instead. [2025-10-20T18:06:17.274702] [INFO ] [MAIN ] CRCL: Wiping existing primary data before full refresh. [2025-10-20T18:06:17.274755] [INFO ] [LOCAL ] CRCL: Starting deletion of primary data. [2025-10-20T18:06:17.274798] [DEBUG] [LOCAL ] CRCL: No primary data file found to delete. [2025-10-20T18:06:17.274876] [INFO ] [MAIN ] CRCL: Starting full primary refresh. [2025-10-20T18:06:17.623156] [INFO ] [REMOTE ] CRCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:17.856556] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-03. [2025-10-20T18:06:17.878105] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:06:17.887206] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:06:18.651399] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-02. [2025-10-20T18:06:19.224372] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T18:06:19.451444] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-04. [2025-10-20T18:06:19.707368] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T18:06:20.251400] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-01. [2025-10-20T18:06:20.516537] [DEBUG] [REMOTE ] CRCL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:06:20.517525] [WARN ] [REMOTE ] CRCL: No data returned for month 2025-01 [2025-10-20T18:06:20.517966] [FATAL] [REMOTE ] CRCL: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:06:20.518050] [FATAL] [MAIN ] CRCL: Failed to fetch primary data. [2025-10-20T18:06:20.518097] [WARN ] [MAIN ] CRCL: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:06:20.518170] [INFO ] [LOCAL ] CRCL: Starting deletion of primary data. [2025-10-20T18:06:20.518255] [DEBUG] [LOCAL ] CRCL: No primary data file found to delete. [2025-10-20T18:06:20.518296] [INFO ] [LOCAL ] CRCL: Starting deletion of derivative data. [2025-10-20T18:06:20.518338] [DEBUG] [LOCAL ] CRCL: No derivative data file found to delete. [2025-10-20T18:06:20.518400] [INFO ] [MAIN ] Update Progress: 525/2367 symbols completed. CRCL took 3.4s. Estimated remaining: 3h 52m 30s. Rate: 7.9 symbols/min [2025-10-20T18:06:20.709455] [INFO ] [MAIN ] CRGY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:20.766992] [DEBUG] [LOCAL ] CRGY: Loaded 53283 existing records from primary storage. [2025-10-20T18:06:20.767490] [INFO ] [MAIN ] CRGY: Loaded 53283 existing records. [2025-10-20T18:06:20.946148] [WARN ] [MAIN ] CRGY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:20.946497] [INFO ] [MAIN ] CRGY: Wiping existing primary data before full refresh. [2025-10-20T18:06:20.946694] [INFO ] [LOCAL ] CRGY: Starting deletion of primary data. [2025-10-20T18:06:20.947022] [DEBUG] [LOCAL ] CRGY: Deleted primary data file. [2025-10-20T18:06:20.947142] [INFO ] [MAIN ] CRGY: Starting full primary refresh. [2025-10-20T18:06:21.056504] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-03. [2025-10-20T18:06:21.345014] [INFO ] [REMOTE ] CRGY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:21.350395] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T18:06:21.851378] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-01. [2025-10-20T18:06:22.651383] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-05. [2025-10-20T18:06:22.756368] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:06:23.451393] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-06. [2025-10-20T18:06:23.639330] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:06:24.251367] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-02. [2025-10-20T18:06:24.328214] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:06:24.374901] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T18:06:25.051372] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-07. [2025-10-20T18:06:25.770152] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T18:06:25.809610] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:06:25.851338] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-03. [2025-10-20T18:06:26.651415] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-04. [2025-10-20T18:06:27.451439] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-06. [2025-10-20T18:06:27.655254] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T18:06:28.256447] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-08. [2025-10-20T18:06:28.269544] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:06:28.813908] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T18:06:29.029995] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T18:06:29.051348] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-07. [2025-10-20T18:06:29.851369] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-04. [2025-10-20T18:06:30.409411] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T18:06:30.651419] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-08. [2025-10-20T18:06:31.451374] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-05. [2025-10-20T18:06:31.922618] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-10-20T18:06:32.070800] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T18:06:32.251373] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-09. [2025-10-20T18:06:32.770171] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T18:06:33.051384] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-05. [2025-10-20T18:06:33.851466] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-06. [2025-10-20T18:06:33.979030] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T18:06:34.651385] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-09. [2025-10-20T18:06:34.949356] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T18:06:35.013984] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T18:06:35.227803] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T18:06:35.451374] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-10. [2025-10-20T18:06:36.191504] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T18:06:36.193175] [INFO ] [REMOTE ] CRBD: Full fetch complete (13026 records). [2025-10-20T18:06:36.193245] [DEBUG] [VALIDATOR ] CRBD: Validating primary data. [2025-10-20T18:06:36.260841] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-06. [2025-10-20T18:06:36.595193] [DEBUG] [VALIDATOR ] CRBD: All expected trading days are present. [2025-10-20T18:06:36.595270] [DEBUG] [VALIDATOR ] CRBD: Primary data validation passed. [2025-10-20T18:06:36.595590] [INFO ] [MAIN ] CRBD: Fetched primary data is valid. [2025-10-20T18:06:36.595628] [DEBUG] [LOCAL ] CRBD: Saving primary data. [2025-10-20T18:06:36.596152] [DEBUG] [LOCAL ] CRBD: Trimmed 13026 records to 8858 (keeping 200 days of data) [2025-10-20T18:06:36.649978] [DEBUG] [LOCAL ] CRBD: Primary data saved (8858 records). [2025-10-20T18:06:36.650099] [INFO ] [MAIN ] CRBD: Full history saved. [2025-10-20T18:06:36.650178] [INFO ] [MAIN ] Update Progress: 526/2367 symbols completed. CRBD took 30.6s. Estimated remaining: 3h 52m 53s. Rate: 7.9 symbols/min [2025-10-20T18:06:36.824299] [INFO ] [MAIN ] CRH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:36.876398] [DEBUG] [LOCAL ] CRH: Loaded 54220 existing records from primary storage. [2025-10-20T18:06:36.876757] [INFO ] [MAIN ] CRH: Loaded 54220 existing records. [2025-10-20T18:06:37.063631] [WARN ] [MAIN ] CRH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:37.058077] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-10. [2025-10-20T18:06:37.064812] [INFO ] [MAIN ] CRH: Wiping existing primary data before full refresh. [2025-10-20T18:06:37.064871] [INFO ] [LOCAL ] CRH: Starting deletion of primary data. [2025-10-20T18:06:37.065087] [DEBUG] [LOCAL ] CRH: Deleted primary data file. [2025-10-20T18:06:37.065144] [INFO ] [MAIN ] CRH: Starting full primary refresh. [2025-10-20T18:06:37.426410] [INFO ] [REMOTE ] CRH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:37.851404] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-07. [2025-10-20T18:06:38.030327] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:06:38.651395] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-07. [2025-10-20T18:06:39.038333] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T18:06:39.046916] [INFO ] [REMOTE ] CRBG: Full fetch complete (76047 records). [2025-10-20T18:06:39.047240] [DEBUG] [VALIDATOR ] CRBG: Validating primary data. [2025-10-20T18:06:39.354506] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T18:06:39.456097] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-08. [2025-10-20T18:06:39.494526] [DEBUG] [VALIDATOR ] CRBG: All expected trading days are present. [2025-10-20T18:06:39.494619] [DEBUG] [VALIDATOR ] CRBG: Primary data validation passed. [2025-10-20T18:06:39.495540] [INFO ] [MAIN ] CRBG: Fetched primary data is valid. [2025-10-20T18:06:39.495598] [DEBUG] [LOCAL ] CRBG: Saving primary data. [2025-10-20T18:06:39.497232] [DEBUG] [LOCAL ] CRBG: Trimmed 76047 records to 52602 (keeping 200 days of data) [2025-10-20T18:06:39.810256] [DEBUG] [LOCAL ] CRBG: Primary data saved (52602 records). [2025-10-20T18:06:39.810401] [INFO ] [MAIN ] CRBG: Full history saved. [2025-10-20T18:06:39.810474] [INFO ] [MAIN ] Update Progress: 527/2367 symbols completed. CRBG took 30.9s. Estimated remaining: 3h 52m 30s. Rate: 7.9 symbols/min [2025-10-20T18:06:39.979372] [INFO ] [MAIN ] CRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:39.979561] [DEBUG] [LOCAL ] CRI: No existing data file found in primary storage. [2025-10-20T18:06:39.979639] [WARN ] [MAIN ] CRI: No existing data found, performing full refresh instead. [2025-10-20T18:06:39.979787] [INFO ] [MAIN ] CRI: Wiping existing primary data before full refresh. [2025-10-20T18:06:39.979831] [INFO ] [LOCAL ] CRI: Starting deletion of primary data. [2025-10-20T18:06:39.979876] [DEBUG] [LOCAL ] CRI: No primary data file found to delete. [2025-10-20T18:06:39.979912] [INFO ] [MAIN ] CRI: Starting full primary refresh. [2025-10-20T18:06:40.258637] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-01. [2025-10-20T18:06:40.406849] [INFO ] [REMOTE ] CRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:41.051349] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-01. [2025-10-20T18:06:41.102475] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-07 (2.45 seconds). [2025-10-20T18:06:41.864266] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-08. [2025-10-20T18:06:41.865956] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-08 (2.41 seconds). [2025-10-20T18:06:42.651433] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-09. [2025-10-20T18:06:43.173156] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-01 (2.91 seconds). [2025-10-20T18:06:43.451421] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-02. [2025-10-20T18:06:43.652956] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T18:06:43.890374] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T18:06:44.028726] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-01 (2.98 seconds). [2025-10-20T18:06:44.251480] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-10. [2025-10-20T18:06:44.901012] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T18:06:45.051366] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-03. [2025-10-20T18:06:45.851360] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-09. [2025-10-20T18:06:46.098979] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:06:46.108905] [INFO ] [REMOTE ] CRC: Full fetch complete (62964 records). [2025-10-20T18:06:46.109083] [DEBUG] [VALIDATOR ] CRC: Validating primary data. [2025-10-20T18:06:46.495861] [DEBUG] [VALIDATOR ] CRC: All expected trading days are present. [2025-10-20T18:06:46.495953] [DEBUG] [VALIDATOR ] CRC: Primary data validation passed. [2025-10-20T18:06:46.496711] [INFO ] [MAIN ] CRC: Fetched primary data is valid. [2025-10-20T18:06:46.496775] [DEBUG] [LOCAL ] CRC: Saving primary data. [2025-10-20T18:06:46.498304] [DEBUG] [LOCAL ] CRC: Trimmed 62964 records to 44124 (keeping 200 days of data) [2025-10-20T18:06:46.667291] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-02. [2025-10-20T18:06:46.761985] [DEBUG] [LOCAL ] CRC: Primary data saved (44124 records). [2025-10-20T18:06:46.762130] [INFO ] [MAIN ] CRC: Full history saved. [2025-10-20T18:06:46.762212] [INFO ] [MAIN ] Update Progress: 528/2367 symbols completed. CRC took 35.0s. Estimated remaining: 3h 52m 20s. Rate: 7.9 symbols/min [2025-10-20T18:06:46.938692] [INFO ] [MAIN ] CRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:46.991887] [DEBUG] [LOCAL ] CRK: Loaded 53025 existing records from primary storage. [2025-10-20T18:06:46.992212] [INFO ] [MAIN ] CRK: Loaded 53025 existing records. [2025-10-20T18:06:47.185945] [WARN ] [MAIN ] CRK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:47.186292] [INFO ] [MAIN ] CRK: Wiping existing primary data before full refresh. [2025-10-20T18:06:47.186349] [INFO ] [LOCAL ] CRK: Starting deletion of primary data. [2025-10-20T18:06:47.186533] [DEBUG] [LOCAL ] CRK: Deleted primary data file. [2025-10-20T18:06:47.186572] [INFO ] [MAIN ] CRK: Starting full primary refresh. [2025-10-20T18:06:47.354011] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-03 (2.30 seconds). [2025-10-20T18:06:47.461035] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-04. [2025-10-20T18:06:47.610062] [INFO ] [REMOTE ] CRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:47.770269] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T18:06:47.909767] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T18:06:48.251443] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-03. [2025-10-20T18:06:49.051415] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-10. [2025-10-20T18:06:49.499615] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T18:06:49.851408] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-05. [2025-10-20T18:06:49.956282] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:06:50.590050] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T18:06:50.600702] [INFO ] [REMOTE ] CRGY: Full fetch complete (76159 records). [2025-10-20T18:06:50.600852] [DEBUG] [VALIDATOR ] CRGY: Validating primary data. [2025-10-20T18:06:50.656663] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-01. [2025-10-20T18:06:50.992795] [DEBUG] [VALIDATOR ] CRGY: All expected trading days are present. [2025-10-20T18:06:50.992877] [DEBUG] [VALIDATOR ] CRGY: Primary data validation passed. [2025-10-20T18:06:50.993732] [INFO ] [MAIN ] CRGY: Fetched primary data is valid. [2025-10-20T18:06:50.993796] [DEBUG] [LOCAL ] CRGY: Saving primary data. [2025-10-20T18:06:50.995182] [DEBUG] [LOCAL ] CRGY: Trimmed 76159 records to 52514 (keeping 200 days of data) [2025-10-20T18:06:51.299621] [DEBUG] [LOCAL ] CRGY: Primary data saved (52514 records). [2025-10-20T18:06:51.299783] [INFO ] [MAIN ] CRGY: Full history saved. [2025-10-20T18:06:51.299863] [INFO ] [MAIN ] Update Progress: 529/2367 symbols completed. CRGY took 30.8s. Estimated remaining: 3h 52m 2s. Rate: 7.9 symbols/min [2025-10-20T18:06:51.471865] [INFO ] [MAIN ] CRL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:51.468347] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-04. [2025-10-20T18:06:51.507202] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T18:06:51.550293] [DEBUG] [LOCAL ] CRL: Loaded 43452 existing records from primary storage. [2025-10-20T18:06:51.550605] [INFO ] [MAIN ] CRL: Loaded 43452 existing records. [2025-10-20T18:06:51.738708] [WARN ] [MAIN ] CRL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:51.738958] [INFO ] [MAIN ] CRL: Wiping existing primary data before full refresh. [2025-10-20T18:06:51.739013] [INFO ] [LOCAL ] CRL: Starting deletion of primary data. [2025-10-20T18:06:51.739204] [DEBUG] [LOCAL ] CRL: Deleted primary data file. [2025-10-20T18:06:51.739240] [INFO ] [MAIN ] CRL: Starting full primary refresh. [2025-10-20T18:06:52.118521] [INFO ] [REMOTE ] CRL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:52.251382] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-01. [2025-10-20T18:06:52.556920] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T18:06:53.051465] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-02. [2025-10-20T18:06:53.294862] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:06:53.387936] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T18:06:53.851414] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-02. [2025-10-20T18:06:54.661970] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-05. [2025-10-20T18:06:54.667886] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T18:06:55.431323] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:06:55.451347] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-03. [2025-10-20T18:06:56.251447] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-06. [2025-10-20T18:06:57.051354] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-03. [2025-10-20T18:06:57.555666] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-05 (2.89 seconds). [2025-10-20T18:06:57.834787] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-10-20T18:06:57.851323] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-04. [2025-10-20T18:06:58.425272] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T18:06:58.644256] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:06:58.654784] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-06. [2025-10-20T18:06:59.451406] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-04. [2025-10-20T18:06:59.782860] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T18:06:59.944620] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:07:00.251354] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-05. [2025-10-20T18:07:01.051381] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-07. [2025-10-20T18:07:01.301859] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:07:01.851409] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-05. [2025-10-20T18:07:02.043180] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T18:07:02.651396] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-06. [2025-10-20T18:07:02.798339] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:07:03.451404] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-07. [2025-10-20T18:07:03.748765] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T18:07:03.862269] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T18:07:04.251452] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-07. [2025-10-20T18:07:05.051434] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-06. [2025-10-20T18:07:05.438574] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T18:07:05.638006] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:07:05.851427] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-08. [2025-10-20T18:07:06.651372] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-08. [2025-10-20T18:07:06.692009] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:07:07.443934] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:07:07.458192] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-08. [2025-10-20T18:07:08.015779] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T18:07:08.251391] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-09. [2025-10-20T18:07:08.958432] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T18:07:09.051338] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-09. [2025-10-20T18:07:09.851359] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-07. [2025-10-20T18:07:09.928464] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:07:10.250876] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:07:10.651380] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-09. [2025-10-20T18:07:11.451445] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-10. [2025-10-20T18:07:12.251404] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-10. [2025-10-20T18:07:12.315663] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-07 (2.46 seconds). [2025-10-20T18:07:12.396759] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T18:07:13.051495] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-08. [2025-10-20T18:07:13.260982] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T18:07:13.267965] [INFO ] [REMOTE ] CRI: Full fetch complete (70923 records). [2025-10-20T18:07:13.268134] [DEBUG] [VALIDATOR ] CRI: Validating primary data. [2025-10-20T18:07:13.656932] [DEBUG] [VALIDATOR ] CRI: All expected trading days are present. [2025-10-20T18:07:13.657113] [DEBUG] [VALIDATOR ] CRI: Primary data validation passed. [2025-10-20T18:07:13.658022] [INFO ] [MAIN ] CRI: Fetched primary data is valid. [2025-10-20T18:07:13.658134] [DEBUG] [LOCAL ] CRI: Saving primary data. [2025-10-20T18:07:13.659687] [DEBUG] [LOCAL ] CRI: Trimmed 70923 records to 49604 (keeping 200 days of data) [2025-10-20T18:07:13.853527] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-10. [2025-10-20T18:07:13.866248] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T18:07:13.922100] [INFO ] [REMOTE ] CRL: Full fetch complete (63531 records). [2025-10-20T18:07:13.933534] [DEBUG] [VALIDATOR ] CRL: Validating primary data. [2025-10-20T18:07:14.094023] [DEBUG] [LOCAL ] CRI: Primary data saved (49604 records). [2025-10-20T18:07:14.104234] [INFO ] [MAIN ] CRI: Full history saved. [2025-10-20T18:07:14.104387] [INFO ] [MAIN ] Update Progress: 530/2367 symbols completed. CRI took 34.3s. Estimated remaining: 3h 52m 47s. Rate: 7.9 symbols/min [2025-10-20T18:07:14.464255] [INFO ] [MAIN ] CRM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:14.625342] [DEBUG] [VALIDATOR ] CRL: All expected trading days are present. [2025-10-20T18:07:14.625478] [DEBUG] [VALIDATOR ] CRL: Primary data validation passed. [2025-10-20T18:07:14.626468] [INFO ] [MAIN ] CRL: Fetched primary data is valid. [2025-10-20T18:07:14.626764] [DEBUG] [LOCAL ] CRL: Saving primary data. [2025-10-20T18:07:14.639076] [DEBUG] [LOCAL ] CRL: Trimmed 63531 records to 42666 (keeping 200 days of data) [2025-10-20T18:07:14.662037] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T18:07:14.774798] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-09. [2025-10-20T18:07:14.800363] [DEBUG] [LOCAL ] CRM: Loaded 54550 existing records from primary storage. [2025-10-20T18:07:14.802090] [INFO ] [MAIN ] CRM: Loaded 54550 existing records. [2025-10-20T18:07:15.006628] [WARN ] [MAIN ] CRM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:15.011871] [INFO ] [MAIN ] CRM: Wiping existing primary data before full refresh. [2025-10-20T18:07:15.012676] [INFO ] [LOCAL ] CRM: Starting deletion of primary data. [2025-10-20T18:07:15.014298] [DEBUG] [LOCAL ] CRM: Deleted primary data file. [2025-10-20T18:07:15.016781] [INFO ] [MAIN ] CRM: Starting full primary refresh. [2025-10-20T18:07:15.298120] [DEBUG] [LOCAL ] CRL: Primary data saved (42666 records). [2025-10-20T18:07:15.329822] [INFO ] [MAIN ] CRL: Full history saved. [2025-10-20T18:07:15.350010] [INFO ] [MAIN ] Update Progress: 531/2367 symbols completed. CRL took 24.1s. Estimated remaining: 3h 52m 18s. Rate: 7.9 symbols/min [2025-10-20T18:07:15.726226] [INFO ] [REMOTE ] CRM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:15.747843] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-01. [2025-10-20T18:07:15.805818] [INFO ] [MAIN ] CRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:15.852070] [DEBUG] [LOCAL ] CRS: Loaded 43714 existing records from primary storage. [2025-10-20T18:07:15.852570] [INFO ] [MAIN ] CRS: Loaded 43714 existing records. [2025-10-20T18:07:16.057240] [WARN ] [MAIN ] CRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:16.057634] [INFO ] [MAIN ] CRS: Wiping existing primary data before full refresh. [2025-10-20T18:07:16.057770] [INFO ] [LOCAL ] CRS: Starting deletion of primary data. [2025-10-20T18:07:16.066102] [DEBUG] [LOCAL ] CRS: Deleted primary data file. [2025-10-20T18:07:16.078487] [INFO ] [MAIN ] CRS: Starting full primary refresh. [2025-10-20T18:07:16.168046] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-10 (2.31 seconds). [2025-10-20T18:07:16.206409] [INFO ] [REMOTE ] CRH: Full fetch complete (77553 records). [2025-10-20T18:07:16.227721] [DEBUG] [VALIDATOR ] CRH: Validating primary data. [2025-10-20T18:07:16.761986] [INFO ] [REMOTE ] CRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:16.800770] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-01. [2025-10-20T18:07:16.916370] [DEBUG] [VALIDATOR ] CRH: All expected trading days are present. [2025-10-20T18:07:16.916480] [DEBUG] [VALIDATOR ] CRH: Primary data validation passed. [2025-10-20T18:07:16.917355] [INFO ] [MAIN ] CRH: Fetched primary data is valid. [2025-10-20T18:07:16.917470] [DEBUG] [LOCAL ] CRH: Saving primary data. [2025-10-20T18:07:16.947857] [DEBUG] [LOCAL ] CRH: Trimmed 77553 records to 53439 (keeping 200 days of data) [2025-10-20T18:07:16.949599] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T18:07:17.052916] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-10. [2025-10-20T18:07:17.283459] [DEBUG] [LOCAL ] CRH: Primary data saved (53439 records). [2025-10-20T18:07:17.283590] [INFO ] [MAIN ] CRH: Full history saved. [2025-10-20T18:07:17.283876] [INFO ] [MAIN ] Update Progress: 532/2367 symbols completed. CRH took 40.6s. Estimated remaining: 3h 51m 50s. Rate: 7.9 symbols/min [2025-10-20T18:07:17.452400] [INFO ] [MAIN ] CRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:17.459894] [DEBUG] [LOCAL ] CRT: Loaded 4975 existing records from primary storage. [2025-10-20T18:07:17.460111] [INFO ] [MAIN ] CRT: Loaded 4975 existing records. [2025-10-20T18:07:17.634243] [WARN ] [MAIN ] CRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:17.634825] [INFO ] [MAIN ] CRT: Wiping existing primary data before full refresh. [2025-10-20T18:07:17.634877] [INFO ] [LOCAL ] CRT: Starting deletion of primary data. [2025-10-20T18:07:17.635008] [DEBUG] [LOCAL ] CRT: Deleted primary data file. [2025-10-20T18:07:17.635046] [INFO ] [MAIN ] CRT: Starting full primary refresh. [2025-10-20T18:07:17.994393] [INFO ] [REMOTE ] CRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:17.994787] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-01. [2025-10-20T18:07:18.148727] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-01 (2.40 seconds). [2025-10-20T18:07:18.364240] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T18:07:18.568041] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T18:07:18.651353] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-02. [2025-10-20T18:07:18.940999] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T18:07:18.949250] [INFO ] [REMOTE ] CRK: Full fetch complete (75841 records). [2025-10-20T18:07:18.949421] [DEBUG] [VALIDATOR ] CRK: Validating primary data. [2025-10-20T18:07:19.110112] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T18:07:19.375374] [DEBUG] [VALIDATOR ] CRK: All expected trading days are present. [2025-10-20T18:07:19.375456] [DEBUG] [VALIDATOR ] CRK: Primary data validation passed. [2025-10-20T18:07:19.376454] [INFO ] [MAIN ] CRK: Fetched primary data is valid. [2025-10-20T18:07:19.376526] [DEBUG] [LOCAL ] CRK: Saving primary data. [2025-10-20T18:07:19.378001] [DEBUG] [LOCAL ] CRK: Trimmed 75841 records to 52250 (keeping 200 days of data) [2025-10-20T18:07:19.459132] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-02. [2025-10-20T18:07:19.676080] [DEBUG] [LOCAL ] CRK: Primary data saved (52250 records). [2025-10-20T18:07:19.676206] [INFO ] [MAIN ] CRK: Full history saved. [2025-10-20T18:07:19.676281] [INFO ] [MAIN ] Update Progress: 533/2367 symbols completed. CRK took 32.9s. Estimated remaining: 3h 51m 25s. Rate: 7.9 symbols/min [2025-10-20T18:07:19.846237] [INFO ] [MAIN ] CSAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:19.884087] [DEBUG] [LOCAL ] CSAN: Loaded 42773 existing records from primary storage. [2025-10-20T18:07:19.884366] [INFO ] [MAIN ] CSAN: Loaded 42773 existing records. [2025-10-20T18:07:20.058065] [WARN ] [MAIN ] CSAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:20.058273] [INFO ] [MAIN ] CSAN: Wiping existing primary data before full refresh. [2025-10-20T18:07:20.058320] [INFO ] [LOCAL ] CSAN: Starting deletion of primary data. [2025-10-20T18:07:20.058499] [DEBUG] [LOCAL ] CSAN: Deleted primary data file. [2025-10-20T18:07:20.058557] [INFO ] [MAIN ] CSAN: Starting full primary refresh. [2025-10-20T18:07:20.260714] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-03. [2025-10-20T18:07:20.405950] [INFO ] [REMOTE ] CSAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:20.937025] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T18:07:20.952467] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T18:07:21.051398] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-03. [2025-10-20T18:07:21.851403] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-01. [2025-10-20T18:07:22.351790] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T18:07:22.651400] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-04. [2025-10-20T18:07:23.009545] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:07:23.451395] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-04. [2025-10-20T18:07:24.058869] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:07:24.251378] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-02. [2025-10-20T18:07:24.336534] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:07:25.051369] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-05. [2025-10-20T18:07:25.456835] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T18:07:25.635404] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:07:25.851369] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-06. [2025-10-20T18:07:26.477313] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T18:07:26.651378] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-07. [2025-10-20T18:07:27.266116] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:07:27.451345] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-02. [2025-10-20T18:07:28.251372] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-08. [2025-10-20T18:07:28.966172] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T18:07:29.051392] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-09. [2025-10-20T18:07:29.688091] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-10-20T18:07:29.688381] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T18:07:29.851440] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-10. [2025-10-20T18:07:30.483332] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T18:07:30.484807] [INFO ] [REMOTE ] CRT: Full fetch complete (7361 records). [2025-10-20T18:07:30.484879] [DEBUG] [VALIDATOR ] CRT: Validating primary data. [2025-10-20T18:07:30.653980] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-03. [2025-10-20T18:07:30.877382] [DEBUG] [VALIDATOR ] CRT: All expected trading days are present. [2025-10-20T18:07:30.877494] [DEBUG] [VALIDATOR ] CRT: Primary data validation passed. [2025-10-20T18:07:30.877854] [INFO ] [MAIN ] CRT: Fetched primary data is valid. [2025-10-20T18:07:30.877919] [DEBUG] [LOCAL ] CRT: Saving primary data. [2025-10-20T18:07:30.878376] [DEBUG] [LOCAL ] CRT: Trimmed 7361 records to 4802 (keeping 200 days of data) [2025-10-20T18:07:30.909760] [DEBUG] [LOCAL ] CRT: Primary data saved (4802 records). [2025-10-20T18:07:30.909972] [INFO ] [MAIN ] CRT: Full history saved. [2025-10-20T18:07:30.910089] [INFO ] [MAIN ] Update Progress: 534/2367 symbols completed. CRT took 13.6s. Estimated remaining: 3h 51m 30s. Rate: 7.9 symbols/min [2025-10-20T18:07:31.100193] [INFO ] [MAIN ] CSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:31.136375] [DEBUG] [LOCAL ] CSL: Loaded 31991 existing records from primary storage. [2025-10-20T18:07:31.136805] [INFO ] [MAIN ] CSL: Loaded 31991 existing records. [2025-10-20T18:07:31.326266] [WARN ] [MAIN ] CSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:31.326596] [INFO ] [MAIN ] CSL: Wiping existing primary data before full refresh. [2025-10-20T18:07:31.326672] [INFO ] [LOCAL ] CSL: Starting deletion of primary data. [2025-10-20T18:07:31.326979] [DEBUG] [LOCAL ] CSL: Deleted primary data file. [2025-10-20T18:07:31.327037] [INFO ] [MAIN ] CSL: Starting full primary refresh. [2025-10-20T18:07:31.456507] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-03. [2025-10-20T18:07:31.725906] [INFO ] [REMOTE ] CSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:32.006276] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T18:07:32.251403] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-04. [2025-10-20T18:07:33.051374] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-01. [2025-10-20T18:07:33.522165] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T18:07:33.567441] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T18:07:33.851350] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-05. [2025-10-20T18:07:34.366063] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T18:07:34.651380] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-05. [2025-10-20T18:07:35.186742] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:07:35.451377] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-04. [2025-10-20T18:07:35.672358] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-10-20T18:07:36.251380] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-06. [2025-10-20T18:07:37.051371] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-06. [2025-10-20T18:07:37.629408] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T18:07:37.851354] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-05. [2025-10-20T18:07:37.977752] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T18:07:38.553226] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T18:07:38.651367] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-07. [2025-10-20T18:07:39.451383] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-07. [2025-10-20T18:07:39.834126] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T18:07:40.006047] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T18:07:40.251612] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-06. [2025-10-20T18:07:41.052837] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-08. [2025-10-20T18:07:41.052754] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T18:07:41.851425] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-08. [2025-10-20T18:07:42.615706] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T18:07:42.656554] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-09. [2025-10-20T18:07:42.679792] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-06 (2.43 seconds). [2025-10-20T18:07:42.915737] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-10-20T18:07:43.451421] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-09. [2025-10-20T18:07:44.251370] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-02. [2025-10-20T18:07:44.290639] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:07:45.051431] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-10. [2025-10-20T18:07:45.116714] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T18:07:45.851354] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-10. [2025-10-20T18:07:45.894708] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T18:07:46.651415] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-03. [2025-10-20T18:07:46.764248] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T18:07:46.773784] [INFO ] [REMOTE ] CRS: Full fetch complete (62496 records). [2025-10-20T18:07:46.773980] [DEBUG] [VALIDATOR ] CRS: Validating primary data. [2025-10-20T18:07:47.180006] [DEBUG] [VALIDATOR ] CRS: All expected trading days are present. [2025-10-20T18:07:47.180085] [DEBUG] [VALIDATOR ] CRS: Primary data validation passed. [2025-10-20T18:07:47.180833] [INFO ] [MAIN ] CRS: Fetched primary data is valid. [2025-10-20T18:07:47.180895] [DEBUG] [LOCAL ] CRS: Saving primary data. [2025-10-20T18:07:47.182146] [DEBUG] [LOCAL ] CRS: Trimmed 62496 records to 43124 (keeping 200 days of data) [2025-10-20T18:07:47.388415] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T18:07:47.457936] [INFO ] [REMOTE ] CSAN: Full fetch complete (60284 records). [2025-10-20T18:07:47.457717] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-07. [2025-10-20T18:07:47.476493] [DEBUG] [LOCAL ] CRS: Primary data saved (43124 records). [2025-10-20T18:07:47.467265] [DEBUG] [VALIDATOR ] CSAN: Validating primary data. [2025-10-20T18:07:47.478000] [INFO ] [MAIN ] CRS: Full history saved. [2025-10-20T18:07:47.478110] [INFO ] [MAIN ] Update Progress: 535/2367 symbols completed. CRS took 32.1s. Estimated remaining: 3h 51m 53s. Rate: 7.9 symbols/min [2025-10-20T18:07:47.827116] [INFO ] [MAIN ] CSR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:47.996356] [DEBUG] [LOCAL ] CSR: Loaded 13649 existing records from primary storage. [2025-10-20T18:07:48.001178] [INFO ] [MAIN ] CSR: Loaded 13649 existing records. [2025-10-20T18:07:48.216374] [DEBUG] [VALIDATOR ] CSAN: All expected trading days are present. [2025-10-20T18:07:48.216560] [DEBUG] [VALIDATOR ] CSAN: Primary data validation passed. [2025-10-20T18:07:48.217387] [INFO ] [MAIN ] CSAN: Fetched primary data is valid. [2025-10-20T18:07:48.217499] [DEBUG] [LOCAL ] CSAN: Saving primary data. [2025-10-20T18:07:48.297594] [WARN ] [MAIN ] CSR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:48.297764] [DEBUG] [LOCAL ] CSAN: Trimmed 60284 records to 42287 (keeping 200 days of data) [2025-10-20T18:07:48.298056] [INFO ] [MAIN ] CSR: Wiping existing primary data before full refresh. [2025-10-20T18:07:48.298150] [INFO ] [LOCAL ] CSR: Starting deletion of primary data. [2025-10-20T18:07:48.299006] [DEBUG] [LOCAL ] CSR: Deleted primary data file. [2025-10-20T18:07:48.299130] [INFO ] [MAIN ] CSR: Starting full primary refresh. [2025-10-20T18:07:48.537287] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:07:48.607902] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-04. [2025-10-20T18:07:48.822547] [INFO ] [REMOTE ] CSR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:48.947630] [DEBUG] [LOCAL ] CSAN: Primary data saved (42287 records). [2025-10-20T18:07:48.947794] [INFO ] [MAIN ] CSAN: Full history saved. [2025-10-20T18:07:48.947875] [INFO ] [MAIN ] Update Progress: 536/2367 symbols completed. CSAN took 29.3s. Estimated remaining: 3h 51m 25s. Rate: 7.9 symbols/min [2025-10-20T18:07:49.056595] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-01. [2025-10-20T18:07:49.118762] [INFO ] [MAIN ] CSTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:49.163407] [DEBUG] [LOCAL ] CSTM: Loaded 48273 existing records from primary storage. [2025-10-20T18:07:49.163795] [INFO ] [MAIN ] CSTM: Loaded 48273 existing records. [2025-10-20T18:07:49.345203] [WARN ] [MAIN ] CSTM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:49.345569] [INFO ] [MAIN ] CSTM: Wiping existing primary data before full refresh. [2025-10-20T18:07:49.345628] [INFO ] [LOCAL ] CSTM: Starting deletion of primary data. [2025-10-20T18:07:49.346206] [DEBUG] [LOCAL ] CSTM: Deleted primary data file. [2025-10-20T18:07:49.346264] [INFO ] [MAIN ] CSTM: Starting full primary refresh. [2025-10-20T18:07:49.700326] [INFO ] [REMOTE ] CSTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:49.851393] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-01. [2025-10-20T18:07:50.046242] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-07 (2.59 seconds). [2025-10-20T18:07:50.068092] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T18:07:50.229404] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:07:50.651432] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-02. [2025-10-20T18:07:51.163437] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T18:07:51.451366] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-03. [2025-10-20T18:07:51.542381] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:07:52.010675] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T18:07:52.251425] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-05. [2025-10-20T18:07:53.051409] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-04. [2025-10-20T18:07:53.211718] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T18:07:53.739084] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T18:07:53.851384] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-06. [2025-10-20T18:07:54.651394] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-05. [2025-10-20T18:07:55.040062] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T18:07:55.241267] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:07:55.451399] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-07. [2025-10-20T18:07:56.251380] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-08. [2025-10-20T18:07:56.879246] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:07:57.051454] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-02. [2025-10-20T18:07:57.851434] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-08. [2025-10-20T18:07:58.308968] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T18:07:58.571299] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:07:58.651353] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-03. [2025-10-20T18:07:59.245784] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T18:07:59.451407] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-09. [2025-10-20T18:07:59.942008] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T18:08:00.251441] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-09. [2025-10-20T18:08:01.051389] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-04. [2025-10-20T18:08:01.673798] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T18:08:01.851345] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-06. [2025-10-20T18:08:02.295608] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-09 (2.84 seconds). [2025-10-20T18:08:02.651351] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-10. [2025-10-20T18:08:02.897225] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:08:02.908353] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:08:03.141202] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T18:08:03.149007] [INFO ] [REMOTE ] CRM: Full fetch complete (77946 records). [2025-10-20T18:08:03.149132] [DEBUG] [VALIDATOR ] CRM: Validating primary data. [2025-10-20T18:08:03.451345] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-07. [2025-10-20T18:08:03.548075] [DEBUG] [VALIDATOR ] CRM: All expected trading days are present. [2025-10-20T18:08:03.548154] [DEBUG] [VALIDATOR ] CRM: Primary data validation passed. [2025-10-20T18:08:03.549036] [INFO ] [MAIN ] CRM: Fetched primary data is valid. [2025-10-20T18:08:03.549099] [DEBUG] [LOCAL ] CRM: Saving primary data. [2025-10-20T18:08:03.550473] [DEBUG] [LOCAL ] CRM: Trimmed 77946 records to 53770 (keeping 200 days of data) [2025-10-20T18:08:03.896857] [DEBUG] [LOCAL ] CRM: Primary data saved (53770 records). [2025-10-20T18:08:03.897253] [INFO ] [MAIN ] CRM: Full history saved. [2025-10-20T18:08:03.897355] [INFO ] [MAIN ] Update Progress: 537/2367 symbols completed. CRM took 49.8s. Estimated remaining: 3h 51m 42s. Rate: 7.9 symbols/min [2025-10-20T18:08:04.072633] [INFO ] [MAIN ] CSV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:04.086093] [DEBUG] [LOCAL ] CSV: Loaded 12794 existing records from primary storage. [2025-10-20T18:08:04.086301] [INFO ] [MAIN ] CSV: Loaded 12794 existing records. [2025-10-20T18:08:04.262887] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-05. [2025-10-20T18:08:04.268743] [WARN ] [MAIN ] CSV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:04.269672] [INFO ] [MAIN ] CSV: Wiping existing primary data before full refresh. [2025-10-20T18:08:04.269743] [INFO ] [LOCAL ] CSV: Starting deletion of primary data. [2025-10-20T18:08:04.270121] [DEBUG] [LOCAL ] CSV: Deleted primary data file. [2025-10-20T18:08:04.270201] [INFO ] [MAIN ] CSV: Starting full primary refresh. [2025-10-20T18:08:04.554006] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:08:04.647479] [INFO ] [REMOTE ] CSV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:05.051364] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-01. [2025-10-20T18:08:05.802095] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:08:05.821001] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:08:05.851331] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-02. [2025-10-20T18:08:06.651361] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-10. [2025-10-20T18:08:06.719037] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T18:08:07.451419] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-03. [2025-10-20T18:08:08.147290] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T18:08:08.152204] [INFO ] [REMOTE ] CSL: Full fetch complete (45617 records). [2025-10-20T18:08:08.152313] [DEBUG] [VALIDATOR ] CSL: Validating primary data. [2025-10-20T18:08:08.252057] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-08. [2025-10-20T18:08:08.542251] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T18:08:08.579357] [DEBUG] [VALIDATOR ] CSL: All expected trading days are present. [2025-10-20T18:08:08.579436] [DEBUG] [VALIDATOR ] CSL: Primary data validation passed. [2025-10-20T18:08:08.580058] [INFO ] [MAIN ] CSL: Fetched primary data is valid. [2025-10-20T18:08:08.580122] [DEBUG] [LOCAL ] CSL: Saving primary data. [2025-10-20T18:08:08.581063] [DEBUG] [LOCAL ] CSL: Trimmed 45617 records to 31310 (keeping 200 days of data) [2025-10-20T18:08:08.773621] [DEBUG] [LOCAL ] CSL: Primary data saved (31310 records). [2025-10-20T18:08:08.773803] [INFO ] [MAIN ] CSL: Full history saved. [2025-10-20T18:08:08.774316] [INFO ] [MAIN ] Update Progress: 538/2367 symbols completed. CSL took 37.9s. Estimated remaining: 3h 51m 25s. Rate: 7.9 symbols/min [2025-10-20T18:08:08.948218] [INFO ] [MAIN ] CSW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:08.948468] [DEBUG] [LOCAL ] CSW: No existing data file found in primary storage. [2025-10-20T18:08:08.948604] [WARN ] [MAIN ] CSW: No existing data found, performing full refresh instead. [2025-10-20T18:08:08.948751] [INFO ] [MAIN ] CSW: Wiping existing primary data before full refresh. [2025-10-20T18:08:08.948792] [INFO ] [LOCAL ] CSW: Starting deletion of primary data. [2025-10-20T18:08:08.948831] [DEBUG] [LOCAL ] CSW: No primary data file found to delete. [2025-10-20T18:08:08.948864] [INFO ] [MAIN ] CSW: Starting full primary refresh. [2025-10-20T18:08:09.056772] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-04. [2025-10-20T18:08:09.186205] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:08:09.401379] [INFO ] [REMOTE ] CSW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:09.851368] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-09. [2025-10-20T18:08:10.154601] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T18:08:10.651372] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-05. [2025-10-20T18:08:10.812512] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T18:08:11.451394] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-01. [2025-10-20T18:08:11.487774] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T18:08:11.660046] [DEBUG] [REMOTE ] CSW: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:08:11.660361] [WARN ] [REMOTE ] CSW: No data returned for month 2025-01 [2025-10-20T18:08:11.660465] [FATAL] [REMOTE ] CSW: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:08:11.660567] [FATAL] [MAIN ] CSW: Failed to fetch primary data. [2025-10-20T18:08:11.660678] [WARN ] [MAIN ] CSW: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:08:11.660775] [INFO ] [LOCAL ] CSW: Starting deletion of primary data. [2025-10-20T18:08:11.660858] [DEBUG] [LOCAL ] CSW: No primary data file found to delete. [2025-10-20T18:08:11.660897] [INFO ] [LOCAL ] CSW: Starting deletion of derivative data. [2025-10-20T18:08:11.660938] [DEBUG] [LOCAL ] CSW: No derivative data file found to delete. [2025-10-20T18:08:11.660985] [INFO ] [MAIN ] Update Progress: 539/2367 symbols completed. CSW took 2.9s. Estimated remaining: 3h 51m 2s. Rate: 7.9 symbols/min [2025-10-20T18:08:11.843276] [INFO ] [MAIN ] CTBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:11.843487] [DEBUG] [LOCAL ] CTBB: No existing data file found in primary storage. [2025-10-20T18:08:11.843539] [WARN ] [MAIN ] CTBB: No existing data found, performing full refresh instead. [2025-10-20T18:08:11.843732] [INFO ] [MAIN ] CTBB: Wiping existing primary data before full refresh. [2025-10-20T18:08:11.843787] [INFO ] [LOCAL ] CTBB: Starting deletion of primary data. [2025-10-20T18:08:11.843830] [DEBUG] [LOCAL ] CTBB: No primary data file found to delete. [2025-10-20T18:08:11.843863] [INFO ] [MAIN ] CTBB: Starting full primary refresh. [2025-10-20T18:08:12.204804] [INFO ] [REMOTE ] CTBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:12.251369] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-01. [2025-10-20T18:08:13.021882] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:08:13.051378] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-10. [2025-10-20T18:08:13.851401] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-02. [2025-10-20T18:08:14.085266] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:08:14.088081] [INFO ] [REMOTE ] CSR: Full fetch complete (17973 records). [2025-10-20T18:08:14.088154] [DEBUG] [VALIDATOR ] CSR: Validating primary data. [2025-10-20T18:08:14.419812] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T18:08:14.482701] [DEBUG] [VALIDATOR ] CSR: All expected trading days are present. [2025-10-20T18:08:14.482789] [DEBUG] [VALIDATOR ] CSR: Primary data validation passed. [2025-10-20T18:08:14.483196] [INFO ] [MAIN ] CSR: Fetched primary data is valid. [2025-10-20T18:08:14.483237] [DEBUG] [LOCAL ] CSR: Saving primary data. [2025-10-20T18:08:14.483801] [DEBUG] [LOCAL ] CSR: Trimmed 17973 records to 13476 (keeping 200 days of data) [2025-10-20T18:08:14.563637] [DEBUG] [LOCAL ] CSR: Primary data saved (13476 records). [2025-10-20T18:08:14.563802] [INFO ] [MAIN ] CSR: Full history saved. [2025-10-20T18:08:14.563879] [INFO ] [MAIN ] Update Progress: 540/2367 symbols completed. CSR took 27.1s. Estimated remaining: 3h 50m 38s. Rate: 7.9 symbols/min [2025-10-20T18:08:14.656604] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-06. [2025-10-20T18:08:14.743444] [INFO ] [MAIN ] CTDD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:14.750128] [DEBUG] [LOCAL ] CTDD: Loaded 5501 existing records from primary storage. [2025-10-20T18:08:14.750335] [INFO ] [MAIN ] CTDD: Loaded 5501 existing records. [2025-10-20T18:08:14.930988] [WARN ] [MAIN ] CTDD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:14.931205] [INFO ] [MAIN ] CTDD: Wiping existing primary data before full refresh. [2025-10-20T18:08:14.931251] [INFO ] [LOCAL ] CTDD: Starting deletion of primary data. [2025-10-20T18:08:14.931419] [DEBUG] [LOCAL ] CTDD: Deleted primary data file. [2025-10-20T18:08:14.931474] [INFO ] [MAIN ] CTDD: Starting full primary refresh. [2025-10-20T18:08:15.375272] [INFO ] [REMOTE ] CTDD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:15.451419] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-01. [2025-10-20T18:08:16.212630] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T18:08:16.251399] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-06. [2025-10-20T18:08:16.566078] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T18:08:17.051437] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-02. [2025-10-20T18:08:17.412657] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:08:17.451967] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T18:08:17.851402] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-07. [2025-10-20T18:08:18.497898] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T18:08:18.651383] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-08. [2025-10-20T18:08:19.451427] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-07. [2025-10-20T18:08:19.540002] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T18:08:20.251367] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-09. [2025-10-20T18:08:20.946515] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T18:08:21.051359] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-08. [2025-10-20T18:08:21.064075] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T18:08:21.851416] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-03. [2025-10-20T18:08:22.289181] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T18:08:22.493587] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:08:22.651418] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-04. [2025-10-20T18:08:23.373115] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T18:08:23.451416] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-09. [2025-10-20T18:08:24.251435] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-10. [2025-10-20T18:08:25.044417] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:08:25.053788] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-03. [2025-10-20T18:08:25.237550] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T18:08:25.239565] [INFO ] [REMOTE ] CSV: Full fetch complete (19121 records). [2025-10-20T18:08:25.239690] [DEBUG] [VALIDATOR ] CSV: Validating primary data. [2025-10-20T18:08:25.596764] [DEBUG] [VALIDATOR ] CSV: All expected trading days are present. [2025-10-20T18:08:25.596844] [DEBUG] [VALIDATOR ] CSV: Primary data validation passed. [2025-10-20T18:08:25.597232] [INFO ] [MAIN ] CSV: Fetched primary data is valid. [2025-10-20T18:08:25.597294] [DEBUG] [LOCAL ] CSV: Saving primary data. [2025-10-20T18:08:25.597881] [DEBUG] [LOCAL ] CSV: Trimmed 19121 records to 12576 (keeping 200 days of data) [2025-10-20T18:08:25.672491] [DEBUG] [LOCAL ] CSV: Primary data saved (12576 records). [2025-10-20T18:08:25.672670] [INFO ] [MAIN ] CSV: Full history saved. [2025-10-20T18:08:25.672776] [INFO ] [MAIN ] Update Progress: 541/2367 symbols completed. CSV took 21.8s. Estimated remaining: 3h 50m 43s. Rate: 7.9 symbols/min [2025-10-20T18:08:25.846380] [INFO ] [MAIN ] CTEV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:25.846544] [DEBUG] [LOCAL ] CTEV: No existing data file found in primary storage. [2025-10-20T18:08:25.846581] [WARN ] [MAIN ] CTEV: No existing data found, performing full refresh instead. [2025-10-20T18:08:25.846717] [INFO ] [MAIN ] CTEV: Wiping existing primary data before full refresh. [2025-10-20T18:08:25.846766] [INFO ] [LOCAL ] CTEV: Starting deletion of primary data. [2025-10-20T18:08:25.846807] [DEBUG] [LOCAL ] CTEV: No primary data file found to delete. [2025-10-20T18:08:25.846839] [INFO ] [MAIN ] CTEV: Starting full primary refresh. [2025-10-20T18:08:25.857913] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-05. [2025-10-20T18:08:26.138490] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T18:08:26.216313] [INFO ] [REMOTE ] CTEV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:26.651449] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-01. [2025-10-20T18:08:26.664502] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T18:08:26.865381] [DEBUG] [REMOTE ] CTEV: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:08:26.865752] [WARN ] [REMOTE ] CTEV: No data returned for month 2025-01 [2025-10-20T18:08:26.865853] [FATAL] [REMOTE ] CTEV: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:08:26.865903] [FATAL] [MAIN ] CTEV: Failed to fetch primary data. [2025-10-20T18:08:26.865936] [WARN ] [MAIN ] CTEV: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:08:26.865974] [INFO ] [LOCAL ] CTEV: Starting deletion of primary data. [2025-10-20T18:08:26.866048] [DEBUG] [LOCAL ] CTEV: No primary data file found to delete. [2025-10-20T18:08:26.866094] [INFO ] [LOCAL ] CTEV: Starting deletion of derivative data. [2025-10-20T18:08:26.866150] [DEBUG] [LOCAL ] CTEV: No derivative data file found to delete. [2025-10-20T18:08:26.866212] [INFO ] [MAIN ] Update Progress: 542/2367 symbols completed. CTEV took 1.2s. Estimated remaining: 3h 50m 14s. Rate: 7.9 symbols/min [2025-10-20T18:08:27.034708] [INFO ] [MAIN ] CTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:27.068299] [DEBUG] [LOCAL ] CTO: Loaded 28050 existing records from primary storage. [2025-10-20T18:08:27.068689] [INFO ] [MAIN ] CTO: Loaded 28050 existing records. [2025-10-20T18:08:27.284079] [WARN ] [MAIN ] CTO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:27.284407] [INFO ] [MAIN ] CTO: Wiping existing primary data before full refresh. [2025-10-20T18:08:27.284469] [INFO ] [LOCAL ] CTO: Starting deletion of primary data. [2025-10-20T18:08:27.284971] [DEBUG] [LOCAL ] CTO: Deleted primary data file. [2025-10-20T18:08:27.285029] [INFO ] [MAIN ] CTO: Starting full primary refresh. [2025-10-20T18:08:27.459543] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-04. [2025-10-20T18:08:27.754147] [INFO ] [REMOTE ] CTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:28.136260] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T18:08:28.251378] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-10. [2025-10-20T18:08:29.051381] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-01. [2025-10-20T18:08:29.851344] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-06. [2025-10-20T18:08:30.163807] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:08:30.170630] [INFO ] [REMOTE ] CSTM: Full fetch complete (69526 records). [2025-10-20T18:08:30.170787] [DEBUG] [VALIDATOR ] CSTM: Validating primary data. [2025-10-20T18:08:30.406496] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T18:08:30.487254] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T18:08:30.588950] [DEBUG] [VALIDATOR ] CSTM: All expected trading days are present. [2025-10-20T18:08:30.589033] [DEBUG] [VALIDATOR ] CSTM: Primary data validation passed. [2025-10-20T18:08:30.589826] [INFO ] [MAIN ] CSTM: Fetched primary data is valid. [2025-10-20T18:08:30.589887] [DEBUG] [LOCAL ] CSTM: Saving primary data. [2025-10-20T18:08:30.591337] [DEBUG] [LOCAL ] CSTM: Trimmed 69526 records to 47473 (keeping 200 days of data) [2025-10-20T18:08:30.671447] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-07. [2025-10-20T18:08:30.860034] [DEBUG] [LOCAL ] CSTM: Primary data saved (47473 records). [2025-10-20T18:08:30.860138] [INFO ] [MAIN ] CSTM: Full history saved. [2025-10-20T18:08:30.860215] [INFO ] [MAIN ] Update Progress: 543/2367 symbols completed. CSTM took 41.9s. Estimated remaining: 3h 49m 54s. Rate: 7.9 symbols/min [2025-10-20T18:08:31.023841] [INFO ] [MAIN ] CTOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:31.056991] [DEBUG] [LOCAL ] CTOS: Loaded 37827 existing records from primary storage. [2025-10-20T18:08:31.057295] [INFO ] [MAIN ] CTOS: Loaded 37827 existing records. [2025-10-20T18:08:31.177044] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T18:08:31.243407] [WARN ] [MAIN ] CTOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:31.243614] [INFO ] [MAIN ] CTOS: Wiping existing primary data before full refresh. [2025-10-20T18:08:31.243693] [INFO ] [LOCAL ] CTOS: Starting deletion of primary data. [2025-10-20T18:08:31.244026] [DEBUG] [LOCAL ] CTOS: Deleted primary data file. [2025-10-20T18:08:31.244065] [INFO ] [MAIN ] CTOS: Starting full primary refresh. [2025-10-20T18:08:31.456520] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-02. [2025-10-20T18:08:31.594137] [INFO ] [REMOTE ] CTOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:32.251368] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-01. [2025-10-20T18:08:32.690814] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:08:33.051376] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-05. [2025-10-20T18:08:33.294073] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:08:33.417910] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-10-20T18:08:33.851364] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-03. [2025-10-20T18:08:34.651377] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-06. [2025-10-20T18:08:35.335832] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T18:08:35.380070] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:08:35.451369] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-08. [2025-10-20T18:08:35.888614] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T18:08:36.251364] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-02. [2025-10-20T18:08:37.051618] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-09. [2025-10-20T18:08:37.388379] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:08:37.717485] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T18:08:37.851425] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-03. [2025-10-20T18:08:38.651406] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-07. [2025-10-20T18:08:38.889588] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T18:08:39.333965] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T18:08:39.451373] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-08. [2025-10-20T18:08:40.193242] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T18:08:40.251372] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-04. [2025-10-20T18:08:41.051392] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-09. [2025-10-20T18:08:41.538728] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T18:08:41.640703] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T18:08:41.851374] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-10. [2025-10-20T18:08:42.560085] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T18:08:42.561502] [INFO ] [REMOTE ] CTBB: Full fetch complete (10059 records). [2025-10-20T18:08:42.561579] [DEBUG] [VALIDATOR ] CTBB: Validating primary data. [2025-10-20T18:08:42.656567] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-04. [2025-10-20T18:08:42.929567] [DEBUG] [VALIDATOR ] CTBB: All expected trading days are present. [2025-10-20T18:08:42.929634] [DEBUG] [VALIDATOR ] CTBB: Primary data validation passed. [2025-10-20T18:08:42.929965] [INFO ] [MAIN ] CTBB: Fetched primary data is valid. [2025-10-20T18:08:42.930011] [DEBUG] [LOCAL ] CTBB: Saving primary data. [2025-10-20T18:08:42.930440] [DEBUG] [LOCAL ] CTBB: Trimmed 10059 records to 6573 (keeping 200 days of data) [2025-10-20T18:08:42.969691] [DEBUG] [LOCAL ] CTBB: Primary data saved (6573 records). [2025-10-20T18:08:42.969823] [INFO ] [MAIN ] CTBB: Full history saved. [2025-10-20T18:08:42.969895] [INFO ] [MAIN ] Update Progress: 544/2367 symbols completed. CTBB took 31.3s. Estimated remaining: 3h 50m 2s. Rate: 7.9 symbols/min [2025-10-20T18:08:43.137097] [INFO ] [MAIN ] CTRA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:43.186471] [DEBUG] [LOCAL ] CTRA: Loaded 54427 existing records from primary storage. [2025-10-20T18:08:43.186822] [INFO ] [MAIN ] CTRA: Loaded 54427 existing records. [2025-10-20T18:08:43.368080] [WARN ] [MAIN ] CTRA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:43.368300] [INFO ] [MAIN ] CTRA: Wiping existing primary data before full refresh. [2025-10-20T18:08:43.368359] [INFO ] [LOCAL ] CTRA: Starting deletion of primary data. [2025-10-20T18:08:43.368638] [DEBUG] [LOCAL ] CTRA: Deleted primary data file. [2025-10-20T18:08:43.368729] [INFO ] [MAIN ] CTRA: Starting full primary refresh. [2025-10-20T18:08:43.456545] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-10. [2025-10-20T18:08:43.753424] [INFO ] [REMOTE ] CTRA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:44.251423] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-05. [2025-10-20T18:08:44.666013] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T18:08:44.667439] [INFO ] [REMOTE ] CTDD: Full fetch complete (7921 records). [2025-10-20T18:08:44.667518] [DEBUG] [VALIDATOR ] CTDD: Validating primary data. [2025-10-20T18:08:45.087246] [DEBUG] [VALIDATOR ] CTDD: All expected trading days are present. [2025-10-20T18:08:45.071061] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-01. [2025-10-20T18:08:45.087678] [DEBUG] [VALIDATOR ] CTDD: Primary data validation passed. [2025-10-20T18:08:45.091383] [INFO ] [MAIN ] CTDD: Fetched primary data is valid. [2025-10-20T18:08:45.092841] [DEBUG] [LOCAL ] CTDD: Saving primary data. [2025-10-20T18:08:45.094107] [DEBUG] [LOCAL ] CTDD: Trimmed 7921 records to 5284 (keeping 200 days of data) [2025-10-20T18:08:45.113402] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-04 (2.46 seconds). [2025-10-20T18:08:45.145304] [DEBUG] [LOCAL ] CTDD: Primary data saved (5284 records). [2025-10-20T18:08:45.145895] [INFO ] [MAIN ] CTDD: Full history saved. [2025-10-20T18:08:45.146006] [INFO ] [MAIN ] Update Progress: 545/2367 symbols completed. CTDD took 30.6s. Estimated remaining: 3h 49m 36s. Rate: 7.9 symbols/min [2025-10-20T18:08:45.326930] [INFO ] [MAIN ] CTRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:45.327370] [DEBUG] [LOCAL ] CTRE: No existing data file found in primary storage. [2025-10-20T18:08:45.327495] [WARN ] [MAIN ] CTRE: No existing data found, performing full refresh instead. [2025-10-20T18:08:45.327770] [INFO ] [MAIN ] CTRE: Wiping existing primary data before full refresh. [2025-10-20T18:08:45.342809] [INFO ] [LOCAL ] CTRE: Starting deletion of primary data. [2025-10-20T18:08:45.343708] [DEBUG] [LOCAL ] CTRE: No primary data file found to delete. [2025-10-20T18:08:45.344226] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T18:08:45.344389] [INFO ] [MAIN ] CTRE: Starting full primary refresh. [2025-10-20T18:08:45.704129] [INFO ] [REMOTE ] CTRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:45.851402] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-05. [2025-10-20T18:08:46.651359] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-01. [2025-10-20T18:08:46.957192] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:08:47.082122] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T18:08:47.451378] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-06. [2025-10-20T18:08:47.937149] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T18:08:48.251376] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-02. [2025-10-20T18:08:48.345131] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T18:08:49.051386] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-02. [2025-10-20T18:08:49.834133] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:08:49.851330] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-03. [2025-10-20T18:08:50.626070] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:08:50.651331] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-03. [2025-10-20T18:08:51.217439] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T18:08:51.451397] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-06. [2025-10-20T18:08:52.017805] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T18:08:52.251414] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-04. [2025-10-20T18:08:52.795579] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T18:08:53.051444] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-07. [2025-10-20T18:08:53.654992] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T18:08:53.851381] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-07. [2025-10-20T18:08:54.376293] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T18:08:54.651427] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-08. [2025-10-20T18:08:55.451430] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-05. [2025-10-20T18:08:55.511629] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T18:08:55.572449] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T18:08:56.251426] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-08. [2025-10-20T18:08:57.051425] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-04. [2025-10-20T18:08:57.498564] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-10-20T18:08:57.817208] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:08:57.851374] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-09. [2025-10-20T18:08:58.651350] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-09. [2025-10-20T18:08:59.209519] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T18:08:59.376807] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T18:08:59.451351] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-10. [2025-10-20T18:08:59.968059] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T18:09:00.251378] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-05. [2025-10-20T18:09:00.968455] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T18:09:00.974833] [INFO ] [REMOTE ] CTOS: Full fetch complete (54095 records). [2025-10-20T18:09:00.974955] [DEBUG] [VALIDATOR ] CTOS: Validating primary data. [2025-10-20T18:09:01.056468] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-10. [2025-10-20T18:09:01.377930] [DEBUG] [VALIDATOR ] CTOS: All expected trading days are present. [2025-10-20T18:09:01.378021] [DEBUG] [VALIDATOR ] CTOS: Primary data validation passed. [2025-10-20T18:09:01.378761] [INFO ] [MAIN ] CTOS: Fetched primary data is valid. [2025-10-20T18:09:01.378829] [DEBUG] [LOCAL ] CTOS: Saving primary data. [2025-10-20T18:09:01.380147] [DEBUG] [LOCAL ] CTOS: Trimmed 54095 records to 37326 (keeping 200 days of data) [2025-10-20T18:09:01.594985] [DEBUG] [LOCAL ] CTOS: Primary data saved (37326 records). [2025-10-20T18:09:01.595136] [INFO ] [MAIN ] CTOS: Full history saved. [2025-10-20T18:09:01.595214] [INFO ] [MAIN ] Update Progress: 546/2367 symbols completed. CTOS took 30.7s. Estimated remaining: 3h 49m 58s. Rate: 7.9 symbols/min [2025-10-20T18:09:01.770606] [INFO ] [MAIN ] CTRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:01.809400] [DEBUG] [LOCAL ] CTRI: Loaded 40805 existing records from primary storage. [2025-10-20T18:09:01.809938] [INFO ] [MAIN ] CTRI: Loaded 40805 existing records. [2025-10-20T18:09:01.865186] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-06. [2025-10-20T18:09:02.015849] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T18:09:02.070671] [WARN ] [MAIN ] CTRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:02.071195] [INFO ] [MAIN ] CTRI: Wiping existing primary data before full refresh. [2025-10-20T18:09:02.071251] [INFO ] [LOCAL ] CTRI: Starting deletion of primary data. [2025-10-20T18:09:02.072044] [DEBUG] [LOCAL ] CTRI: Deleted primary data file. [2025-10-20T18:09:02.072097] [INFO ] [MAIN ] CTRI: Starting full primary refresh. [2025-10-20T18:09:02.337795] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:09:02.387776] [INFO ] [REMOTE ] CTO: Full fetch complete (41242 records). [2025-10-20T18:09:02.397086] [DEBUG] [VALIDATOR ] CTO: Validating primary data. [2025-10-20T18:09:02.403419] [FATAL] [VALIDATOR ] CTO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:09:02.440517] [FATAL] [MAIN ] CTO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:09:02.465846] [WARN ] [MAIN ] CTO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:09:02.488776] [INFO ] [LOCAL ] CTO: Starting deletion of primary data. [2025-10-20T18:09:02.488668] [INFO ] [REMOTE ] CTRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:02.489052] [DEBUG] [LOCAL ] CTO: No primary data file found to delete. [2025-10-20T18:09:02.489411] [INFO ] [LOCAL ] CTO: Starting deletion of derivative data. [2025-10-20T18:09:02.489633] [DEBUG] [LOCAL ] CTO: Deleted derivative data file. [2025-10-20T18:09:02.489805] [INFO ] [MAIN ] Update Progress: 547/2367 symbols completed. CTO took 35.6s. Estimated remaining: 3h 49m 28s. Rate: 7.9 symbols/min [2025-10-20T18:09:02.659085] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-06. [2025-10-20T18:09:02.662775] [INFO ] [MAIN ] CTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:02.684291] [DEBUG] [LOCAL ] CTS: Loaded 20047 existing records from primary storage. [2025-10-20T18:09:02.684564] [INFO ] [MAIN ] CTS: Loaded 20047 existing records. [2025-10-20T18:09:02.859893] [WARN ] [MAIN ] CTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:02.860152] [INFO ] [MAIN ] CTS: Wiping existing primary data before full refresh. [2025-10-20T18:09:02.860199] [INFO ] [LOCAL ] CTS: Starting deletion of primary data. [2025-10-20T18:09:02.860378] [DEBUG] [LOCAL ] CTS: Deleted primary data file. [2025-10-20T18:09:02.860434] [INFO ] [MAIN ] CTS: Starting full primary refresh. [2025-10-20T18:09:03.221507] [INFO ] [REMOTE ] CTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:03.451401] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-01. [2025-10-20T18:09:03.919071] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-06 (2.05 seconds). [2025-10-20T18:09:04.251340] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-01. [2025-10-20T18:09:04.432180] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:09:04.475356] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:09:05.051455] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-07. [2025-10-20T18:09:05.590987] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T18:09:05.851404] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-02. [2025-10-20T18:09:06.657164] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-07. [2025-10-20T18:09:06.686456] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:09:06.812538] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:09:07.451390] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-08. [2025-10-20T18:09:08.251366] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-03. [2025-10-20T18:09:08.471764] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:09:08.988116] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T18:09:09.051344] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-02. [2025-10-20T18:09:09.461724] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T18:09:09.851397] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-04. [2025-10-20T18:09:10.276361] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:09:10.651420] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-09. [2025-10-20T18:09:10.729029] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T18:09:11.451424] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-05. [2025-10-20T18:09:12.155628] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T18:09:12.251362] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-06. [2025-10-20T18:09:12.322111] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T18:09:12.911536] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T18:09:13.051380] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-07. [2025-10-20T18:09:13.851478] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-03. [2025-10-20T18:09:14.236260] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T18:09:14.651366] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-10. [2025-10-20T18:09:14.684841] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:09:15.451411] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-08. [2025-10-20T18:09:16.251373] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-04. [2025-10-20T18:09:16.644318] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T18:09:16.652394] [INFO ] [REMOTE ] CTRA: Full fetch complete (77430 records). [2025-10-20T18:09:16.652625] [DEBUG] [VALIDATOR ] CTRA: Validating primary data. [2025-10-20T18:09:16.653544] [FATAL] [VALIDATOR ] CTRA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:09:16.653671] [FATAL] [MAIN ] CTRA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:09:16.653735] [WARN ] [MAIN ] CTRA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:09:16.653785] [INFO ] [LOCAL ] CTRA: Starting deletion of primary data. [2025-10-20T18:09:16.653856] [DEBUG] [LOCAL ] CTRA: No primary data file found to delete. [2025-10-20T18:09:16.653895] [INFO ] [LOCAL ] CTRA: Starting deletion of derivative data. [2025-10-20T18:09:16.654102] [DEBUG] [LOCAL ] CTRA: Deleted derivative data file. [2025-10-20T18:09:16.654212] [INFO ] [MAIN ] Update Progress: 548/2367 symbols completed. CTRA took 33.7s. Estimated remaining: 3h 49m 43s. Rate: 7.9 symbols/min [2025-10-20T18:09:16.850476] [INFO ] [MAIN ] CTVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:16.909214] [DEBUG] [LOCAL ] CTVA: Loaded 54075 existing records from primary storage. [2025-10-20T18:09:16.909697] [INFO ] [MAIN ] CTVA: Loaded 54075 existing records. [2025-10-20T18:09:17.064456] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-08. [2025-10-20T18:09:17.112280] [WARN ] [MAIN ] CTVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:17.113830] [INFO ] [MAIN ] CTVA: Wiping existing primary data before full refresh. [2025-10-20T18:09:17.114002] [INFO ] [LOCAL ] CTVA: Starting deletion of primary data. [2025-10-20T18:09:17.114493] [DEBUG] [LOCAL ] CTVA: Deleted primary data file. [2025-10-20T18:09:17.114566] [INFO ] [MAIN ] CTVA: Starting full primary refresh. [2025-10-20T18:09:17.550161] [INFO ] [REMOTE ] CTVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:17.574174] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-10-20T18:09:17.585890] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T18:09:17.851466] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-05. [2025-10-20T18:09:17.902789] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T18:09:18.651360] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-09. [2025-10-20T18:09:19.335196] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:09:19.451385] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-09. [2025-10-20T18:09:19.720147] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:09:20.251383] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-10. [2025-10-20T18:09:21.051391] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-06. [2025-10-20T18:09:21.201900] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T18:09:21.236325] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T18:09:21.239957] [INFO ] [REMOTE ] CTS: Full fetch complete (27126 records). [2025-10-20T18:09:21.240072] [DEBUG] [VALIDATOR ] CTS: Validating primary data. [2025-10-20T18:09:21.240544] [FATAL] [VALIDATOR ] CTS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:09:21.240629] [FATAL] [MAIN ] CTS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:09:21.240711] [WARN ] [MAIN ] CTS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:09:21.240784] [INFO ] [LOCAL ] CTS: Starting deletion of primary data. [2025-10-20T18:09:21.240873] [DEBUG] [LOCAL ] CTS: No primary data file found to delete. [2025-10-20T18:09:21.240911] [INFO ] [LOCAL ] CTS: Starting deletion of derivative data. [2025-10-20T18:09:21.241041] [DEBUG] [LOCAL ] CTS: Deleted derivative data file. [2025-10-20T18:09:21.241111] [INFO ] [MAIN ] Update Progress: 549/2367 symbols completed. CTS took 18.8s. Estimated remaining: 3h 49m 25s. Rate: 7.9 symbols/min [2025-10-20T18:09:21.424729] [INFO ] [MAIN ] CUBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:21.424943] [DEBUG] [LOCAL ] CUBB: No existing data file found in primary storage. [2025-10-20T18:09:21.425000] [WARN ] [MAIN ] CUBB: No existing data found, performing full refresh instead. [2025-10-20T18:09:21.425117] [INFO ] [MAIN ] CUBB: Wiping existing primary data before full refresh. [2025-10-20T18:09:21.425159] [INFO ] [LOCAL ] CUBB: Starting deletion of primary data. [2025-10-20T18:09:21.425200] [DEBUG] [LOCAL ] CUBB: No primary data file found to delete. [2025-10-20T18:09:21.425232] [INFO ] [MAIN ] CUBB: Starting full primary refresh. [2025-10-20T18:09:21.830309] [INFO ] [REMOTE ] CUBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:21.851441] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-01. [2025-10-20T18:09:22.094604] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:09:22.287394] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-10-20T18:09:22.651409] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-07. [2025-10-20T18:09:23.451415] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-02. [2025-10-20T18:09:23.741799] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-10-20T18:09:24.084825] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:09:24.251407] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-03. [2025-10-20T18:09:24.604550] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T18:09:25.051410] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-01. [2025-10-20T18:09:25.851446] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-04. [2025-10-20T18:09:26.130074] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T18:09:26.651391] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-08. [2025-10-20T18:09:26.919538] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T18:09:27.451370] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-10. [2025-10-20T18:09:27.847896] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T18:09:28.251405] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-02. [2025-10-20T18:09:28.785914] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T18:09:28.793302] [INFO ] [REMOTE ] CTRE: Full fetch complete (71386 records). [2025-10-20T18:09:28.793460] [DEBUG] [VALIDATOR ] CTRE: Validating primary data. [2025-10-20T18:09:29.059136] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-05. [2025-10-20T18:09:29.178083] [DEBUG] [VALIDATOR ] CTRE: All expected trading days are present. [2025-10-20T18:09:29.178252] [DEBUG] [VALIDATOR ] CTRE: Primary data validation passed. [2025-10-20T18:09:29.179123] [INFO ] [MAIN ] CTRE: Fetched primary data is valid. [2025-10-20T18:09:29.179187] [DEBUG] [LOCAL ] CTRE: Saving primary data. [2025-10-20T18:09:29.180597] [DEBUG] [LOCAL ] CTRE: Trimmed 71386 records to 49507 (keeping 200 days of data) [2025-10-20T18:09:29.391464] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-10-20T18:09:29.485439] [DEBUG] [LOCAL ] CTRE: Primary data saved (49507 records). [2025-10-20T18:09:29.485603] [INFO ] [MAIN ] CTRE: Full history saved. [2025-10-20T18:09:29.485701] [INFO ] [MAIN ] Update Progress: 550/2367 symbols completed. CTRE took 44.3s. Estimated remaining: 3h 49m 20s. Rate: 7.9 symbols/min [2025-10-20T18:09:29.660032] [INFO ] [MAIN ] CUBE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:29.714017] [DEBUG] [LOCAL ] CUBE: Loaded 51666 existing records from primary storage. [2025-10-20T18:09:29.714362] [INFO ] [MAIN ] CUBE: Loaded 51666 existing records. [2025-10-20T18:09:29.856273] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-06. [2025-10-20T18:09:29.921592] [WARN ] [MAIN ] CUBE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:29.922178] [INFO ] [MAIN ] CUBE: Wiping existing primary data before full refresh. [2025-10-20T18:09:29.922320] [INFO ] [LOCAL ] CUBE: Starting deletion of primary data. [2025-10-20T18:09:29.922579] [DEBUG] [LOCAL ] CUBE: Deleted primary data file. [2025-10-20T18:09:29.922636] [INFO ] [MAIN ] CUBE: Starting full primary refresh. [2025-10-20T18:09:30.113013] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T18:09:30.213003] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-06 (0.36 seconds). [2025-10-20T18:09:30.329718] [INFO ] [REMOTE ] CUBE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:30.651393] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-01. [2025-10-20T18:09:31.451429] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-09. [2025-10-20T18:09:32.251408] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-03. [2025-10-20T18:09:32.363841] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T18:09:33.051401] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-07. [2025-10-20T18:09:33.110074] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:09:33.480312] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T18:09:33.566241] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T18:09:33.851380] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-04. [2025-10-20T18:09:34.651389] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-10. [2025-10-20T18:09:35.451442] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-08. [2025-10-20T18:09:35.684664] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:09:35.792746] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:09:36.235753] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T18:09:36.243069] [INFO ] [REMOTE ] CTRI: Full fetch complete (53957 records). [2025-10-20T18:09:36.243205] [DEBUG] [VALIDATOR ] CTRI: Validating primary data. [2025-10-20T18:09:36.256565] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-02. [2025-10-20T18:09:36.701298] [DEBUG] [VALIDATOR ] CTRI: All expected trading days are present. [2025-10-20T18:09:36.701433] [DEBUG] [VALIDATOR ] CTRI: Primary data validation passed. [2025-10-20T18:09:36.702890] [INFO ] [MAIN ] CTRI: Fetched primary data is valid. [2025-10-20T18:09:36.702965] [DEBUG] [LOCAL ] CTRI: Saving primary data. [2025-10-20T18:09:36.704583] [DEBUG] [LOCAL ] CTRI: Trimmed 53957 records to 40518 (keeping 200 days of data) [2025-10-20T18:09:36.961530] [DEBUG] [LOCAL ] CTRI: Primary data saved (40518 records). [2025-10-20T18:09:36.961777] [INFO ] [MAIN ] CTRI: Full history saved. [2025-10-20T18:09:36.961884] [INFO ] [MAIN ] Update Progress: 551/2367 symbols completed. CTRI took 35.4s. Estimated remaining: 3h 49m 12s. Rate: 7.9 symbols/min [2025-10-20T18:09:37.056565] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-09. [2025-10-20T18:09:37.166585] [INFO ] [MAIN ] CUBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:37.166881] [DEBUG] [LOCAL ] CUBI: No existing data file found in primary storage. [2025-10-20T18:09:37.166929] [WARN ] [MAIN ] CUBI: No existing data found, performing full refresh instead. [2025-10-20T18:09:37.167060] [INFO ] [MAIN ] CUBI: Wiping existing primary data before full refresh. [2025-10-20T18:09:37.167104] [INFO ] [LOCAL ] CUBI: Starting deletion of primary data. [2025-10-20T18:09:37.167146] [DEBUG] [LOCAL ] CUBI: No primary data file found to delete. [2025-10-20T18:09:37.167181] [INFO ] [MAIN ] CUBI: Starting full primary refresh. [2025-10-20T18:09:37.464452] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T18:09:37.582814] [INFO ] [REMOTE ] CUBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:37.851361] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-10. [2025-10-20T18:09:37.957858] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T18:09:38.289480] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T18:09:38.290225] [INFO ] [REMOTE ] CUBB: Full fetch complete (954 records). [2025-10-20T18:09:38.290291] [DEBUG] [VALIDATOR ] CUBB: Validating primary data. [2025-10-20T18:09:38.464626] [WARN ] [VALIDATOR ] CUBB: Validation failed - insufficient historical data. Have 182 unique days, need 200 days [2025-10-20T18:09:38.464815] [FATAL] [MAIN ] CUBB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:09:38.464882] [WARN ] [MAIN ] CUBB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:09:38.464952] [INFO ] [LOCAL ] CUBB: Starting deletion of primary data. [2025-10-20T18:09:38.465015] [DEBUG] [LOCAL ] CUBB: No primary data file found to delete. [2025-10-20T18:09:38.465048] [INFO ] [LOCAL ] CUBB: Starting deletion of derivative data. [2025-10-20T18:09:38.465086] [DEBUG] [LOCAL ] CUBB: No derivative data file found to delete. [2025-10-20T18:09:38.465137] [INFO ] [MAIN ] Update Progress: 552/2367 symbols completed. CUBB took 17.2s. Estimated remaining: 3h 48m 44s. Rate: 7.9 symbols/min [2025-10-20T18:09:38.637834] [INFO ] [MAIN ] CUK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:38.663711] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-03. [2025-10-20T18:09:38.687179] [DEBUG] [LOCAL ] CUK: Loaded 51814 existing records from primary storage. [2025-10-20T18:09:38.687525] [INFO ] [MAIN ] CUK: Loaded 51814 existing records. [2025-10-20T18:09:38.865892] [WARN ] [MAIN ] CUK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:38.866123] [INFO ] [MAIN ] CUK: Wiping existing primary data before full refresh. [2025-10-20T18:09:38.866170] [INFO ] [LOCAL ] CUK: Starting deletion of primary data. [2025-10-20T18:09:38.866575] [DEBUG] [LOCAL ] CUK: Deleted primary data file. [2025-10-20T18:09:38.866634] [INFO ] [MAIN ] CUK: Starting full primary refresh. [2025-10-20T18:09:39.223864] [INFO ] [REMOTE ] CUK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:39.451392] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-01. [2025-10-20T18:09:40.163589] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:09:40.251407] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-05. [2025-10-20T18:09:41.051441] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-01. [2025-10-20T18:09:41.347320] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T18:09:41.851396] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-02. [2025-10-20T18:09:42.086514] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T18:09:42.447969] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T18:09:42.651366] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-06. [2025-10-20T18:09:43.344168] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:09:43.451350] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-03. [2025-10-20T18:09:44.251404] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-02. [2025-10-20T18:09:44.385175] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T18:09:45.051354] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-07. [2025-10-20T18:09:45.294146] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:09:45.330571] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T18:09:45.851422] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-04. [2025-10-20T18:09:46.651375] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-03. [2025-10-20T18:09:46.876441] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T18:09:47.451387] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-04. [2025-10-20T18:09:47.813065] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T18:09:47.903348] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T18:09:48.251439] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-05. [2025-10-20T18:09:48.896067] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T18:09:49.051461] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-05. [2025-10-20T18:09:49.688353] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:09:49.851403] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-08. [2025-10-20T18:09:50.506056] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T18:09:50.651345] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-06. [2025-10-20T18:09:51.451585] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-04. [2025-10-20T18:09:51.747820] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T18:09:52.034040] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:09:52.251383] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-09. [2025-10-20T18:09:52.826036] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T18:09:53.051449] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-07. [2025-10-20T18:09:53.851422] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-06. [2025-10-20T18:09:54.021797] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T18:09:54.464678] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:09:54.651445] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-08. [2025-10-20T18:09:55.451392] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-05. [2025-10-20T18:09:55.616856] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:09:56.251345] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-07. [2025-10-20T18:09:56.342022] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T18:09:56.497896] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T18:09:57.051430] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-09. [2025-10-20T18:09:57.690022] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:09:57.851392] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-08. [2025-10-20T18:09:58.376753] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T18:09:58.651373] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-10. [2025-10-20T18:09:59.451443] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-06. [2025-10-20T18:09:59.587867] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:10:00.251388] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-10. [2025-10-20T18:10:00.557545] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:10:00.577420] [INFO ] [REMOTE ] CUBE: Full fetch complete (73307 records). [2025-10-20T18:10:00.579375] [DEBUG] [VALIDATOR ] CUBE: Validating primary data. [2025-10-20T18:10:00.580912] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T18:10:00.984594] [DEBUG] [VALIDATOR ] CUBE: All expected trading days are present. [2025-10-20T18:10:00.984743] [DEBUG] [VALIDATOR ] CUBE: Primary data validation passed. [2025-10-20T18:10:00.985688] [INFO ] [MAIN ] CUBE: Fetched primary data is valid. [2025-10-20T18:10:00.985755] [DEBUG] [LOCAL ] CUBE: Saving primary data. [2025-10-20T18:10:00.987194] [DEBUG] [LOCAL ] CUBE: Trimmed 73307 records to 50888 (keeping 200 days of data) [2025-10-20T18:10:01.054967] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-07. [2025-10-20T18:10:01.304128] [DEBUG] [LOCAL ] CUBE: Primary data saved (50888 records). [2025-10-20T18:10:01.304277] [INFO ] [MAIN ] CUBE: Full history saved. [2025-10-20T18:10:01.304355] [INFO ] [MAIN ] Update Progress: 553/2367 symbols completed. CUBE took 31.8s. Estimated remaining: 3h 49m 27s. Rate: 7.9 symbols/min [2025-10-20T18:10:01.472313] [INFO ] [MAIN ] CULP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:01.476807] [DEBUG] [LOCAL ] CULP: Loaded 2947 existing records from primary storage. [2025-10-20T18:10:01.477014] [INFO ] [MAIN ] CULP: Loaded 2947 existing records. [2025-10-20T18:10:01.674115] [WARN ] [MAIN ] CULP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:01.674363] [INFO ] [MAIN ] CULP: Wiping existing primary data before full refresh. [2025-10-20T18:10:01.674410] [INFO ] [LOCAL ] CULP: Starting deletion of primary data. [2025-10-20T18:10:01.674577] [DEBUG] [LOCAL ] CULP: Deleted primary data file. [2025-10-20T18:10:01.674633] [INFO ] [MAIN ] CULP: Starting full primary refresh. [2025-10-20T18:10:01.856560] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-09. [2025-10-20T18:10:02.218622] [INFO ] [REMOTE ] CULP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:02.265755] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T18:10:02.312231] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T18:10:02.321923] [INFO ] [REMOTE ] CTVA: Full fetch complete (77220 records). [2025-10-20T18:10:02.322144] [DEBUG] [VALIDATOR ] CTVA: Validating primary data. [2025-10-20T18:10:02.659268] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-01. [2025-10-20T18:10:02.950766] [DEBUG] [VALIDATOR ] CTVA: All expected trading days are present. [2025-10-20T18:10:02.950891] [DEBUG] [VALIDATOR ] CTVA: Primary data validation passed. [2025-10-20T18:10:02.952253] [INFO ] [MAIN ] CTVA: Fetched primary data is valid. [2025-10-20T18:10:02.952341] [DEBUG] [LOCAL ] CTVA: Saving primary data. [2025-10-20T18:10:02.954480] [DEBUG] [LOCAL ] CTVA: Trimmed 77220 records to 53302 (keeping 200 days of data) [2025-10-20T18:10:03.288355] [DEBUG] [LOCAL ] CTVA: Primary data saved (53302 records). [2025-10-20T18:10:03.288489] [INFO ] [MAIN ] CTVA: Full history saved. [2025-10-20T18:10:03.289006] [INFO ] [MAIN ] Update Progress: 554/2367 symbols completed. CTVA took 46.6s. Estimated remaining: 3h 49m 1s. Rate: 7.9 symbols/min [2025-10-20T18:10:03.457563] [INFO ] [MAIN ] CURB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:03.457628] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-08. [2025-10-20T18:10:03.490173] [DEBUG] [LOCAL ] CURB: Loaded 33392 existing records from primary storage. [2025-10-20T18:10:03.490445] [INFO ] [MAIN ] CURB: Loaded 33392 existing records. [2025-10-20T18:10:03.601782] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:10:03.688130] [WARN ] [MAIN ] CURB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:03.688508] [INFO ] [MAIN ] CURB: Wiping existing primary data before full refresh. [2025-10-20T18:10:03.688580] [INFO ] [LOCAL ] CURB: Starting deletion of primary data. [2025-10-20T18:10:03.689077] [DEBUG] [LOCAL ] CURB: Deleted primary data file. [2025-10-20T18:10:03.689145] [INFO ] [MAIN ] CURB: Starting full primary refresh. [2025-10-20T18:10:04.057283] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-10-20T18:10:04.076515] [INFO ] [REMOTE ] CURB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:04.251349] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-01. [2025-10-20T18:10:04.484971] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T18:10:05.051398] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-09. [2025-10-20T18:10:05.851383] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-10. [2025-10-20T18:10:05.984183] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T18:10:06.234400] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:10:06.651481] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-10. [2025-10-20T18:10:07.451450] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-02. [2025-10-20T18:10:08.032195] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-10 (2.18 seconds). [2025-10-20T18:10:08.042229] [INFO ] [REMOTE ] CUK: Full fetch complete (74358 records). [2025-10-20T18:10:08.042386] [DEBUG] [VALIDATOR ] CUK: Validating primary data. [2025-10-20T18:10:08.155133] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T18:10:08.180097] [INFO ] [REMOTE ] CUBI: Full fetch complete (39177 records). [2025-10-20T18:10:08.198465] [DEBUG] [VALIDATOR ] CUBI: Validating primary data. [2025-10-20T18:10:08.272576] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-02. [2025-10-20T18:10:08.750145] [DEBUG] [VALIDATOR ] CUK: All expected trading days are present. [2025-10-20T18:10:08.765217] [DEBUG] [VALIDATOR ] CUK: Primary data validation passed. [2025-10-20T18:10:08.787724] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T18:10:08.803550] [INFO ] [MAIN ] CUK: Fetched primary data is valid. [2025-10-20T18:10:08.837074] [DEBUG] [LOCAL ] CUK: Saving primary data. [2025-10-20T18:10:08.969308] [DEBUG] [LOCAL ] CUK: Trimmed 74358 records to 51072 (keeping 200 days of data) [2025-10-20T18:10:08.971683] [DEBUG] [VALIDATOR ] CUBI: All expected trading days are present. [2025-10-20T18:10:08.971083] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:10:08.977495] [DEBUG] [VALIDATOR ] CUBI: Primary data validation passed. [2025-10-20T18:10:08.979048] [INFO ] [MAIN ] CUBI: Fetched primary data is valid. [2025-10-20T18:10:09.026352] [DEBUG] [LOCAL ] CUBI: Saving primary data. [2025-10-20T18:10:09.061135] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-03. [2025-10-20T18:10:09.069049] [DEBUG] [LOCAL ] CUBI: Trimmed 39177 records to 27276 (keeping 200 days of data) [2025-10-20T18:10:09.313415] [DEBUG] [LOCAL ] CUBI: Primary data saved (27276 records). [2025-10-20T18:10:09.323210] [INFO ] [MAIN ] CUBI: Full history saved. [2025-10-20T18:10:09.351184] [INFO ] [MAIN ] Update Progress: 555/2367 symbols completed. CUBI took 32.4s. Estimated remaining: 3h 48m 49s. Rate: 7.9 symbols/min [2025-10-20T18:10:09.545157] [INFO ] [MAIN ] CURV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:09.589808] [DEBUG] [LOCAL ] CURV: Loaded 27541 existing records from primary storage. [2025-10-20T18:10:09.595583] [INFO ] [MAIN ] CURV: Loaded 27541 existing records. [2025-10-20T18:10:09.838760] [WARN ] [MAIN ] CURV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:09.840258] [INFO ] [MAIN ] CURV: Wiping existing primary data before full refresh. [2025-10-20T18:10:09.843237] [INFO ] [LOCAL ] CURV: Starting deletion of primary data. [2025-10-20T18:10:09.845838] [DEBUG] [LOCAL ] CURV: Deleted primary data file. [2025-10-20T18:10:09.846251] [INFO ] [MAIN ] CURV: Starting full primary refresh. [2025-10-20T18:10:09.851728] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-03. [2025-10-20T18:10:09.979012] [DEBUG] [LOCAL ] CUK: Primary data saved (51072 records). [2025-10-20T18:10:10.014070] [INFO ] [MAIN ] CUK: Full history saved. [2025-10-20T18:10:10.025047] [INFO ] [MAIN ] Update Progress: 556/2367 symbols completed. CUK took 31.6s. Estimated remaining: 3h 48m 19s. Rate: 7.9 symbols/min [2025-10-20T18:10:10.382476] [INFO ] [MAIN ] CUZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:10.425407] [INFO ] [REMOTE ] CURV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:10.430803] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T18:10:10.464892] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:10:10.510688] [DEBUG] [LOCAL ] CUZ: Loaded 50363 existing records from primary storage. [2025-10-20T18:10:10.510999] [INFO ] [MAIN ] CUZ: Loaded 50363 existing records. [2025-10-20T18:10:10.658902] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-04. [2025-10-20T18:10:10.690990] [WARN ] [MAIN ] CUZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:10.691338] [INFO ] [MAIN ] CUZ: Wiping existing primary data before full refresh. [2025-10-20T18:10:10.691385] [INFO ] [LOCAL ] CUZ: Starting deletion of primary data. [2025-10-20T18:10:10.691683] [DEBUG] [LOCAL ] CUZ: Deleted primary data file. [2025-10-20T18:10:10.691739] [INFO ] [MAIN ] CUZ: Starting full primary refresh. [2025-10-20T18:10:11.042760] [INFO ] [REMOTE ] CUZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:11.374141] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T18:10:11.451451] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-04. [2025-10-20T18:10:12.251406] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-05. [2025-10-20T18:10:12.720084] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T18:10:12.922966] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T18:10:13.051401] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-05. [2025-10-20T18:10:13.851423] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-06. [2025-10-20T18:10:14.235103] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:10:14.393281] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:10:14.651479] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-01. [2025-10-20T18:10:15.451442] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-01. [2025-10-20T18:10:16.210437] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T18:10:16.251398] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-02. [2025-10-20T18:10:16.688270] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:10:17.051363] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-06. [2025-10-20T18:10:17.557694] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T18:10:17.851390] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-03. [2025-10-20T18:10:18.516808] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T18:10:18.651357] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-07. [2025-10-20T18:10:19.377206] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T18:10:19.451351] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-07. [2025-10-20T18:10:19.935948] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T18:10:20.015994] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T18:10:20.251375] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-02. [2025-10-20T18:10:21.051403] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-04. [2025-10-20T18:10:21.227525] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:10:21.851403] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-08. [2025-10-20T18:10:22.227515] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-10-20T18:10:22.651410] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-09. [2025-10-20T18:10:22.743151] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:10:23.302612] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T18:10:23.451365] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-03. [2025-10-20T18:10:24.251369] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-05. [2025-10-20T18:10:24.650374] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T18:10:25.051363] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-04. [2025-10-20T18:10:25.851370] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-10. [2025-10-20T18:10:25.962805] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T18:10:26.076116] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T18:10:26.468165] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T18:10:26.469335] [INFO ] [REMOTE ] CULP: Full fetch complete (4535 records). [2025-10-20T18:10:26.469407] [DEBUG] [VALIDATOR ] CULP: Validating primary data. [2025-10-20T18:10:26.658015] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-06. [2025-10-20T18:10:26.837065] [DEBUG] [VALIDATOR ] CULP: All expected trading days are present. [2025-10-20T18:10:26.837149] [DEBUG] [VALIDATOR ] CULP: Primary data validation passed. [2025-10-20T18:10:26.837310] [INFO ] [MAIN ] CULP: Fetched primary data is valid. [2025-10-20T18:10:26.837351] [DEBUG] [LOCAL ] CULP: Saving primary data. [2025-10-20T18:10:26.837831] [DEBUG] [LOCAL ] CULP: Trimmed 4535 records to 2882 (keeping 200 days of data) [2025-10-20T18:10:26.856743] [DEBUG] [LOCAL ] CULP: Primary data saved (2882 records). [2025-10-20T18:10:26.856920] [INFO ] [MAIN ] CULP: Full history saved. [2025-10-20T18:10:26.857055] [INFO ] [MAIN ] Update Progress: 557/2367 symbols completed. CULP took 25.6s. Estimated remaining: 3h 48m 41s. Rate: 7.9 symbols/min [2025-10-20T18:10:27.451434] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-08. [2025-10-20T18:10:28.251369] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-05. [2025-10-20T18:10:28.329826] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:10:28.895879] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T18:10:29.051357] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-09. [2025-10-20T18:10:29.316285] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:10:29.851449] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-06. [2025-10-20T18:10:30.329797] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:10:30.651430] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-10. [2025-10-20T18:10:30.851020] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T18:10:31.451483] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-07. [2025-10-20T18:10:31.549301] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T18:10:31.555202] [INFO ] [REMOTE ] CURB: Full fetch complete (47218 records). [2025-10-20T18:10:31.555453] [DEBUG] [VALIDATOR ] CURB: Validating primary data. [2025-10-20T18:10:31.556381] [FATAL] [VALIDATOR ] CURB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:10:31.556490] [FATAL] [MAIN ] CURB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:10:31.556566] [WARN ] [MAIN ] CURB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:10:31.556699] [INFO ] [LOCAL ] CURB: Starting deletion of primary data. [2025-10-20T18:10:31.556788] [DEBUG] [LOCAL ] CURB: No primary data file found to delete. [2025-10-20T18:10:31.556828] [INFO ] [LOCAL ] CURB: Starting deletion of derivative data. [2025-10-20T18:10:31.557290] [DEBUG] [LOCAL ] CURB: Deleted derivative data file. [2025-10-20T18:10:31.557969] [INFO ] [MAIN ] Update Progress: 558/2367 symbols completed. CURB took 28.3s. Estimated remaining: 3h 48m 24s. Rate: 7.9 symbols/min [2025-10-20T18:10:32.251395] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-07. [2025-10-20T18:10:32.598981] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T18:10:33.051384] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-08. [2025-10-20T18:10:33.563266] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T18:10:33.851574] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-08. [2025-10-20T18:10:34.500025] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:10:34.651480] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-09. [2025-10-20T18:10:35.438623] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:10:35.451332] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-09. [2025-10-20T18:10:35.725131] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:10:36.251388] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-10. [2025-10-20T18:10:37.204424] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T18:10:37.215445] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-10. [2025-10-20T18:10:37.364368] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:10:37.369639] [INFO ] [REMOTE ] CURV: Full fetch complete (38962 records). [2025-10-20T18:10:37.369817] [DEBUG] [VALIDATOR ] CURV: Validating primary data. [2025-10-20T18:10:37.776756] [DEBUG] [VALIDATOR ] CURV: All expected trading days are present. [2025-10-20T18:10:37.776863] [DEBUG] [VALIDATOR ] CURV: Primary data validation passed. [2025-10-20T18:10:37.777431] [INFO ] [MAIN ] CURV: Fetched primary data is valid. [2025-10-20T18:10:37.777469] [DEBUG] [LOCAL ] CURV: Saving primary data. [2025-10-20T18:10:37.778465] [DEBUG] [LOCAL ] CURV: Trimmed 38962 records to 27263 (keeping 200 days of data) [2025-10-20T18:10:37.936405] [DEBUG] [LOCAL ] CURV: Primary data saved (27263 records). [2025-10-20T18:10:37.936569] [INFO ] [MAIN ] CURV: Full history saved. [2025-10-20T18:10:37.936688] [INFO ] [MAIN ] Update Progress: 559/2367 symbols completed. CURV took 28.6s. Estimated remaining: 3h 48m 13s. Rate: 7.9 symbols/min [2025-10-20T18:10:38.676680] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:10:38.683875] [INFO ] [REMOTE ] CUZ: Full fetch complete (70899 records). [2025-10-20T18:10:38.684029] [DEBUG] [VALIDATOR ] CUZ: Validating primary data. [2025-10-20T18:10:39.076086] [DEBUG] [VALIDATOR ] CUZ: All expected trading days are present. [2025-10-20T18:10:39.076205] [DEBUG] [VALIDATOR ] CUZ: Primary data validation passed. [2025-10-20T18:10:39.077089] [INFO ] [MAIN ] CUZ: Fetched primary data is valid. [2025-10-20T18:10:39.077157] [DEBUG] [LOCAL ] CUZ: Saving primary data. [2025-10-20T18:10:39.078628] [DEBUG] [LOCAL ] CUZ: Trimmed 70899 records to 49575 (keeping 200 days of data) [2025-10-20T18:10:39.383751] [DEBUG] [LOCAL ] CUZ: Primary data saved (49575 records). [2025-10-20T18:10:39.383898] [INFO ] [MAIN ] CUZ: Full history saved. [2025-10-20T18:10:39.383978] [INFO ] [MAIN ] Update Progress: 560/2367 symbols completed. CUZ took 29.4s. Estimated remaining: 3h 47m 45s. Rate: 7.9 symbols/min [2025-10-20T18:10:40.108312] [INFO ] [MAIN ] CVLG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:40.183464] [INFO ] [MAIN ] CVI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:40.230499] [INFO ] [MAIN ] CVE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:40.253168] [INFO ] [MAIN ] CVEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:40.266217] [DEBUG] [LOCAL ] CVLG: Loaded 15760 existing records from primary storage. [2025-10-20T18:10:40.266966] [INFO ] [MAIN ] CVLG: Loaded 15760 existing records. [2025-10-20T18:10:40.395901] [DEBUG] [LOCAL ] CVEO: Loaded 15568 existing records from primary storage. [2025-10-20T18:10:40.449583] [INFO ] [MAIN ] CVEO: Loaded 15568 existing records. [2025-10-20T18:10:40.491925] [WARN ] [MAIN ] CVLG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:40.505357] [INFO ] [MAIN ] CVLG: Wiping existing primary data before full refresh. [2025-10-20T18:10:40.540743] [INFO ] [LOCAL ] CVLG: Starting deletion of primary data. [2025-10-20T18:10:40.612015] [DEBUG] [LOCAL ] CVLG: Deleted primary data file. [2025-10-20T18:10:40.611917] [DEBUG] [LOCAL ] CVI: Loaded 47330 existing records from primary storage. [2025-10-20T18:10:40.647349] [INFO ] [MAIN ] CVLG: Starting full primary refresh. [2025-10-20T18:10:40.673194] [INFO ] [MAIN ] CVI: Loaded 47330 existing records. [2025-10-20T18:10:40.672894] [DEBUG] [LOCAL ] CVE: Loaded 54342 existing records from primary storage. [2025-10-20T18:10:40.898175] [INFO ] [MAIN ] CVE: Loaded 54342 existing records. [2025-10-20T18:10:40.994342] [WARN ] [MAIN ] CVEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:41.020831] [INFO ] [MAIN ] CVEO: Wiping existing primary data before full refresh. [2025-10-20T18:10:41.057218] [INFO ] [LOCAL ] CVEO: Starting deletion of primary data. [2025-10-20T18:10:41.124004] [DEBUG] [LOCAL ] CVEO: Deleted primary data file. [2025-10-20T18:10:41.206507] [INFO ] [MAIN ] CVEO: Starting full primary refresh. [2025-10-20T18:10:41.338263] [WARN ] [MAIN ] CVI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:41.459624] [INFO ] [MAIN ] CVI: Wiping existing primary data before full refresh. [2025-10-20T18:10:41.479803] [WARN ] [MAIN ] CVE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:41.528767] [INFO ] [LOCAL ] CVI: Starting deletion of primary data. [2025-10-20T18:10:41.555046] [INFO ] [MAIN ] CVE: Wiping existing primary data before full refresh. [2025-10-20T18:10:41.623941] [DEBUG] [LOCAL ] CVI: Deleted primary data file. [2025-10-20T18:10:41.602794] [INFO ] [LOCAL ] CVE: Starting deletion of primary data. [2025-10-20T18:10:41.671586] [INFO ] [MAIN ] CVI: Starting full primary refresh. [2025-10-20T18:10:41.778334] [DEBUG] [LOCAL ] CVE: Deleted primary data file. [2025-10-20T18:10:41.823840] [INFO ] [REMOTE ] CVLG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:41.829195] [INFO ] [MAIN ] CVE: Starting full primary refresh. [2025-10-20T18:10:41.848615] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-01. [2025-10-20T18:10:42.257685] [INFO ] [REMOTE ] CVEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:42.268677] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-01. [2025-10-20T18:10:42.795931] [INFO ] [REMOTE ] CVE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:42.796426] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-01. [2025-10-20T18:10:42.791725] [INFO ] [REMOTE ] CVI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:42.797599] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-01. [2025-10-20T18:10:43.141092] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T18:10:43.145064] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-02. [2025-10-20T18:10:43.211580] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:10:43.214015] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-02. [2025-10-20T18:10:43.895411] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:10:43.898028] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-03. [2025-10-20T18:10:44.074326] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T18:10:44.077943] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-03. [2025-10-20T18:10:44.223623] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:10:44.251343] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-02. [2025-10-20T18:10:44.339356] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:10:44.445636] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T18:10:44.702386] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:10:45.051420] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-04. [2025-10-20T18:10:45.334528] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T18:10:45.851394] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-03. [2025-10-20T18:10:45.949919] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T18:10:46.651387] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-02. [2025-10-20T18:10:47.365840] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T18:10:47.451331] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-04. [2025-10-20T18:10:48.080402] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T18:10:48.251365] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-04. [2025-10-20T18:10:48.946556] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T18:10:49.051366] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-05. [2025-10-20T18:10:49.182629] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T18:10:49.771755] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T18:10:49.851347] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-06. [2025-10-20T18:10:50.651407] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-05. [2025-10-20T18:10:50.900635] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:10:51.451366] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-07. [2025-10-20T18:10:51.684276] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T18:10:52.184565] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:10:52.251461] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-05. [2025-10-20T18:10:53.051435] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-03. [2025-10-20T18:10:53.848337] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T18:10:53.857378] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-08. [2025-10-20T18:10:54.553792] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:10:54.651379] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-06. [2025-10-20T18:10:54.756399] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T18:10:55.376416] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:10:55.451438] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-09. [2025-10-20T18:10:56.195974] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T18:10:56.251367] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-04. [2025-10-20T18:10:57.051407] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-07. [2025-10-20T18:10:57.851397] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-10. [2025-10-20T18:10:58.002667] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T18:10:58.148090] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:10:58.651405] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-05. [2025-10-20T18:10:58.674395] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T18:10:58.676777] [INFO ] [REMOTE ] CVLG: Full fetch complete (21409 records). [2025-10-20T18:10:58.676872] [DEBUG] [VALIDATOR ] CVLG: Validating primary data. [2025-10-20T18:10:59.066437] [DEBUG] [VALIDATOR ] CVLG: All expected trading days are present. [2025-10-20T18:10:59.066516] [DEBUG] [VALIDATOR ] CVLG: Primary data validation passed. [2025-10-20T18:10:59.066953] [INFO ] [MAIN ] CVLG: Fetched primary data is valid. [2025-10-20T18:10:59.067014] [DEBUG] [LOCAL ] CVLG: Saving primary data. [2025-10-20T18:10:59.067567] [DEBUG] [LOCAL ] CVLG: Trimmed 21409 records to 15478 (keeping 200 days of data) [2025-10-20T18:10:59.157616] [DEBUG] [LOCAL ] CVLG: Primary data saved (15478 records). [2025-10-20T18:10:59.157769] [INFO ] [MAIN ] CVLG: Full history saved. [2025-10-20T18:10:59.157848] [INFO ] [MAIN ] Update Progress: 561/2367 symbols completed. CVLG took 19.8s. Estimated remaining: 3h 48m 17s. Rate: 7.9 symbols/min [2025-10-20T18:10:59.326607] [INFO ] [MAIN ] CVNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:59.375936] [DEBUG] [LOCAL ] CVNA: Loaded 53628 existing records from primary storage. [2025-10-20T18:10:59.376239] [INFO ] [MAIN ] CVNA: Loaded 53628 existing records. [2025-10-20T18:10:59.456683] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-08. [2025-10-20T18:10:59.567168] [WARN ] [MAIN ] CVNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:59.567576] [INFO ] [MAIN ] CVNA: Wiping existing primary data before full refresh. [2025-10-20T18:10:59.567665] [INFO ] [LOCAL ] CVNA: Starting deletion of primary data. [2025-10-20T18:10:59.568046] [DEBUG] [LOCAL ] CVNA: Deleted primary data file. [2025-10-20T18:10:59.568106] [INFO ] [MAIN ] CVNA: Starting full primary refresh. [2025-10-20T18:10:59.954414] [INFO ] [REMOTE ] CVNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:00.251419] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-06. [2025-10-20T18:11:00.365880] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T18:11:00.546675] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T18:11:01.051392] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-01. [2025-10-20T18:11:01.851454] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-09. [2025-10-20T18:11:02.213116] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-10-20T18:11:02.651432] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-06. [2025-10-20T18:11:02.759878] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T18:11:02.951899] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T18:11:03.451370] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-07. [2025-10-20T18:11:04.251444] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-10. [2025-10-20T18:11:04.311156] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T18:11:05.051380] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-02. [2025-10-20T18:11:05.113152] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T18:11:05.156778] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T18:11:05.159791] [INFO ] [REMOTE ] CVEO: Full fetch complete (20853 records). [2025-10-20T18:11:05.159982] [DEBUG] [VALIDATOR ] CVEO: Validating primary data. [2025-10-20T18:11:05.562953] [DEBUG] [VALIDATOR ] CVEO: All expected trading days are present. [2025-10-20T18:11:05.563031] [DEBUG] [VALIDATOR ] CVEO: Primary data validation passed. [2025-10-20T18:11:05.563494] [INFO ] [MAIN ] CVEO: Fetched primary data is valid. [2025-10-20T18:11:05.563533] [DEBUG] [LOCAL ] CVEO: Saving primary data. [2025-10-20T18:11:05.564171] [DEBUG] [LOCAL ] CVEO: Trimmed 20853 records to 15328 (keeping 200 days of data) [2025-10-20T18:11:05.652587] [DEBUG] [LOCAL ] CVEO: Primary data saved (15328 records). [2025-10-20T18:11:05.652792] [INFO ] [MAIN ] CVEO: Full history saved. [2025-10-20T18:11:05.652884] [INFO ] [MAIN ] Update Progress: 562/2367 symbols completed. CVEO took 26.3s. Estimated remaining: 3h 48m 6s. Rate: 7.9 symbols/min [2025-10-20T18:11:05.822519] [INFO ] [MAIN ] CVS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:05.851484] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-08. [2025-10-20T18:11:05.873222] [DEBUG] [LOCAL ] CVS: Loaded 54474 existing records from primary storage. [2025-10-20T18:11:05.873576] [INFO ] [MAIN ] CVS: Loaded 54474 existing records. [2025-10-20T18:11:06.053750] [WARN ] [MAIN ] CVS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:06.053967] [INFO ] [MAIN ] CVS: Wiping existing primary data before full refresh. [2025-10-20T18:11:06.054013] [INFO ] [LOCAL ] CVS: Starting deletion of primary data. [2025-10-20T18:11:06.054309] [DEBUG] [LOCAL ] CVS: Deleted primary data file. [2025-10-20T18:11:06.054346] [INFO ] [MAIN ] CVS: Starting full primary refresh. [2025-10-20T18:11:06.406949] [INFO ] [REMOTE ] CVS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:06.651380] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-01. [2025-10-20T18:11:06.947518] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T18:11:07.429487] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T18:11:07.451368] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-09. [2025-10-20T18:11:08.251446] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-03. [2025-10-20T18:11:08.812747] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-01 (2.16 seconds). [2025-10-20T18:11:09.051397] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-07. [2025-10-20T18:11:09.084078] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:11:09.851454] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-10. [2025-10-20T18:11:10.277463] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T18:11:10.651413] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-04. [2025-10-20T18:11:10.695915] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:11:11.451357] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-02. [2025-10-20T18:11:11.681403] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:11:11.688457] [INFO ] [REMOTE ] CVI: Full fetch complete (67205 records). [2025-10-20T18:11:11.688569] [DEBUG] [VALIDATOR ] CVI: Validating primary data. [2025-10-20T18:11:12.069113] [DEBUG] [VALIDATOR ] CVI: All expected trading days are present. [2025-10-20T18:11:12.069193] [DEBUG] [VALIDATOR ] CVI: Primary data validation passed. [2025-10-20T18:11:12.069967] [INFO ] [MAIN ] CVI: Fetched primary data is valid. [2025-10-20T18:11:12.070027] [DEBUG] [LOCAL ] CVI: Saving primary data. [2025-10-20T18:11:12.071310] [DEBUG] [LOCAL ] CVI: Trimmed 67205 records to 46745 (keeping 200 days of data) [2025-10-20T18:11:12.267806] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-08. [2025-10-20T18:11:12.342789] [DEBUG] [LOCAL ] CVI: Primary data saved (46745 records). [2025-10-20T18:11:12.342950] [INFO ] [MAIN ] CVI: Full history saved. [2025-10-20T18:11:12.343052] [INFO ] [MAIN ] Update Progress: 563/2367 symbols completed. CVI took 32.9s. Estimated remaining: 3h 47m 56s. Rate: 7.9 symbols/min [2025-10-20T18:11:12.509737] [INFO ] [MAIN ] CVX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:12.509920] [DEBUG] [LOCAL ] CVX: No existing data file found in primary storage. [2025-10-20T18:11:12.509958] [WARN ] [MAIN ] CVX: No existing data found, performing full refresh instead. [2025-10-20T18:11:12.510073] [INFO ] [MAIN ] CVX: Wiping existing primary data before full refresh. [2025-10-20T18:11:12.510119] [INFO ] [LOCAL ] CVX: Starting deletion of primary data. [2025-10-20T18:11:12.510161] [DEBUG] [LOCAL ] CVX: No primary data file found to delete. [2025-10-20T18:11:12.510192] [INFO ] [MAIN ] CVX: Starting full primary refresh. [2025-10-20T18:11:12.867961] [INFO ] [REMOTE ] CVX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:12.965236] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-04 (2.31 seconds). [2025-10-20T18:11:13.051373] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-05. [2025-10-20T18:11:13.343118] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T18:11:13.851420] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-03. [2025-10-20T18:11:14.107419] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T18:11:14.651382] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-01. [2025-10-20T18:11:15.062142] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-10-20T18:11:15.451409] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-09. [2025-10-20T18:11:15.713179] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T18:11:16.251432] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-06. [2025-10-20T18:11:16.928972] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T18:11:16.972006] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-01 (2.32 seconds). [2025-10-20T18:11:17.051576] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-10. [2025-10-20T18:11:17.851434] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-04. [2025-10-20T18:11:18.429209] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-06 (2.18 seconds). [2025-10-20T18:11:18.651380] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-02. [2025-10-20T18:11:18.960787] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:11:18.968523] [INFO ] [REMOTE ] CVE: Full fetch complete (77716 records). [2025-10-20T18:11:18.968675] [DEBUG] [VALIDATOR ] CVE: Validating primary data. [2025-10-20T18:11:19.354554] [DEBUG] [VALIDATOR ] CVE: All expected trading days are present. [2025-10-20T18:11:19.354638] [DEBUG] [VALIDATOR ] CVE: Primary data validation passed. [2025-10-20T18:11:19.355511] [INFO ] [MAIN ] CVE: Fetched primary data is valid. [2025-10-20T18:11:19.355554] [DEBUG] [LOCAL ] CVE: Saving primary data. [2025-10-20T18:11:19.357005] [DEBUG] [LOCAL ] CVE: Trimmed 77716 records to 53567 (keeping 200 days of data) [2025-10-20T18:11:19.451527] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-07. [2025-10-20T18:11:19.686481] [DEBUG] [LOCAL ] CVE: Primary data saved (53567 records). [2025-10-20T18:11:19.686625] [INFO ] [MAIN ] CVE: Full history saved. [2025-10-20T18:11:19.686737] [INFO ] [MAIN ] Update Progress: 564/2367 symbols completed. CVE took 40.3s. Estimated remaining: 3h 47m 47s. Rate: 7.9 symbols/min [2025-10-20T18:11:19.859062] [INFO ] [MAIN ] CW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:19.893989] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T18:11:19.922625] [DEBUG] [LOCAL ] CW: Loaded 22372 existing records from primary storage. [2025-10-20T18:11:19.922982] [INFO ] [MAIN ] CW: Loaded 22372 existing records. [2025-10-20T18:11:20.098801] [WARN ] [MAIN ] CW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:20.098983] [INFO ] [MAIN ] CW: Wiping existing primary data before full refresh. [2025-10-20T18:11:20.099047] [INFO ] [LOCAL ] CW: Starting deletion of primary data. [2025-10-20T18:11:20.099420] [DEBUG] [LOCAL ] CW: Deleted primary data file. [2025-10-20T18:11:20.099476] [INFO ] [MAIN ] CW: Starting full primary refresh. [2025-10-20T18:11:20.253750] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-05. [2025-10-20T18:11:20.460090] [INFO ] [REMOTE ] CW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:20.781717] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-10-20T18:11:21.051439] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-01. [2025-10-20T18:11:21.557498] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T18:11:21.851396] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-08. [2025-10-20T18:11:22.226134] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:11:22.505776] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-10-20T18:11:22.651380] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-06. [2025-10-20T18:11:23.451356] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-02. [2025-10-20T18:11:23.818693] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T18:11:24.012831] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T18:11:24.251382] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-09. [2025-10-20T18:11:24.526454] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T18:11:25.051371] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-03. [2025-10-20T18:11:25.680573] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T18:11:25.851372] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-10. [2025-10-20T18:11:26.651445] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-07. [2025-10-20T18:11:27.216843] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-03 (2.17 seconds). [2025-10-20T18:11:27.451394] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-04. [2025-10-20T18:11:27.950285] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T18:11:27.958142] [INFO ] [REMOTE ] CVNA: Full fetch complete (76747 records). [2025-10-20T18:11:27.958290] [DEBUG] [VALIDATOR ] CVNA: Validating primary data. [2025-10-20T18:11:28.258257] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-03. [2025-10-20T18:11:28.379635] [DEBUG] [VALIDATOR ] CVNA: All expected trading days are present. [2025-10-20T18:11:28.379777] [DEBUG] [VALIDATOR ] CVNA: Primary data validation passed. [2025-10-20T18:11:28.380776] [INFO ] [MAIN ] CVNA: Fetched primary data is valid. [2025-10-20T18:11:28.380857] [DEBUG] [LOCAL ] CVNA: Saving primary data. [2025-10-20T18:11:28.382395] [DEBUG] [LOCAL ] CVNA: Trimmed 76747 records to 52842 (keeping 200 days of data) [2025-10-20T18:11:28.725887] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T18:11:28.756085] [DEBUG] [LOCAL ] CVNA: Primary data saved (52842 records). [2025-10-20T18:11:28.756619] [INFO ] [MAIN ] CVNA: Full history saved. [2025-10-20T18:11:28.756757] [INFO ] [MAIN ] Update Progress: 565/2367 symbols completed. CVNA took 29.6s. Estimated remaining: 3h 47m 44s. Rate: 7.9 symbols/min [2025-10-20T18:11:28.925630] [INFO ] [MAIN ] CWAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:28.974615] [DEBUG] [LOCAL ] CWAN: Loaded 53387 existing records from primary storage. [2025-10-20T18:11:28.974955] [INFO ] [MAIN ] CWAN: Loaded 53387 existing records. [2025-10-20T18:11:29.056119] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-08. [2025-10-20T18:11:29.159143] [WARN ] [MAIN ] CWAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:29.159474] [INFO ] [MAIN ] CWAN: Wiping existing primary data before full refresh. [2025-10-20T18:11:29.159607] [INFO ] [LOCAL ] CWAN: Starting deletion of primary data. [2025-10-20T18:11:29.159847] [DEBUG] [LOCAL ] CWAN: Deleted primary data file. [2025-10-20T18:11:29.159898] [INFO ] [MAIN ] CWAN: Starting full primary refresh. [2025-10-20T18:11:29.428073] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T18:11:29.559105] [INFO ] [REMOTE ] CWAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:29.591324] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T18:11:29.851391] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-04. [2025-10-20T18:11:30.651372] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-01. [2025-10-20T18:11:30.682493] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T18:11:31.257813] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-10-20T18:11:31.451406] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-09. [2025-10-20T18:11:32.251450] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-05. [2025-10-20T18:11:32.429095] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T18:11:33.051427] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-02. [2025-10-20T18:11:33.261793] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T18:11:33.430230] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T18:11:33.851400] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-06. [2025-10-20T18:11:34.255285] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:11:34.651427] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-03. [2025-10-20T18:11:35.214338] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T18:11:35.451406] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-07. [2025-10-20T18:11:36.251376] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-05. [2025-10-20T18:11:36.295573] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T18:11:36.495459] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T18:11:37.051374] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-08. [2025-10-20T18:11:37.851425] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-04. [2025-10-20T18:11:38.043458] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:11:38.651430] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-09. [2025-10-20T18:11:38.757955] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-05 (2.51 seconds). [2025-10-20T18:11:39.152580] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T18:11:39.451435] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-10. [2025-10-20T18:11:39.675926] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T18:11:40.251440] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-05. [2025-10-20T18:11:40.291097] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:11:40.299138] [INFO ] [REMOTE ] CVS: Full fetch complete (77874 records). [2025-10-20T18:11:40.299263] [DEBUG] [VALIDATOR ] CVS: Validating primary data. [2025-10-20T18:11:40.714621] [DEBUG] [VALIDATOR ] CVS: All expected trading days are present. [2025-10-20T18:11:40.714739] [DEBUG] [VALIDATOR ] CVS: Primary data validation passed. [2025-10-20T18:11:40.715678] [INFO ] [MAIN ] CVS: Fetched primary data is valid. [2025-10-20T18:11:40.715752] [DEBUG] [LOCAL ] CVS: Saving primary data. [2025-10-20T18:11:40.717270] [DEBUG] [LOCAL ] CVS: Trimmed 77874 records to 53694 (keeping 200 days of data) [2025-10-20T18:11:41.045743] [DEBUG] [LOCAL ] CVS: Primary data saved (53694 records). [2025-10-20T18:11:41.045884] [INFO ] [MAIN ] CVS: Full history saved. [2025-10-20T18:11:41.045958] [INFO ] [MAIN ] Update Progress: 566/2367 symbols completed. CVS took 35.4s. Estimated remaining: 3h 47m 52s. Rate: 7.9 symbols/min [2025-10-20T18:11:41.057289] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-10. [2025-10-20T18:11:41.222072] [INFO ] [MAIN ] CWEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:41.264200] [DEBUG] [LOCAL ] CWEN: Loaded 45984 existing records from primary storage. [2025-10-20T18:11:41.264545] [INFO ] [MAIN ] CWEN: Loaded 45984 existing records. [2025-10-20T18:11:41.442746] [WARN ] [MAIN ] CWEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:41.442944] [INFO ] [MAIN ] CWEN: Wiping existing primary data before full refresh. [2025-10-20T18:11:41.442990] [INFO ] [LOCAL ] CWEN: Starting deletion of primary data. [2025-10-20T18:11:41.443202] [DEBUG] [LOCAL ] CWEN: Deleted primary data file. [2025-10-20T18:11:41.443259] [INFO ] [MAIN ] CWEN: Starting full primary refresh. [2025-10-20T18:11:41.808724] [INFO ] [REMOTE ] CWEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:41.851339] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-01. [2025-10-20T18:11:42.004362] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T18:11:42.352674] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T18:11:42.356184] [INFO ] [REMOTE ] CW: Full fetch complete (33141 records). [2025-10-20T18:11:42.356299] [DEBUG] [VALIDATOR ] CW: Validating primary data. [2025-10-20T18:11:42.658733] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-06. [2025-10-20T18:11:42.816078] [DEBUG] [VALIDATOR ] CW: All expected trading days are present. [2025-10-20T18:11:42.816160] [DEBUG] [VALIDATOR ] CW: Primary data validation passed. [2025-10-20T18:11:42.816828] [INFO ] [MAIN ] CW: Fetched primary data is valid. [2025-10-20T18:11:42.816908] [DEBUG] [LOCAL ] CW: Saving primary data. [2025-10-20T18:11:42.818017] [DEBUG] [LOCAL ] CW: Trimmed 33141 records to 22034 (keeping 200 days of data) [2025-10-20T18:11:42.969568] [DEBUG] [LOCAL ] CW: Primary data saved (22034 records). [2025-10-20T18:11:42.969734] [INFO ] [MAIN ] CW: Full history saved. [2025-10-20T18:11:42.969827] [INFO ] [MAIN ] Update Progress: 567/2367 symbols completed. CW took 23.3s. Estimated remaining: 3h 47m 26s. Rate: 7.9 symbols/min [2025-10-20T18:11:43.144638] [INFO ] [MAIN ] CWH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:43.192962] [DEBUG] [LOCAL ] CWH: Loaded 52055 existing records from primary storage. [2025-10-20T18:11:43.193270] [INFO ] [MAIN ] CWH: Loaded 52055 existing records. [2025-10-20T18:11:43.371494] [WARN ] [MAIN ] CWH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:43.371787] [INFO ] [MAIN ] CWH: Wiping existing primary data before full refresh. [2025-10-20T18:11:43.371994] [INFO ] [LOCAL ] CWH: Starting deletion of primary data. [2025-10-20T18:11:43.372327] [DEBUG] [LOCAL ] CWH: Deleted primary data file. [2025-10-20T18:11:43.372382] [INFO ] [MAIN ] CWH: Starting full primary refresh. [2025-10-20T18:11:43.456583] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-06. [2025-10-20T18:11:43.612941] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T18:11:43.766637] [INFO ] [REMOTE ] CWH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:44.251340] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-02. [2025-10-20T18:11:45.051341] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-01. [2025-10-20T18:11:45.097627] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-06 (2.44 seconds). [2025-10-20T18:11:45.462791] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-10-20T18:11:45.856562] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-07. [2025-10-20T18:11:45.866851] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T18:11:46.651399] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-07. [2025-10-20T18:11:46.699714] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T18:11:47.451426] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-03. [2025-10-20T18:11:48.251446] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-02. [2025-10-20T18:11:48.313208] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-07 (2.46 seconds). [2025-10-20T18:11:48.376780] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T18:11:49.051416] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-08. [2025-10-20T18:11:49.120176] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T18:11:49.853372] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-08. [2025-10-20T18:11:49.861470] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:11:50.651383] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-04. [2025-10-20T18:11:51.257741] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T18:11:51.311990] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:11:51.451437] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-09. [2025-10-20T18:11:52.244385] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:11:52.253607] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-03. [2025-10-20T18:11:52.968228] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T18:11:53.051419] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-05. [2025-10-20T18:11:53.738081] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T18:11:53.851509] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-10. [2025-10-20T18:11:54.651365] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-04. [2025-10-20T18:11:54.729109] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:11:55.456604] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-09. [2025-10-20T18:11:55.473009] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T18:11:55.482412] [INFO ] [REMOTE ] CWAN: Full fetch complete (75556 records). [2025-10-20T18:11:55.482568] [DEBUG] [VALIDATOR ] CWAN: Validating primary data. [2025-10-20T18:11:55.883212] [DEBUG] [VALIDATOR ] CWAN: All expected trading days are present. [2025-10-20T18:11:55.883311] [DEBUG] [VALIDATOR ] CWAN: Primary data validation passed. [2025-10-20T18:11:55.884368] [INFO ] [MAIN ] CWAN: Fetched primary data is valid. [2025-10-20T18:11:55.884457] [DEBUG] [LOCAL ] CWAN: Saving primary data. [2025-10-20T18:11:55.886558] [DEBUG] [LOCAL ] CWAN: Trimmed 75556 records to 52668 (keeping 200 days of data) [2025-10-20T18:11:56.196959] [DEBUG] [LOCAL ] CWAN: Primary data saved (52668 records). [2025-10-20T18:11:56.197085] [INFO ] [MAIN ] CWAN: Full history saved. [2025-10-20T18:11:56.197165] [INFO ] [MAIN ] Update Progress: 568/2367 symbols completed. CWAN took 27.4s. Estimated remaining: 3h 47m 37s. Rate: 7.9 symbols/min [2025-10-20T18:11:56.260267] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-06. [2025-10-20T18:11:56.378186] [INFO ] [MAIN ] CWK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:56.421837] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:11:56.478670] [DEBUG] [LOCAL ] CWK: Loaded 50364 existing records from primary storage. [2025-10-20T18:11:56.478989] [INFO ] [MAIN ] CWK: Loaded 50364 existing records. [2025-10-20T18:11:56.657694] [WARN ] [MAIN ] CWK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:56.657918] [INFO ] [MAIN ] CWK: Wiping existing primary data before full refresh. [2025-10-20T18:11:56.657986] [INFO ] [LOCAL ] CWK: Starting deletion of primary data. [2025-10-20T18:11:56.658196] [DEBUG] [LOCAL ] CWK: Deleted primary data file. [2025-10-20T18:11:56.658255] [INFO ] [MAIN ] CWK: Starting full primary refresh. [2025-10-20T18:11:57.019987] [INFO ] [REMOTE ] CWK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:57.051381] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-01. [2025-10-20T18:11:57.677400] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T18:11:57.851374] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-05. [2025-10-20T18:11:57.953908] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T18:11:58.159595] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:11:58.651372] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-02. [2025-10-20T18:11:59.297881] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T18:11:59.451387] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-07. [2025-10-20T18:11:59.877406] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:12:00.251439] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-03. [2025-10-20T18:12:01.051431] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-10. [2025-10-20T18:12:01.428266] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T18:12:01.851364] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-08. [2025-10-20T18:12:02.101312] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T18:12:02.651365] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-04. [2025-10-20T18:12:03.329631] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:12:03.446111] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-10 (2.39 seconds). [2025-10-20T18:12:03.453284] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-06. [2025-10-20T18:12:03.454945] [INFO ] [REMOTE ] CVX: Full fetch complete (77940 records). [2025-10-20T18:12:03.455036] [DEBUG] [VALIDATOR ] CVX: Validating primary data. [2025-10-20T18:12:03.844850] [DEBUG] [VALIDATOR ] CVX: All expected trading days are present. [2025-10-20T18:12:03.844964] [DEBUG] [VALIDATOR ] CVX: Primary data validation passed. [2025-10-20T18:12:03.845907] [INFO ] [MAIN ] CVX: Fetched primary data is valid. [2025-10-20T18:12:03.846054] [DEBUG] [LOCAL ] CVX: Saving primary data. [2025-10-20T18:12:03.847964] [DEBUG] [LOCAL ] CVX: Trimmed 77940 records to 53760 (keeping 200 days of data) [2025-10-20T18:12:03.982688] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T18:12:04.222658] [DEBUG] [LOCAL ] CVX: Primary data saved (53760 records). [2025-10-20T18:12:04.222799] [INFO ] [MAIN ] CVX: Full history saved. [2025-10-20T18:12:04.222871] [INFO ] [MAIN ] Update Progress: 569/2367 symbols completed. CVX took 51.9s. Estimated remaining: 3h 47m 30s. Rate: 7.9 symbols/min [2025-10-20T18:12:04.256477] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-09. [2025-10-20T18:12:04.390500] [INFO ] [MAIN ] CWT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:04.417831] [DEBUG] [LOCAL ] CWT: Loaded 28510 existing records from primary storage. [2025-10-20T18:12:04.418090] [INFO ] [MAIN ] CWT: Loaded 28510 existing records. [2025-10-20T18:12:04.593610] [WARN ] [MAIN ] CWT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:04.593957] [INFO ] [MAIN ] CWT: Wiping existing primary data before full refresh. [2025-10-20T18:12:04.594017] [INFO ] [LOCAL ] CWT: Starting deletion of primary data. [2025-10-20T18:12:04.594264] [DEBUG] [LOCAL ] CWT: Deleted primary data file. [2025-10-20T18:12:04.594320] [INFO ] [MAIN ] CWT: Starting full primary refresh. [2025-10-20T18:12:04.958029] [INFO ] [REMOTE ] CWT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:05.025239] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T18:12:05.051347] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-05. [2025-10-20T18:12:05.851347] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-01. [2025-10-20T18:12:06.025632] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T18:12:06.492034] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:12:06.651342] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-07. [2025-10-20T18:12:07.222488] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T18:12:07.451388] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-10. [2025-10-20T18:12:08.251409] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-06. [2025-10-20T18:12:08.400385] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:12:09.051370] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-08. [2025-10-20T18:12:09.222727] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T18:12:09.230436] [INFO ] [REMOTE ] CWEN: Full fetch complete (65523 records). [2025-10-20T18:12:09.230603] [DEBUG] [VALIDATOR ] CWEN: Validating primary data. [2025-10-20T18:12:09.637197] [DEBUG] [VALIDATOR ] CWEN: All expected trading days are present. [2025-10-20T18:12:09.637301] [DEBUG] [VALIDATOR ] CWEN: Primary data validation passed. [2025-10-20T18:12:09.638126] [INFO ] [MAIN ] CWEN: Fetched primary data is valid. [2025-10-20T18:12:09.638210] [DEBUG] [LOCAL ] CWEN: Saving primary data. [2025-10-20T18:12:09.639884] [DEBUG] [LOCAL ] CWEN: Trimmed 65523 records to 45314 (keeping 200 days of data) [2025-10-20T18:12:09.852072] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-02. [2025-10-20T18:12:09.885412] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T18:12:09.953293] [DEBUG] [LOCAL ] CWEN: Primary data saved (45314 records). [2025-10-20T18:12:09.953449] [INFO ] [MAIN ] CWEN: Full history saved. [2025-10-20T18:12:09.953970] [INFO ] [MAIN ] Update Progress: 570/2367 symbols completed. CWEN took 28.9s. Estimated remaining: 3h 47m 17s. Rate: 7.9 symbols/min [2025-10-20T18:12:10.126676] [INFO ] [MAIN ] CX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:10.175870] [DEBUG] [LOCAL ] CX: Loaded 53605 existing records from primary storage. [2025-10-20T18:12:10.176198] [INFO ] [MAIN ] CX: Loaded 53605 existing records. [2025-10-20T18:12:10.357789] [WARN ] [MAIN ] CX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:10.358030] [INFO ] [MAIN ] CX: Wiping existing primary data before full refresh. [2025-10-20T18:12:10.358075] [INFO ] [LOCAL ] CX: Starting deletion of primary data. [2025-10-20T18:12:10.358461] [DEBUG] [LOCAL ] CX: Deleted primary data file. [2025-10-20T18:12:10.358516] [INFO ] [MAIN ] CX: Starting full primary refresh. [2025-10-20T18:12:10.655231] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-07. [2025-10-20T18:12:10.777119] [INFO ] [REMOTE ] CX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:10.810706] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:12:11.110770] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T18:12:11.451354] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-09. [2025-10-20T18:12:12.066192] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:12:12.251408] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-08. [2025-10-20T18:12:13.051391] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-03. [2025-10-20T18:12:13.087560] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T18:12:13.851400] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-10. [2025-10-20T18:12:13.991709] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:12:14.372772] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:12:14.651411] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-04. [2025-10-20T18:12:15.265499] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T18:12:15.272945] [INFO ] [REMOTE ] CWH: Full fetch complete (73174 records). [2025-10-20T18:12:15.273101] [DEBUG] [VALIDATOR ] CWH: Validating primary data. [2025-10-20T18:12:15.456442] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-09. [2025-10-20T18:12:15.682292] [DEBUG] [VALIDATOR ] CWH: All expected trading days are present. [2025-10-20T18:12:15.682399] [DEBUG] [VALIDATOR ] CWH: Primary data validation passed. [2025-10-20T18:12:15.683697] [INFO ] [MAIN ] CWH: Fetched primary data is valid. [2025-10-20T18:12:15.683781] [DEBUG] [LOCAL ] CWH: Saving primary data. [2025-10-20T18:12:15.685798] [DEBUG] [LOCAL ] CWH: Trimmed 73174 records to 51314 (keeping 200 days of data) [2025-10-20T18:12:16.001110] [DEBUG] [LOCAL ] CWH: Primary data saved (51314 records). [2025-10-20T18:12:16.001246] [INFO ] [MAIN ] CWH: Full history saved. [2025-10-20T18:12:16.001322] [INFO ] [MAIN ] Update Progress: 571/2367 symbols completed. CWH took 33.0s. Estimated remaining: 3h 47m 4s. Rate: 7.9 symbols/min [2025-10-20T18:12:16.171768] [INFO ] [MAIN ] CXE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:16.192734] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T18:12:16.204584] [DEBUG] [LOCAL ] CXE: Loaded 6233 existing records from primary storage. [2025-10-20T18:12:16.204800] [INFO ] [MAIN ] CXE: Loaded 6233 existing records. [2025-10-20T18:12:16.259348] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-01. [2025-10-20T18:12:16.385184] [WARN ] [MAIN ] CXE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:16.385391] [INFO ] [MAIN ] CXE: Wiping existing primary data before full refresh. [2025-10-20T18:12:16.385439] [INFO ] [LOCAL ] CXE: Starting deletion of primary data. [2025-10-20T18:12:16.385600] [DEBUG] [LOCAL ] CXE: Deleted primary data file. [2025-10-20T18:12:16.385693] [INFO ] [MAIN ] CXE: Starting full primary refresh. [2025-10-20T18:12:16.741398] [INFO ] [REMOTE ] CXE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:16.954146] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:12:17.051375] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-01. [2025-10-20T18:12:17.851356] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-05. [2025-10-20T18:12:17.948533] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:12:18.000614] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T18:12:18.651439] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-02. [2025-10-20T18:12:18.778084] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:12:19.451446] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-02. [2025-10-20T18:12:20.125043] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T18:12:20.228421] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:12:20.251356] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-03. [2025-10-20T18:12:21.051386] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-03. [2025-10-20T18:12:21.703292] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T18:12:21.851365] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-04. [2025-10-20T18:12:21.904576] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:12:22.465452] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:12:22.651422] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-04. [2025-10-20T18:12:23.451378] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-10. [2025-10-20T18:12:24.251425] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-06. [2025-10-20T18:12:24.428450] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T18:12:25.040350] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T18:12:25.051348] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-05. [2025-10-20T18:12:25.320018] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:12:25.327818] [INFO ] [REMOTE ] CWK: Full fetch complete (71322 records). [2025-10-20T18:12:25.327966] [DEBUG] [VALIDATOR ] CWK: Validating primary data. [2025-10-20T18:12:25.727596] [DEBUG] [VALIDATOR ] CWK: All expected trading days are present. [2025-10-20T18:12:25.727709] [DEBUG] [VALIDATOR ] CWK: Primary data validation passed. [2025-10-20T18:12:25.728530] [INFO ] [MAIN ] CWK: Fetched primary data is valid. [2025-10-20T18:12:25.728574] [DEBUG] [LOCAL ] CWK: Saving primary data. [2025-10-20T18:12:25.730153] [DEBUG] [LOCAL ] CWK: Trimmed 71322 records to 49668 (keeping 200 days of data) [2025-10-20T18:12:25.822555] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T18:12:25.851739] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-05. [2025-10-20T18:12:26.030012] [DEBUG] [LOCAL ] CWK: Primary data saved (49668 records). [2025-10-20T18:12:26.030153] [INFO ] [MAIN ] CWK: Full history saved. [2025-10-20T18:12:26.030230] [INFO ] [MAIN ] Update Progress: 572/2367 symbols completed. CWK took 29.8s. Estimated remaining: 3h 47m 5s. Rate: 7.9 symbols/min [2025-10-20T18:12:26.196583] [INFO ] [MAIN ] CXH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:26.203993] [DEBUG] [LOCAL ] CXH: Loaded 4462 existing records from primary storage. [2025-10-20T18:12:26.204205] [INFO ] [MAIN ] CXH: Loaded 4462 existing records. [2025-10-20T18:12:26.378736] [WARN ] [MAIN ] CXH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:26.378971] [INFO ] [MAIN ] CXH: Wiping existing primary data before full refresh. [2025-10-20T18:12:26.379026] [INFO ] [LOCAL ] CXH: Starting deletion of primary data. [2025-10-20T18:12:26.379209] [DEBUG] [LOCAL ] CXH: Deleted primary data file. [2025-10-20T18:12:26.379265] [INFO ] [MAIN ] CXH: Starting full primary refresh. [2025-10-20T18:12:26.656625] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-06. [2025-10-20T18:12:26.749773] [INFO ] [REMOTE ] CXH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:27.330153] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T18:12:27.451338] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-01. [2025-10-20T18:12:27.686195] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T18:12:27.824628] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-01 (0.37 seconds). [2025-10-20T18:12:28.251394] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-02. [2025-10-20T18:12:28.702171] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T18:12:29.051392] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-03. [2025-10-20T18:12:29.411753] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-10-20T18:12:29.851414] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-04. [2025-10-20T18:12:30.342823] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T18:12:30.651388] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-05. [2025-10-20T18:12:31.234977] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T18:12:31.451372] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-07. [2025-10-20T18:12:32.251380] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-06. [2025-10-20T18:12:32.652946] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-10-20T18:12:32.915248] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:12:33.051402] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-06. [2025-10-20T18:12:33.851433] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-07. [2025-10-20T18:12:34.312018] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T18:12:34.650291] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T18:12:34.659756] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-07. [2025-10-20T18:12:35.372819] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T18:12:35.451402] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-07. [2025-10-20T18:12:36.251436] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-08. [2025-10-20T18:12:37.051456] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-08. [2025-10-20T18:12:37.112757] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T18:12:37.395108] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T18:12:37.461304] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-10-20T18:12:37.851489] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-08. [2025-10-20T18:12:38.651369] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-09. [2025-10-20T18:12:39.451413] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-09. [2025-10-20T18:12:39.585208] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:12:39.640016] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T18:12:40.139575] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T18:12:40.251436] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-09. [2025-10-20T18:12:41.051458] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-10. [2025-10-20T18:12:41.851412] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-08. [2025-10-20T18:12:42.085358] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:12:42.118571] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T18:12:42.124078] [INFO ] [REMOTE ] CWT: Full fetch complete (40973 records). [2025-10-20T18:12:42.124301] [DEBUG] [VALIDATOR ] CWT: Validating primary data. [2025-10-20T18:12:42.125364] [FATAL] [VALIDATOR ] CWT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:12:42.125479] [FATAL] [MAIN ] CWT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:12:42.125537] [WARN ] [MAIN ] CWT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:12:42.125619] [INFO ] [LOCAL ] CWT: Starting deletion of primary data. [2025-10-20T18:12:42.125937] [DEBUG] [LOCAL ] CWT: No primary data file found to delete. [2025-10-20T18:12:42.125994] [INFO ] [LOCAL ] CWT: Starting deletion of derivative data. [2025-10-20T18:12:42.126184] [DEBUG] [LOCAL ] CWT: Deleted derivative data file. [2025-10-20T18:12:42.126312] [INFO ] [MAIN ] Update Progress: 573/2367 symbols completed. CWT took 37.9s. Estimated remaining: 3h 47m 24s. Rate: 7.9 symbols/min [2025-10-20T18:12:42.351169] [INFO ] [MAIN ] CXM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:42.402245] [DEBUG] [LOCAL ] CXM: Loaded 48245 existing records from primary storage. [2025-10-20T18:12:42.402630] [INFO ] [MAIN ] CXM: Loaded 48245 existing records. [2025-10-20T18:12:42.521812] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T18:12:42.632233] [WARN ] [MAIN ] CXM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:42.632460] [INFO ] [MAIN ] CXM: Wiping existing primary data before full refresh. [2025-10-20T18:12:42.632509] [INFO ] [LOCAL ] CXM: Starting deletion of primary data. [2025-10-20T18:12:42.632766] [DEBUG] [LOCAL ] CXM: Deleted primary data file. [2025-10-20T18:12:42.632823] [INFO ] [MAIN ] CXM: Starting full primary refresh. [2025-10-20T18:12:42.656492] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-10. [2025-10-20T18:12:43.013928] [INFO ] [REMOTE ] CXM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:43.174107] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T18:12:43.175004] [INFO ] [REMOTE ] CXH: Full fetch complete (5302 records). [2025-10-20T18:12:43.175074] [DEBUG] [VALIDATOR ] CXH: Validating primary data. [2025-10-20T18:12:43.456486] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-09. [2025-10-20T18:12:43.550698] [DEBUG] [VALIDATOR ] CXH: All expected trading days are present. [2025-10-20T18:12:43.550784] [DEBUG] [VALIDATOR ] CXH: Primary data validation passed. [2025-10-20T18:12:43.550942] [INFO ] [MAIN ] CXH: Fetched primary data is valid. [2025-10-20T18:12:43.550981] [DEBUG] [LOCAL ] CXH: Saving primary data. [2025-10-20T18:12:43.551382] [DEBUG] [LOCAL ] CXH: Trimmed 5302 records to 4421 (keeping 200 days of data) [2025-10-20T18:12:43.577539] [DEBUG] [LOCAL ] CXH: Primary data saved (4421 records). [2025-10-20T18:12:43.577712] [INFO ] [MAIN ] CXH: Full history saved. [2025-10-20T18:12:43.577823] [INFO ] [MAIN ] Update Progress: 574/2367 symbols completed. CXH took 17.5s. Estimated remaining: 3h 46m 57s. Rate: 7.9 symbols/min [2025-10-20T18:12:43.752119] [INFO ] [MAIN ] CXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:43.783045] [DEBUG] [LOCAL ] CXT: Loaded 31488 existing records from primary storage. [2025-10-20T18:12:43.783334] [INFO ] [MAIN ] CXT: Loaded 31488 existing records. [2025-10-20T18:12:43.984266] [WARN ] [MAIN ] CXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:43.984465] [INFO ] [MAIN ] CXT: Wiping existing primary data before full refresh. [2025-10-20T18:12:43.984513] [INFO ] [LOCAL ] CXT: Starting deletion of primary data. [2025-10-20T18:12:43.984737] [DEBUG] [LOCAL ] CXT: Deleted primary data file. [2025-10-20T18:12:43.984793] [INFO ] [MAIN ] CXT: Starting full primary refresh. [2025-10-20T18:12:44.184443] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-10-20T18:12:44.256457] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-10. [2025-10-20T18:12:44.363615] [INFO ] [REMOTE ] CXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:45.051350] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-10. [2025-10-20T18:12:45.851441] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-01. [2025-10-20T18:12:45.876001] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T18:12:45.877488] [INFO ] [REMOTE ] CXE: Full fetch complete (11510 records). [2025-10-20T18:12:45.877557] [DEBUG] [VALIDATOR ] CXE: Validating primary data. [2025-10-20T18:12:46.177487] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T18:12:46.230232] [INFO ] [REMOTE ] CX: Full fetch complete (76578 records). [2025-10-20T18:12:46.259038] [DEBUG] [VALIDATOR ] CX: Validating primary data. [2025-10-20T18:12:46.387891] [DEBUG] [VALIDATOR ] CXE: All expected trading days are present. [2025-10-20T18:12:46.414145] [DEBUG] [VALIDATOR ] CXE: Primary data validation passed. [2025-10-20T18:12:46.418135] [INFO ] [MAIN ] CXE: Fetched primary data is valid. [2025-10-20T18:12:46.418488] [DEBUG] [LOCAL ] CXE: Saving primary data. [2025-10-20T18:12:46.445100] [DEBUG] [LOCAL ] CXE: Trimmed 11510 records to 6342 (keeping 200 days of data) [2025-10-20T18:12:46.654910] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-01. [2025-10-20T18:12:46.678035] [DEBUG] [LOCAL ] CXE: Primary data saved (6342 records). [2025-10-20T18:12:46.684186] [INFO ] [MAIN ] CXE: Full history saved. [2025-10-20T18:12:46.684320] [INFO ] [MAIN ] Update Progress: 575/2367 symbols completed. CXE took 30.7s. Estimated remaining: 3h 46m 35s. Rate: 7.9 symbols/min [2025-10-20T18:12:46.897795] [DEBUG] [VALIDATOR ] CX: All expected trading days are present. [2025-10-20T18:12:46.898242] [DEBUG] [VALIDATOR ] CX: Primary data validation passed. [2025-10-20T18:12:46.910297] [INFO ] [MAIN ] CX: Fetched primary data is valid. [2025-10-20T18:12:46.915555] [DEBUG] [LOCAL ] CX: Saving primary data. [2025-10-20T18:12:46.979527] [INFO ] [MAIN ] CXW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:46.980732] [DEBUG] [LOCAL ] CX: Trimmed 76578 records to 52835 (keeping 200 days of data) [2025-10-20T18:12:47.235856] [DEBUG] [LOCAL ] CXW: Loaded 44487 existing records from primary storage. [2025-10-20T18:12:47.258355] [INFO ] [MAIN ] CXW: Loaded 44487 existing records. [2025-10-20T18:12:47.281408] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:12:47.485515] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-02. [2025-10-20T18:12:47.509440] [WARN ] [MAIN ] CXW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:47.527151] [INFO ] [MAIN ] CXW: Wiping existing primary data before full refresh. [2025-10-20T18:12:47.528219] [INFO ] [LOCAL ] CXW: Starting deletion of primary data. [2025-10-20T18:12:47.531803] [DEBUG] [LOCAL ] CXW: Deleted primary data file. [2025-10-20T18:12:47.532428] [INFO ] [MAIN ] CXW: Starting full primary refresh. [2025-10-20T18:12:47.751932] [DEBUG] [LOCAL ] CX: Primary data saved (52835 records). [2025-10-20T18:12:47.790732] [INFO ] [MAIN ] CX: Full history saved. [2025-10-20T18:12:47.824121] [INFO ] [MAIN ] Update Progress: 576/2367 symbols completed. CX took 37.9s. Estimated remaining: 3h 46m 8s. Rate: 7.9 symbols/min [2025-10-20T18:12:48.050825] [INFO ] [REMOTE ] CXW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:48.171735] [INFO ] [MAIN ] CYD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:48.201819] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T18:12:48.237103] [DEBUG] [LOCAL ] CYD: Loaded 27356 existing records from primary storage. [2025-10-20T18:12:48.237422] [INFO ] [MAIN ] CYD: Loaded 27356 existing records. [2025-10-20T18:12:48.258410] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-02. [2025-10-20T18:12:48.432166] [WARN ] [MAIN ] CYD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:48.432396] [INFO ] [MAIN ] CYD: Wiping existing primary data before full refresh. [2025-10-20T18:12:48.432447] [INFO ] [LOCAL ] CYD: Starting deletion of primary data. [2025-10-20T18:12:48.432704] [DEBUG] [LOCAL ] CYD: Deleted primary data file. [2025-10-20T18:12:48.432759] [INFO ] [MAIN ] CYD: Starting full primary refresh. [2025-10-20T18:12:48.811432] [INFO ] [REMOTE ] CYD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:48.834731] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T18:12:49.051399] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-01. [2025-10-20T18:12:49.759929] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T18:12:49.851386] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-03. [2025-10-20T18:12:50.651361] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-03. [2025-10-20T18:12:50.697807] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T18:12:51.451508] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-01. [2025-10-20T18:12:51.457088] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:12:52.031296] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:12:52.031562] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T18:12:52.251395] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-02. [2025-10-20T18:12:53.051363] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-04. [2025-10-20T18:12:53.218300] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T18:12:53.851401] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-04. [2025-10-20T18:12:54.484593] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:12:54.651386] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-03. [2025-10-20T18:12:55.262278] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T18:12:55.451409] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-05. [2025-10-20T18:12:56.037868] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T18:12:56.251378] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-02. [2025-10-20T18:12:56.872937] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T18:12:57.051385] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-06. [2025-10-20T18:12:57.844940] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T18:12:57.853403] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-04. [2025-10-20T18:12:58.651366] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-03. [2025-10-20T18:12:58.700403] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:12:59.115019] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T18:12:59.451388] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-05. [2025-10-20T18:12:59.910043] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T18:13:00.251415] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-07. [2025-10-20T18:13:00.563612] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T18:13:01.051450] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-05. [2025-10-20T18:13:01.851356] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-06. [2025-10-20T18:13:02.056698] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:13:02.601827] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:13:02.651358] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-04. [2025-10-20T18:13:02.794174] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:13:03.451429] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-06. [2025-10-20T18:13:04.054886] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T18:13:04.251411] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-05. [2025-10-20T18:13:04.683091] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T18:13:05.051474] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-07. [2025-10-20T18:13:05.777801] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T18:13:05.851353] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-06. [2025-10-20T18:13:06.540063] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:13:06.651373] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-08. [2025-10-20T18:13:07.234136] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:13:07.451380] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-08. [2025-10-20T18:13:07.644793] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:13:08.251426] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-09. [2025-10-20T18:13:09.051367] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-07. [2025-10-20T18:13:09.130660] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T18:13:09.207462] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T18:13:09.851350] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-09. [2025-10-20T18:13:10.315737] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T18:13:10.651370] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-07. [2025-10-20T18:13:11.424227] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T18:13:11.451349] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-10. [2025-10-20T18:13:12.251350] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-10. [2025-10-20T18:13:12.340508] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T18:13:12.594822] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T18:13:12.603787] [INFO ] [REMOTE ] CXM: Full fetch complete (68728 records). [2025-10-20T18:13:12.603953] [DEBUG] [VALIDATOR ] CXM: Validating primary data. [2025-10-20T18:13:13.009925] [DEBUG] [VALIDATOR ] CXM: All expected trading days are present. [2025-10-20T18:13:13.010010] [DEBUG] [VALIDATOR ] CXM: Primary data validation passed. [2025-10-20T18:13:13.010817] [INFO ] [MAIN ] CXM: Fetched primary data is valid. [2025-10-20T18:13:13.010879] [DEBUG] [LOCAL ] CXM: Saving primary data. [2025-10-20T18:13:13.012208] [DEBUG] [LOCAL ] CXM: Trimmed 68728 records to 47532 (keeping 200 days of data) [2025-10-20T18:13:13.056926] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-08. [2025-10-20T18:13:13.284442] [DEBUG] [LOCAL ] CXM: Primary data saved (47532 records). [2025-10-20T18:13:13.284608] [INFO ] [MAIN ] CXM: Full history saved. [2025-10-20T18:13:13.284738] [INFO ] [MAIN ] Update Progress: 577/2367 symbols completed. CXM took 31.2s. Estimated remaining: 3h 46m 55s. Rate: 7.9 symbols/min [2025-10-20T18:13:13.477144] [INFO ] [MAIN ] CYH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:13.490848] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T18:13:13.496009] [INFO ] [REMOTE ] CYD: Full fetch complete (35612 records). [2025-10-20T18:13:13.496089] [DEBUG] [VALIDATOR ] CYD: Validating primary data. [2025-10-20T18:13:13.811711] [DEBUG] [LOCAL ] CYH: Loaded 48440 existing records from primary storage. [2025-10-20T18:13:13.822696] [INFO ] [MAIN ] CYH: Loaded 48440 existing records. [2025-10-20T18:13:13.868778] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-08. [2025-10-20T18:13:13.992276] [DEBUG] [VALIDATOR ] CYD: All expected trading days are present. [2025-10-20T18:13:14.001206] [DEBUG] [VALIDATOR ] CYD: Primary data validation passed. [2025-10-20T18:13:14.004424] [INFO ] [MAIN ] CYD: Fetched primary data is valid. [2025-10-20T18:13:14.004521] [DEBUG] [LOCAL ] CYD: Saving primary data. [2025-10-20T18:13:14.058176] [DEBUG] [LOCAL ] CYD: Trimmed 35612 records to 27167 (keeping 200 days of data) [2025-10-20T18:13:14.087083] [WARN ] [MAIN ] CYH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:14.147394] [INFO ] [MAIN ] CYH: Wiping existing primary data before full refresh. [2025-10-20T18:13:14.147976] [INFO ] [LOCAL ] CYH: Starting deletion of primary data. [2025-10-20T18:13:14.175479] [DEBUG] [LOCAL ] CYH: Deleted primary data file. [2025-10-20T18:13:14.175854] [INFO ] [MAIN ] CYH: Starting full primary refresh. [2025-10-20T18:13:14.581143] [DEBUG] [LOCAL ] CYD: Primary data saved (27167 records). [2025-10-20T18:13:14.582594] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T18:13:14.588322] [INFO ] [MAIN ] CYD: Full history saved. [2025-10-20T18:13:14.600173] [INFO ] [MAIN ] Update Progress: 578/2367 symbols completed. CYD took 26.8s. Estimated remaining: 3h 46m 28s. Rate: 7.9 symbols/min [2025-10-20T18:13:14.659956] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-09. [2025-10-20T18:13:14.709414] [INFO ] [REMOTE ] CYH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:14.847363] [INFO ] [MAIN ] D: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:14.847611] [DEBUG] [LOCAL ] D: No existing data file found in primary storage. [2025-10-20T18:13:14.847680] [WARN ] [MAIN ] D: No existing data found, performing full refresh instead. [2025-10-20T18:13:14.847796] [INFO ] [MAIN ] D: Wiping existing primary data before full refresh. [2025-10-20T18:13:14.847836] [INFO ] [LOCAL ] D: Starting deletion of primary data. [2025-10-20T18:13:14.847876] [DEBUG] [LOCAL ] D: No primary data file found to delete. [2025-10-20T18:13:14.847907] [INFO ] [MAIN ] D: Starting full primary refresh. [2025-10-20T18:13:15.193210] [INFO ] [REMOTE ] D: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:15.451385] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-01. [2025-10-20T18:13:15.589029] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:13:16.182980] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T18:13:16.251343] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-10. [2025-10-20T18:13:17.051354] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-09. [2025-10-20T18:13:17.159365] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T18:13:17.431042] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T18:13:17.435992] [INFO ] [REMOTE ] CXT: Full fetch complete (45706 records). [2025-10-20T18:13:17.436088] [DEBUG] [VALIDATOR ] CXT: Validating primary data. [2025-10-20T18:13:17.800440] [DEBUG] [VALIDATOR ] CXT: All expected trading days are present. [2025-10-20T18:13:17.800522] [DEBUG] [VALIDATOR ] CXT: Primary data validation passed. [2025-10-20T18:13:17.801206] [INFO ] [MAIN ] CXT: Fetched primary data is valid. [2025-10-20T18:13:17.801268] [DEBUG] [LOCAL ] CXT: Saving primary data. [2025-10-20T18:13:17.802205] [DEBUG] [LOCAL ] CXT: Trimmed 45706 records to 30937 (keeping 200 days of data) [2025-10-20T18:13:17.853421] [INFO ] [REMOTE ] D: Fetching primary data for 2025-01. [2025-10-20T18:13:17.990876] [DEBUG] [LOCAL ] CXT: Primary data saved (30937 records). [2025-10-20T18:13:17.990999] [INFO ] [MAIN ] CXT: Full history saved. [2025-10-20T18:13:17.991074] [INFO ] [MAIN ] Update Progress: 579/2367 symbols completed. CXT took 34.4s. Estimated remaining: 3h 46m 8s. Rate: 7.9 symbols/min [2025-10-20T18:13:18.160206] [INFO ] [MAIN ] DAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:18.173838] [DEBUG] [LOCAL ] DAC: Loaded 12751 existing records from primary storage. [2025-10-20T18:13:18.174109] [INFO ] [MAIN ] DAC: Loaded 12751 existing records. [2025-10-20T18:13:18.328352] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:13:18.394239] [WARN ] [MAIN ] DAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:18.394638] [INFO ] [MAIN ] DAC: Wiping existing primary data before full refresh. [2025-10-20T18:13:18.394729] [INFO ] [LOCAL ] DAC: Starting deletion of primary data. [2025-10-20T18:13:18.394983] [DEBUG] [LOCAL ] DAC: Deleted primary data file. [2025-10-20T18:13:18.395044] [INFO ] [MAIN ] DAC: Starting full primary refresh. [2025-10-20T18:13:18.656588] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-10. [2025-10-20T18:13:18.779552] [INFO ] [REMOTE ] DAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:19.451409] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-01. [2025-10-20T18:13:19.969671] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-01 (2.12 seconds). [2025-10-20T18:13:20.215022] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T18:13:20.251334] [INFO ] [REMOTE ] D: Fetching primary data for 2025-02. [2025-10-20T18:13:20.328235] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T18:13:20.334503] [INFO ] [REMOTE ] CXW: Full fetch complete (64874 records). [2025-10-20T18:13:20.334619] [DEBUG] [VALIDATOR ] CXW: Validating primary data. [2025-10-20T18:13:20.720251] [DEBUG] [VALIDATOR ] CXW: All expected trading days are present. [2025-10-20T18:13:20.720368] [DEBUG] [VALIDATOR ] CXW: Primary data validation passed. [2025-10-20T18:13:20.721143] [INFO ] [MAIN ] CXW: Fetched primary data is valid. [2025-10-20T18:13:20.721206] [DEBUG] [LOCAL ] CXW: Saving primary data. [2025-10-20T18:13:20.722454] [DEBUG] [LOCAL ] CXW: Trimmed 64874 records to 43775 (keeping 200 days of data) [2025-10-20T18:13:20.989221] [DEBUG] [LOCAL ] CXW: Primary data saved (43775 records). [2025-10-20T18:13:20.989358] [INFO ] [MAIN ] CXW: Full history saved. [2025-10-20T18:13:20.989436] [INFO ] [MAIN ] Update Progress: 580/2367 symbols completed. CXW took 34.3s. Estimated remaining: 3h 45m 46s. Rate: 7.9 symbols/min [2025-10-20T18:13:21.056556] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-02. [2025-10-20T18:13:21.157182] [INFO ] [MAIN ] DAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:21.205141] [DEBUG] [LOCAL ] DAL: Loaded 54533 existing records from primary storage. [2025-10-20T18:13:21.205505] [INFO ] [MAIN ] DAL: Loaded 54533 existing records. [2025-10-20T18:13:21.383035] [WARN ] [MAIN ] DAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:21.383222] [INFO ] [MAIN ] DAL: Wiping existing primary data before full refresh. [2025-10-20T18:13:21.383342] [INFO ] [LOCAL ] DAL: Starting deletion of primary data. [2025-10-20T18:13:21.383981] [DEBUG] [LOCAL ] DAL: Deleted primary data file. [2025-10-20T18:13:21.384037] [INFO ] [MAIN ] DAL: Starting full primary refresh. [2025-10-20T18:13:21.732621] [INFO ] [REMOTE ] DAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:21.851331] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-01. [2025-10-20T18:13:21.998378] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:13:22.651377] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-02. [2025-10-20T18:13:22.701825] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:13:23.451403] [INFO ] [REMOTE ] D: Fetching primary data for 2025-03. [2025-10-20T18:13:23.703574] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:13:23.926939] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T18:13:24.251427] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-02. [2025-10-20T18:13:25.051431] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-03. [2025-10-20T18:13:25.314339] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T18:13:25.851358] [INFO ] [REMOTE ] D: Fetching primary data for 2025-04. [2025-10-20T18:13:25.947597] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T18:13:26.267449] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-10-20T18:13:26.651409] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-04. [2025-10-20T18:13:27.451369] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-03. [2025-10-20T18:13:27.549205] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T18:13:27.754055] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T18:13:28.251431] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-05. [2025-10-20T18:13:29.051410] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-03. [2025-10-20T18:13:29.157699] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T18:13:29.317482] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T18:13:29.851420] [INFO ] [REMOTE ] D: Fetching primary data for 2025-05. [2025-10-20T18:13:30.579158] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T18:13:30.651365] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-04. [2025-10-20T18:13:31.451367] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-04. [2025-10-20T18:13:31.788431] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T18:13:32.251358] [INFO ] [REMOTE ] D: Fetching primary data for 2025-06. [2025-10-20T18:13:32.448027] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:13:33.051372] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-05. [2025-10-20T18:13:33.446674] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T18:13:33.851399] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-05. [2025-10-20T18:13:34.122544] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T18:13:34.651345] [INFO ] [REMOTE ] D: Fetching primary data for 2025-07. [2025-10-20T18:13:34.900195] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:13:35.451402] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-06. [2025-10-20T18:13:36.110413] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-10-20T18:13:36.251384] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-06. [2025-10-20T18:13:36.396147] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T18:13:36.491759] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:13:37.051415] [INFO ] [REMOTE ] D: Fetching primary data for 2025-08. [2025-10-20T18:13:37.851372] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-06. [2025-10-20T18:13:38.151468] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T18:13:38.651350] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-07. [2025-10-20T18:13:38.973054] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T18:13:39.074253] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-10-20T18:13:39.451342] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-07. [2025-10-20T18:13:40.251332] [INFO ] [REMOTE ] D: Fetching primary data for 2025-09. [2025-10-20T18:13:40.809084] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T18:13:41.051359] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-08. [2025-10-20T18:13:41.369925] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T18:13:41.851391] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-08. [2025-10-20T18:13:42.249917] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T18:13:42.651345] [INFO ] [REMOTE ] D: Fetching primary data for 2025-10. [2025-10-20T18:13:42.736861] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T18:13:43.451456] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-07. [2025-10-20T18:13:43.513741] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T18:13:43.773294] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T18:13:43.782791] [INFO ] [REMOTE ] D: Full fetch complete (77812 records). [2025-10-20T18:13:43.783272] [DEBUG] [VALIDATOR ] D: Validating primary data. [2025-10-20T18:13:44.207759] [DEBUG] [VALIDATOR ] D: All expected trading days are present. [2025-10-20T18:13:44.207864] [DEBUG] [VALIDATOR ] D: Primary data validation passed. [2025-10-20T18:13:44.208782] [INFO ] [MAIN ] D: Fetched primary data is valid. [2025-10-20T18:13:44.208849] [DEBUG] [LOCAL ] D: Saving primary data. [2025-10-20T18:13:44.210428] [DEBUG] [LOCAL ] D: Trimmed 77812 records to 53649 (keeping 200 days of data) [2025-10-20T18:13:44.264728] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-09. [2025-10-20T18:13:44.352477] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T18:13:44.560749] [DEBUG] [LOCAL ] D: Primary data saved (53649 records). [2025-10-20T18:13:44.560890] [INFO ] [MAIN ] D: Full history saved. [2025-10-20T18:13:44.560975] [INFO ] [MAIN ] Update Progress: 581/2367 symbols completed. D took 30.0s. Estimated remaining: 3h 46m 28s. Rate: 7.9 symbols/min [2025-10-20T18:13:44.726828] [INFO ] [MAIN ] DAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:44.772135] [DEBUG] [LOCAL ] DAN: Loaded 50069 existing records from primary storage. [2025-10-20T18:13:44.772439] [INFO ] [MAIN ] DAN: Loaded 50069 existing records. [2025-10-20T18:13:44.946758] [WARN ] [MAIN ] DAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:44.946982] [INFO ] [MAIN ] DAN: Wiping existing primary data before full refresh. [2025-10-20T18:13:44.947029] [INFO ] [LOCAL ] DAN: Starting deletion of primary data. [2025-10-20T18:13:44.947219] [DEBUG] [LOCAL ] DAN: Deleted primary data file. [2025-10-20T18:13:44.947257] [INFO ] [MAIN ] DAN: Starting full primary refresh. [2025-10-20T18:13:45.056478] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-08. [2025-10-20T18:13:45.327663] [INFO ] [REMOTE ] DAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:45.636143] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T18:13:45.851396] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-01. [2025-10-20T18:13:45.977010] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T18:13:46.651366] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-09. [2025-10-20T18:13:47.271984] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T18:13:47.451422] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-02. [2025-10-20T18:13:48.251434] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-10. [2025-10-20T18:13:48.482853] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:13:49.051793] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-10. [2025-10-20T18:13:49.057205] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:13:49.851398] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-03. [2025-10-20T18:13:49.886814] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T18:13:49.896619] [INFO ] [REMOTE ] CYH: Full fetch complete (69897 records). [2025-10-20T18:13:49.896821] [DEBUG] [VALIDATOR ] CYH: Validating primary data. [2025-10-20T18:13:50.081483] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:13:50.138959] [INFO ] [REMOTE ] DAL: Full fetch complete (77931 records). [2025-10-20T18:13:50.151298] [DEBUG] [VALIDATOR ] DAL: Validating primary data. [2025-10-20T18:13:50.661044] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-09. [2025-10-20T18:13:50.700921] [DEBUG] [VALIDATOR ] CYH: All expected trading days are present. [2025-10-20T18:13:50.719562] [DEBUG] [VALIDATOR ] CYH: Primary data validation passed. [2025-10-20T18:13:50.729899] [INFO ] [MAIN ] CYH: Fetched primary data is valid. [2025-10-20T18:13:50.774895] [DEBUG] [LOCAL ] CYH: Saving primary data. [2025-10-20T18:13:50.854168] [DEBUG] [LOCAL ] CYH: Trimmed 69897 records to 47742 (keeping 200 days of data) [2025-10-20T18:13:51.101960] [DEBUG] [VALIDATOR ] DAL: All expected trading days are present. [2025-10-20T18:13:51.102032] [DEBUG] [VALIDATOR ] DAL: Primary data validation passed. [2025-10-20T18:13:51.102848] [INFO ] [MAIN ] DAL: Fetched primary data is valid. [2025-10-20T18:13:51.107768] [DEBUG] [LOCAL ] DAL: Saving primary data. [2025-10-20T18:13:51.117546] [DEBUG] [LOCAL ] DAL: Trimmed 77931 records to 53753 (keeping 200 days of data) [2025-10-20T18:13:51.411256] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T18:13:51.453348] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-04. [2025-10-20T18:13:51.615233] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T18:13:51.637352] [DEBUG] [LOCAL ] CYH: Primary data saved (47742 records). [2025-10-20T18:13:51.638399] [INFO ] [MAIN ] CYH: Full history saved. [2025-10-20T18:13:51.639510] [INFO ] [MAIN ] Update Progress: 582/2367 symbols completed. CYH took 38.4s. Estimated remaining: 3h 46m 18s. Rate: 7.9 symbols/min [2025-10-20T18:13:51.828177] [INFO ] [MAIN ] DAO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:51.919837] [DEBUG] [LOCAL ] DAO: Loaded 20206 existing records from primary storage. [2025-10-20T18:13:51.921473] [INFO ] [MAIN ] DAO: Loaded 20206 existing records. [2025-10-20T18:13:52.109882] [WARN ] [MAIN ] DAO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:52.111745] [INFO ] [MAIN ] DAO: Wiping existing primary data before full refresh. [2025-10-20T18:13:52.112847] [INFO ] [LOCAL ] DAO: Starting deletion of primary data. [2025-10-20T18:13:52.113718] [DEBUG] [LOCAL ] DAO: Deleted primary data file. [2025-10-20T18:13:52.117978] [DEBUG] [LOCAL ] DAL: Primary data saved (53753 records). [2025-10-20T18:13:52.118129] [INFO ] [MAIN ] DAO: Starting full primary refresh. [2025-10-20T18:13:52.118339] [INFO ] [MAIN ] DAL: Full history saved. [2025-10-20T18:13:52.118487] [INFO ] [MAIN ] Update Progress: 583/2367 symbols completed. DAL took 31.1s. Estimated remaining: 3h 45m 49s. Rate: 7.9 symbols/min [2025-10-20T18:13:52.261856] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-10. [2025-10-20T18:13:52.516940] [INFO ] [MAIN ] DAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:52.677125] [INFO ] [REMOTE ] DAO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:52.712730] [DEBUG] [LOCAL ] DAR: Loaded 52762 existing records from primary storage. [2025-10-20T18:13:52.713141] [INFO ] [MAIN ] DAR: Loaded 52762 existing records. [2025-10-20T18:13:52.891095] [WARN ] [MAIN ] DAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:52.891370] [INFO ] [MAIN ] DAR: Wiping existing primary data before full refresh. [2025-10-20T18:13:52.891448] [INFO ] [LOCAL ] DAR: Starting deletion of primary data. [2025-10-20T18:13:52.891733] [DEBUG] [LOCAL ] DAR: Deleted primary data file. [2025-10-20T18:13:52.891832] [INFO ] [MAIN ] DAR: Starting full primary refresh. [2025-10-20T18:13:53.056673] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-01. [2025-10-20T18:13:53.221105] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:13:53.296638] [INFO ] [REMOTE ] DAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:53.319803] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:13:53.322314] [INFO ] [REMOTE ] DAC: Full fetch complete (19260 records). [2025-10-20T18:13:53.322453] [DEBUG] [VALIDATOR ] DAC: Validating primary data. [2025-10-20T18:13:53.711348] [DEBUG] [VALIDATOR ] DAC: All expected trading days are present. [2025-10-20T18:13:53.711430] [DEBUG] [VALIDATOR ] DAC: Primary data validation passed. [2025-10-20T18:13:53.711926] [INFO ] [MAIN ] DAC: Fetched primary data is valid. [2025-10-20T18:13:53.711987] [DEBUG] [LOCAL ] DAC: Saving primary data. [2025-10-20T18:13:53.712708] [DEBUG] [LOCAL ] DAC: Trimmed 19260 records to 12590 (keeping 200 days of data) [2025-10-20T18:13:53.793127] [DEBUG] [LOCAL ] DAC: Primary data saved (12590 records). [2025-10-20T18:13:53.793261] [INFO ] [MAIN ] DAC: Full history saved. [2025-10-20T18:13:53.793334] [INFO ] [MAIN ] Update Progress: 584/2367 symbols completed. DAC took 35.8s. Estimated remaining: 3h 45m 23s. Rate: 7.9 symbols/min [2025-10-20T18:13:53.857235] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-05. [2025-10-20T18:13:53.973278] [INFO ] [MAIN ] DAVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:54.019901] [DEBUG] [LOCAL ] DAVA: Loaded 42431 existing records from primary storage. [2025-10-20T18:13:54.020412] [INFO ] [MAIN ] DAVA: Loaded 42431 existing records. [2025-10-20T18:13:54.212597] [WARN ] [MAIN ] DAVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:54.213018] [INFO ] [MAIN ] DAVA: Wiping existing primary data before full refresh. [2025-10-20T18:13:54.213084] [INFO ] [LOCAL ] DAVA: Starting deletion of primary data. [2025-10-20T18:13:54.213330] [DEBUG] [LOCAL ] DAVA: Deleted primary data file. [2025-10-20T18:13:54.213386] [INFO ] [MAIN ] DAVA: Starting full primary refresh. [2025-10-20T18:13:54.355060] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T18:13:54.586618] [INFO ] [REMOTE ] DAVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:54.651367] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-02. [2025-10-20T18:13:55.445442] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T18:13:55.455126] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-01. [2025-10-20T18:13:55.983182] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:13:56.251429] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-01. [2025-10-20T18:13:56.882453] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:13:57.051422] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-03. [2025-10-20T18:13:57.842845] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T18:13:57.852669] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-02. [2025-10-20T18:13:58.112600] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T18:13:58.651410] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-04. [2025-10-20T18:13:59.040816] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T18:13:59.426932] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T18:13:59.451340] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-05. [2025-10-20T18:14:00.251328] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-06. [2025-10-20T18:14:00.577106] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T18:14:01.051367] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-06. [2025-10-20T18:14:01.625721] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T18:14:01.862620] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-02. [2025-10-20T18:14:01.868499] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:14:02.651373] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-07. [2025-10-20T18:14:03.451353] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-07. [2025-10-20T18:14:03.542241] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T18:14:03.555332] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T18:14:04.251367] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-08. [2025-10-20T18:14:05.051357] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-03. [2025-10-20T18:14:05.236558] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T18:14:05.369761] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T18:14:05.851429] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-09. [2025-10-20T18:14:06.572372] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T18:14:06.658013] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-10. [2025-10-20T18:14:06.663261] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:14:07.451368] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-04. [2025-10-20T18:14:07.802603] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T18:14:07.805823] [INFO ] [REMOTE ] DAO: Full fetch complete (32419 records). [2025-10-20T18:14:07.805906] [DEBUG] [VALIDATOR ] DAO: Validating primary data. [2025-10-20T18:14:08.179119] [DEBUG] [VALIDATOR ] DAO: All expected trading days are present. [2025-10-20T18:14:08.179271] [DEBUG] [VALIDATOR ] DAO: Primary data validation passed. [2025-10-20T18:14:08.179861] [INFO ] [MAIN ] DAO: Fetched primary data is valid. [2025-10-20T18:14:08.179947] [DEBUG] [LOCAL ] DAO: Saving primary data. [2025-10-20T18:14:08.180705] [DEBUG] [LOCAL ] DAO: Trimmed 32419 records to 19694 (keeping 200 days of data) [2025-10-20T18:14:08.260545] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-03. [2025-10-20T18:14:08.296510] [DEBUG] [LOCAL ] DAO: Primary data saved (19694 records). [2025-10-20T18:14:08.296732] [INFO ] [MAIN ] DAO: Full history saved. [2025-10-20T18:14:08.296944] [INFO ] [MAIN ] Update Progress: 585/2367 symbols completed. DAO took 16.7s. Estimated remaining: 3h 45m 37s. Rate: 7.9 symbols/min [2025-10-20T18:14:08.477169] [INFO ] [MAIN ] DAY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:08.477327] [DEBUG] [LOCAL ] DAY: No existing data file found in primary storage. [2025-10-20T18:14:08.477372] [WARN ] [MAIN ] DAY: No existing data found, performing full refresh instead. [2025-10-20T18:14:08.477502] [INFO ] [MAIN ] DAY: Wiping existing primary data before full refresh. [2025-10-20T18:14:08.477543] [INFO ] [LOCAL ] DAY: Starting deletion of primary data. [2025-10-20T18:14:08.477583] [DEBUG] [LOCAL ] DAY: No primary data file found to delete. [2025-10-20T18:14:08.477614] [INFO ] [MAIN ] DAY: Starting full primary refresh. [2025-10-20T18:14:08.835304] [INFO ] [REMOTE ] DAY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:09.051339] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-08. [2025-10-20T18:14:09.190017] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T18:14:09.833225] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T18:14:09.851346] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-05. [2025-10-20T18:14:10.449914] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T18:14:10.651364] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-09. [2025-10-20T18:14:11.388272] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:14:11.451369] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-06. [2025-10-20T18:14:11.904096] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:14:12.251355] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-10. [2025-10-20T18:14:13.056527] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-04. [2025-10-20T18:14:13.069616] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:14:13.636084] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T18:14:13.643632] [INFO ] [REMOTE ] DAN: Full fetch complete (71321 records). [2025-10-20T18:14:13.643815] [DEBUG] [VALIDATOR ] DAN: Validating primary data. [2025-10-20T18:14:13.856475] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-01. [2025-10-20T18:14:14.050284] [DEBUG] [VALIDATOR ] DAN: All expected trading days are present. [2025-10-20T18:14:14.050390] [DEBUG] [VALIDATOR ] DAN: Primary data validation passed. [2025-10-20T18:14:14.051213] [INFO ] [MAIN ] DAN: Fetched primary data is valid. [2025-10-20T18:14:14.051275] [DEBUG] [LOCAL ] DAN: Saving primary data. [2025-10-20T18:14:14.052750] [DEBUG] [LOCAL ] DAN: Trimmed 71321 records to 49407 (keeping 200 days of data) [2025-10-20T18:14:14.291916] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T18:14:14.379064] [DEBUG] [LOCAL ] DAN: Primary data saved (49407 records). [2025-10-20T18:14:14.379221] [INFO ] [MAIN ] DAN: Full history saved. [2025-10-20T18:14:14.379540] [INFO ] [MAIN ] Update Progress: 586/2367 symbols completed. DAN took 29.8s. Estimated remaining: 3h 45m 25s. Rate: 7.9 symbols/min [2025-10-20T18:14:14.552986] [INFO ] [MAIN ] DB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:14.605157] [DEBUG] [LOCAL ] DB: Loaded 53281 existing records from primary storage. [2025-10-20T18:14:14.605595] [INFO ] [MAIN ] DB: Loaded 53281 existing records. [2025-10-20T18:14:14.656499] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-07. [2025-10-20T18:14:14.793181] [WARN ] [MAIN ] DB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:14:14.793389] [INFO ] [MAIN ] DB: Wiping existing primary data before full refresh. [2025-10-20T18:14:14.793435] [INFO ] [LOCAL ] DB: Starting deletion of primary data. [2025-10-20T18:14:14.793944] [DEBUG] [LOCAL ] DB: Deleted primary data file. [2025-10-20T18:14:14.794002] [INFO ] [MAIN ] DB: Starting full primary refresh. [2025-10-20T18:14:15.151713] [INFO ] [REMOTE ] DB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:15.451346] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-01. [2025-10-20T18:14:15.685046] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:14:16.193897] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T18:14:16.251334] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-08. [2025-10-20T18:14:17.051348] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-02. [2025-10-20T18:14:17.304219] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T18:14:17.526233] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:14:17.851367] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-09. [2025-10-20T18:14:18.596043] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:14:18.651419] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-05. [2025-10-20T18:14:19.420003] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T18:14:19.451379] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-10. [2025-10-20T18:14:20.102355] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T18:14:20.251403] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-03. [2025-10-20T18:14:21.051402] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-06. [2025-10-20T18:14:21.262828] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T18:14:21.270531] [INFO ] [REMOTE ] DAR: Full fetch complete (74980 records). [2025-10-20T18:14:21.270729] [DEBUG] [VALIDATOR ] DAR: Validating primary data. [2025-10-20T18:14:21.691028] [DEBUG] [VALIDATOR ] DAR: All expected trading days are present. [2025-10-20T18:14:21.691237] [DEBUG] [VALIDATOR ] DAR: Primary data validation passed. [2025-10-20T18:14:21.692189] [INFO ] [MAIN ] DAR: Fetched primary data is valid. [2025-10-20T18:14:21.692343] [DEBUG] [LOCAL ] DAR: Saving primary data. [2025-10-20T18:14:21.693860] [DEBUG] [LOCAL ] DAR: Trimmed 74980 records to 51998 (keeping 200 days of data) [2025-10-20T18:14:21.859108] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-02. [2025-10-20T18:14:22.019056] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T18:14:22.041327] [DEBUG] [LOCAL ] DAR: Primary data saved (51998 records). [2025-10-20T18:14:22.041443] [INFO ] [MAIN ] DAR: Full history saved. [2025-10-20T18:14:22.041514] [INFO ] [MAIN ] Update Progress: 587/2367 symbols completed. DAR took 29.9s. Estimated remaining: 3h 45m 17s. Rate: 7.9 symbols/min [2025-10-20T18:14:22.216223] [INFO ] [MAIN ] DBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:22.216452] [DEBUG] [LOCAL ] DBD: No existing data file found in primary storage. [2025-10-20T18:14:22.216554] [WARN ] [MAIN ] DBD: No existing data found, performing full refresh instead. [2025-10-20T18:14:22.216815] [INFO ] [MAIN ] DBD: Wiping existing primary data before full refresh. [2025-10-20T18:14:22.216883] [INFO ] [LOCAL ] DBD: Starting deletion of primary data. [2025-10-20T18:14:22.216996] [DEBUG] [LOCAL ] DBD: No primary data file found to delete. [2025-10-20T18:14:22.217133] [INFO ] [MAIN ] DBD: Starting full primary refresh. [2025-10-20T18:14:22.331177] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T18:14:22.600007] [INFO ] [REMOTE ] DBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:22.651357] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-01. [2025-10-20T18:14:23.424299] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:14:23.451411] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-04. [2025-10-20T18:14:23.629518] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T18:14:24.251491] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-03. [2025-10-20T18:14:25.051363] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-07. [2025-10-20T18:14:25.124403] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:14:25.851389] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-02. [2025-10-20T18:14:26.341950] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T18:14:26.418105] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T18:14:26.651429] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-04. [2025-10-20T18:14:26.832586] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:14:27.451382] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-03. [2025-10-20T18:14:28.109943] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T18:14:28.251437] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-08. [2025-10-20T18:14:28.453812] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:14:29.051469] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-04. [2025-10-20T18:14:29.841334] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:14:29.854275] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-05. [2025-10-20T18:14:30.134266] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T18:14:30.651378] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-05. [2025-10-20T18:14:31.451393] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-09. [2025-10-20T18:14:31.709602] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T18:14:32.251353] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-06. [2025-10-20T18:14:32.541603] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:14:33.051440] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-06. [2025-10-20T18:14:33.343749] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T18:14:33.655129] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:14:33.851367] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-05. [2025-10-20T18:14:34.651391] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-10. [2025-10-20T18:14:34.842906] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T18:14:34.903135] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T18:14:35.451385] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-07. [2025-10-20T18:14:36.251381] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-07. [2025-10-20T18:14:36.548803] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T18:14:36.558031] [INFO ] [REMOTE ] DAVA: Full fetch complete (57179 records). [2025-10-20T18:14:36.558184] [DEBUG] [VALIDATOR ] DAVA: Validating primary data. [2025-10-20T18:14:37.035438] [DEBUG] [VALIDATOR ] DAVA: All expected trading days are present. [2025-10-20T18:14:37.035528] [DEBUG] [VALIDATOR ] DAVA: Primary data validation passed. [2025-10-20T18:14:37.036477] [INFO ] [MAIN ] DAVA: Fetched primary data is valid. [2025-10-20T18:14:37.036541] [DEBUG] [LOCAL ] DAVA: Saving primary data. [2025-10-20T18:14:37.037944] [DEBUG] [LOCAL ] DAVA: Trimmed 57179 records to 41996 (keeping 200 days of data) [2025-10-20T18:14:37.087202] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-06. [2025-10-20T18:14:37.283178] [DEBUG] [LOCAL ] DAVA: Primary data saved (41996 records). [2025-10-20T18:14:37.283309] [INFO ] [MAIN ] DAVA: Full history saved. [2025-10-20T18:14:37.283385] [INFO ] [MAIN ] Update Progress: 588/2367 symbols completed. DAVA took 43.5s. Estimated remaining: 3h 45m 33s. Rate: 7.9 symbols/min [2025-10-20T18:14:37.458991] [INFO ] [MAIN ] DBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:37.492011] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:14:37.536633] [DEBUG] [LOCAL ] DBI: Loaded 44390 existing records from primary storage. [2025-10-20T18:14:37.536957] [INFO ] [MAIN ] DBI: Loaded 44390 existing records. [2025-10-20T18:14:37.716192] [WARN ] [MAIN ] DBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:14:37.716444] [INFO ] [MAIN ] DBI: Wiping existing primary data before full refresh. [2025-10-20T18:14:37.716517] [INFO ] [LOCAL ] DBI: Starting deletion of primary data. [2025-10-20T18:14:37.716910] [DEBUG] [LOCAL ] DBI: Deleted primary data file. [2025-10-20T18:14:37.716967] [INFO ] [MAIN ] DBI: Starting full primary refresh. [2025-10-20T18:14:37.855248] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-08. [2025-10-20T18:14:38.115113] [INFO ] [REMOTE ] DBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:38.117316] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:14:38.142272] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:14:38.651361] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-07. [2025-10-20T18:14:39.451432] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-01. [2025-10-20T18:14:39.735403] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T18:14:39.981402] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T18:14:40.251411] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-08. [2025-10-20T18:14:40.731272] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T18:14:41.051356] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-02. [2025-10-20T18:14:41.384515] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T18:14:41.851463] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-09. [2025-10-20T18:14:42.631157] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:14:42.651347] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-03. [2025-10-20T18:14:42.683550] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:14:43.451423] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-09. [2025-10-20T18:14:44.251899] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-08. [2025-10-20T18:14:44.251757] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T18:14:45.051397] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-10. [2025-10-20T18:14:45.325670] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T18:14:45.851411] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-04. [2025-10-20T18:14:46.077778] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:14:46.081763] [INFO ] [REMOTE ] DBD: Full fetch complete (31685 records). [2025-10-20T18:14:46.081876] [DEBUG] [VALIDATOR ] DBD: Validating primary data. [2025-10-20T18:14:46.082436] [FATAL] [VALIDATOR ] DBD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:14:46.082523] [FATAL] [MAIN ] DBD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:14:46.082565] [WARN ] [MAIN ] DBD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:14:46.082627] [INFO ] [LOCAL ] DBD: Starting deletion of primary data. [2025-10-20T18:14:46.082734] [DEBUG] [LOCAL ] DBD: No primary data file found to delete. [2025-10-20T18:14:46.082776] [INFO ] [LOCAL ] DBD: Starting deletion of derivative data. [2025-10-20T18:14:46.082816] [DEBUG] [LOCAL ] DBD: No derivative data file found to delete. [2025-10-20T18:14:46.082870] [INFO ] [MAIN ] Update Progress: 589/2367 symbols completed. DBD took 24.0s. Estimated remaining: 3h 45m 29s. Rate: 7.9 symbols/min [2025-10-20T18:14:46.254282] [INFO ] [MAIN ] DBL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:46.254476] [DEBUG] [LOCAL ] DBL: No existing data file found in primary storage. [2025-10-20T18:14:46.254547] [WARN ] [MAIN ] DBL: No existing data found, performing full refresh instead. [2025-10-20T18:14:46.254823] [INFO ] [MAIN ] DBL: Wiping existing primary data before full refresh. [2025-10-20T18:14:46.254889] [INFO ] [LOCAL ] DBL: Starting deletion of primary data. [2025-10-20T18:14:46.254977] [DEBUG] [LOCAL ] DBL: No primary data file found to delete. [2025-10-20T18:14:46.255027] [INFO ] [MAIN ] DBL: Starting full primary refresh. [2025-10-20T18:14:46.394257] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T18:14:46.656574] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-10. [2025-10-20T18:14:46.660922] [INFO ] [REMOTE ] DBL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:47.315992] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:14:47.451400] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-05. [2025-10-20T18:14:48.236876] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T18:14:48.244517] [INFO ] [REMOTE ] DB: Full fetch complete (75563 records). [2025-10-20T18:14:48.244630] [DEBUG] [VALIDATOR ] DB: Validating primary data. [2025-10-20T18:14:48.256593] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-01. [2025-10-20T18:14:48.659513] [DEBUG] [VALIDATOR ] DB: All expected trading days are present. [2025-10-20T18:14:48.659582] [DEBUG] [VALIDATOR ] DB: Primary data validation passed. [2025-10-20T18:14:48.660442] [INFO ] [MAIN ] DB: Fetched primary data is valid. [2025-10-20T18:14:48.660505] [DEBUG] [LOCAL ] DB: Saving primary data. [2025-10-20T18:14:48.661924] [DEBUG] [LOCAL ] DB: Trimmed 75563 records to 52484 (keeping 200 days of data) [2025-10-20T18:14:48.975272] [DEBUG] [LOCAL ] DB: Primary data saved (52484 records). [2025-10-20T18:14:48.975377] [INFO ] [MAIN ] DB: Full history saved. [2025-10-20T18:14:48.975674] [INFO ] [MAIN ] Update Progress: 590/2367 symbols completed. DB took 34.6s. Estimated remaining: 3h 45m 7s. Rate: 7.9 symbols/min [2025-10-20T18:14:49.067488] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-09. [2025-10-20T18:14:49.115460] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T18:14:49.116251] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T18:14:49.196801] [INFO ] [MAIN ] DBRG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:49.244838] [DEBUG] [LOCAL ] DBRG: Loaded 50682 existing records from primary storage. [2025-10-20T18:14:49.245123] [INFO ] [MAIN ] DBRG: Loaded 50682 existing records. [2025-10-20T18:14:49.423119] [WARN ] [MAIN ] DBRG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:14:49.423348] [INFO ] [MAIN ] DBRG: Wiping existing primary data before full refresh. [2025-10-20T18:14:49.423401] [INFO ] [LOCAL ] DBRG: Starting deletion of primary data. [2025-10-20T18:14:49.423612] [DEBUG] [LOCAL ] DBRG: Deleted primary data file. [2025-10-20T18:14:49.423714] [INFO ] [MAIN ] DBRG: Starting full primary refresh. [2025-10-20T18:14:49.773521] [INFO ] [REMOTE ] DBRG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:49.851346] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-01. [2025-10-20T18:14:50.651359] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-02. [2025-10-20T18:14:50.745356] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:14:51.413684] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T18:14:51.451329] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-03. [2025-10-20T18:14:51.549432] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:14:52.217489] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T18:14:52.251401] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-02. [2025-10-20T18:14:53.051401] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-06. [2025-10-20T18:14:53.815977] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:14:53.851339] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-10. [2025-10-20T18:14:54.651375] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-03. [2025-10-20T18:14:54.766065] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T18:14:55.451423] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-07. [2025-10-20T18:14:55.495075] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T18:14:55.501922] [INFO ] [REMOTE ] DAY: Full fetch complete (71787 records). [2025-10-20T18:14:55.502040] [DEBUG] [VALIDATOR ] DAY: Validating primary data. [2025-10-20T18:14:55.896544] [DEBUG] [VALIDATOR ] DAY: All expected trading days are present. [2025-10-20T18:14:55.896627] [DEBUG] [VALIDATOR ] DAY: Primary data validation passed. [2025-10-20T18:14:55.897458] [INFO ] [MAIN ] DAY: Fetched primary data is valid. [2025-10-20T18:14:55.897511] [DEBUG] [LOCAL ] DAY: Saving primary data. [2025-10-20T18:14:55.898922] [DEBUG] [LOCAL ] DAY: Trimmed 71787 records to 49167 (keeping 200 days of data) [2025-10-20T18:14:56.183777] [DEBUG] [LOCAL ] DAY: Primary data saved (49167 records). [2025-10-20T18:14:56.183904] [INFO ] [MAIN ] DAY: Full history saved. [2025-10-20T18:14:56.183972] [INFO ] [MAIN ] Update Progress: 591/2367 symbols completed. DAY took 47.9s. Estimated remaining: 3h 44m 58s. Rate: 7.9 symbols/min [2025-10-20T18:14:56.255837] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-04. [2025-10-20T18:14:56.365898] [INFO ] [MAIN ] DCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:56.396947] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:14:56.436858] [DEBUG] [LOCAL ] DCI: Loaded 37560 existing records from primary storage. [2025-10-20T18:14:56.437157] [INFO ] [MAIN ] DCI: Loaded 37560 existing records. [2025-10-20T18:14:56.615612] [WARN ] [MAIN ] DCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:14:56.615853] [INFO ] [MAIN ] DCI: Wiping existing primary data before full refresh. [2025-10-20T18:14:56.615905] [INFO ] [LOCAL ] DCI: Starting deletion of primary data. [2025-10-20T18:14:56.616134] [DEBUG] [LOCAL ] DCI: Deleted primary data file. [2025-10-20T18:14:56.616189] [INFO ] [MAIN ] DCI: Starting full primary refresh. [2025-10-20T18:14:57.014533] [INFO ] [REMOTE ] DCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:57.015381] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T18:14:57.046265] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T18:14:57.055715] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-01. [2025-10-20T18:14:57.851366] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-08. [2025-10-20T18:14:58.491873] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:14:58.651377] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-02. [2025-10-20T18:14:59.311758] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:14:59.451375] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-05. [2025-10-20T18:14:59.906691] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T18:15:00.045102] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T18:15:00.251422] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-04. [2025-10-20T18:15:01.051342] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-03. [2025-10-20T18:15:01.851417] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-09. [2025-10-20T18:15:02.268707] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T18:15:02.389996] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:15:02.651405] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-05. [2025-10-20T18:15:03.451778] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-04. [2025-10-20T18:15:03.457049] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:15:04.251366] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-06. [2025-10-20T18:15:04.481090] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T18:15:05.051349] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-10. [2025-10-20T18:15:05.078735] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:15:05.124881] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T18:15:05.851415] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-05. [2025-10-20T18:15:06.307446] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:15:06.316343] [INFO ] [REMOTE ] DBI: Full fetch complete (64952 records). [2025-10-20T18:15:06.316557] [DEBUG] [VALIDATOR ] DBI: Validating primary data. [2025-10-20T18:15:06.656514] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-06. [2025-10-20T18:15:06.814424] [DEBUG] [VALIDATOR ] DBI: All expected trading days are present. [2025-10-20T18:15:06.814512] [DEBUG] [VALIDATOR ] DBI: Primary data validation passed. [2025-10-20T18:15:06.815319] [INFO ] [MAIN ] DBI: Fetched primary data is valid. [2025-10-20T18:15:06.815386] [DEBUG] [LOCAL ] DBI: Saving primary data. [2025-10-20T18:15:06.816784] [DEBUG] [LOCAL ] DBI: Trimmed 64952 records to 43623 (keeping 200 days of data) [2025-10-20T18:15:07.069745] [DEBUG] [LOCAL ] DBI: Primary data saved (43623 records). [2025-10-20T18:15:07.069874] [INFO ] [MAIN ] DBI: Full history saved. [2025-10-20T18:15:07.069971] [INFO ] [MAIN ] Update Progress: 592/2367 symbols completed. DBI took 29.8s. Estimated remaining: 3h 45m 0s. Rate: 7.9 symbols/min [2025-10-20T18:15:07.245408] [INFO ] [MAIN ] DCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:07.260947] [DEBUG] [LOCAL ] DCO: Loaded 14832 existing records from primary storage. [2025-10-20T18:15:07.261165] [INFO ] [MAIN ] DCO: Loaded 14832 existing records. [2025-10-20T18:15:07.402298] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:15:07.466342] [WARN ] [MAIN ] DCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:07.466940] [INFO ] [MAIN ] DCO: Wiping existing primary data before full refresh. [2025-10-20T18:15:07.461465] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-07. [2025-10-20T18:15:07.467765] [INFO ] [LOCAL ] DCO: Starting deletion of primary data. [2025-10-20T18:15:07.467945] [DEBUG] [LOCAL ] DCO: Deleted primary data file. [2025-10-20T18:15:07.467997] [INFO ] [MAIN ] DCO: Starting full primary refresh. [2025-10-20T18:15:07.860631] [INFO ] [REMOTE ] DCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:07.918458] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T18:15:08.251358] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-07. [2025-10-20T18:15:08.393978] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T18:15:09.051377] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-01. [2025-10-20T18:15:09.733918] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T18:15:09.851377] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-02. [2025-10-20T18:15:10.251013] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T18:15:10.346435] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T18:15:10.651389] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-03. [2025-10-20T18:15:11.451360] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-06. [2025-10-20T18:15:11.481008] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:15:12.251364] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-08. [2025-10-20T18:15:12.624140] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T18:15:13.051398] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-04. [2025-10-20T18:15:13.746583] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T18:15:13.851357] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-05. [2025-10-20T18:15:13.965986] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T18:15:14.651382] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-09. [2025-10-20T18:15:14.662500] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T18:15:15.451364] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-06. [2025-10-20T18:15:16.251372] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-08. [2025-10-20T18:15:16.393914] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T18:15:16.507281] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:15:16.978558] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T18:15:17.051502] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-07. [2025-10-20T18:15:17.851386] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-10. [2025-10-20T18:15:18.458022] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:15:18.651437] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-07. [2025-10-20T18:15:19.451426] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-09. [2025-10-20T18:15:19.506807] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T18:15:19.751186] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T18:15:19.759305] [INFO ] [REMOTE ] DBRG: Full fetch complete (72495 records). [2025-10-20T18:15:19.759470] [DEBUG] [VALIDATOR ] DBRG: Validating primary data. [2025-10-20T18:15:20.218414] [DEBUG] [VALIDATOR ] DBRG: All expected trading days are present. [2025-10-20T18:15:20.218508] [DEBUG] [VALIDATOR ] DBRG: Primary data validation passed. [2025-10-20T18:15:20.219493] [INFO ] [MAIN ] DBRG: Fetched primary data is valid. [2025-10-20T18:15:20.219577] [DEBUG] [LOCAL ] DBRG: Saving primary data. [2025-10-20T18:15:20.221242] [DEBUG] [LOCAL ] DBRG: Trimmed 72495 records to 49985 (keeping 200 days of data) [2025-10-20T18:15:20.295500] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-08. [2025-10-20T18:15:20.349900] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T18:15:20.563378] [DEBUG] [LOCAL ] DBRG: Primary data saved (49985 records). [2025-10-20T18:15:20.563529] [INFO ] [MAIN ] DBRG: Full history saved. [2025-10-20T18:15:20.563890] [INFO ] [MAIN ] Update Progress: 593/2367 symbols completed. DBRG took 31.6s. Estimated remaining: 3h 45m 10s. Rate: 7.9 symbols/min [2025-10-20T18:15:20.733680] [INFO ] [MAIN ] DD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:20.785967] [DEBUG] [LOCAL ] DD: Loaded 53704 existing records from primary storage. [2025-10-20T18:15:20.786296] [INFO ] [MAIN ] DD: Loaded 53704 existing records. [2025-10-20T18:15:20.973709] [WARN ] [MAIN ] DD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:20.973942] [INFO ] [MAIN ] DD: Wiping existing primary data before full refresh. [2025-10-20T18:15:20.974011] [INFO ] [LOCAL ] DD: Starting deletion of primary data. [2025-10-20T18:15:20.974383] [DEBUG] [LOCAL ] DD: Deleted primary data file. [2025-10-20T18:15:20.974452] [INFO ] [MAIN ] DD: Starting full primary refresh. [2025-10-20T18:15:21.056473] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-10. [2025-10-20T18:15:21.329268] [INFO ] [REMOTE ] DD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:21.668527] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T18:15:21.670213] [INFO ] [REMOTE ] DBL: Full fetch complete (12640 records). [2025-10-20T18:15:21.670287] [DEBUG] [VALIDATOR ] DBL: Validating primary data. [2025-10-20T18:15:21.670703] [FATAL] [VALIDATOR ] DBL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:15:21.670797] [FATAL] [MAIN ] DBL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:15:21.670876] [WARN ] [MAIN ] DBL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:15:21.670972] [INFO ] [LOCAL ] DBL: Starting deletion of primary data. [2025-10-20T18:15:21.671052] [DEBUG] [LOCAL ] DBL: No primary data file found to delete. [2025-10-20T18:15:21.671089] [INFO ] [LOCAL ] DBL: Starting deletion of derivative data. [2025-10-20T18:15:21.671134] [DEBUG] [LOCAL ] DBL: No derivative data file found to delete. [2025-10-20T18:15:21.671189] [INFO ] [MAIN ] Update Progress: 594/2367 symbols completed. DBL took 35.6s. Estimated remaining: 3h 44m 43s. Rate: 7.9 symbols/min [2025-10-20T18:15:21.852710] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-01. [2025-10-20T18:15:21.880032] [INFO ] [MAIN ] DDD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:21.882500] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:15:21.937833] [DEBUG] [LOCAL ] DDD: Loaded 51691 existing records from primary storage. [2025-10-20T18:15:21.938156] [INFO ] [MAIN ] DDD: Loaded 51691 existing records. [2025-10-20T18:15:22.110761] [WARN ] [MAIN ] DDD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:22.110963] [INFO ] [MAIN ] DDD: Wiping existing primary data before full refresh. [2025-10-20T18:15:22.111010] [INFO ] [LOCAL ] DDD: Starting deletion of primary data. [2025-10-20T18:15:22.111184] [DEBUG] [LOCAL ] DDD: Deleted primary data file. [2025-10-20T18:15:22.111221] [INFO ] [MAIN ] DDD: Starting full primary refresh. [2025-10-20T18:15:22.458303] [INFO ] [REMOTE ] DDD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:22.651366] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-08. [2025-10-20T18:15:23.451425] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-01. [2025-10-20T18:15:23.746333] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T18:15:23.781023] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T18:15:24.251397] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-09. [2025-10-20T18:15:25.051393] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-09. [2025-10-20T18:15:25.342417] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:15:25.851392] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-02. [2025-10-20T18:15:25.879001] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:15:25.995160] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T18:15:26.651370] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-10. [2025-10-20T18:15:27.451395] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-10. [2025-10-20T18:15:27.660668] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T18:15:28.040156] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T18:15:28.045339] [INFO ] [REMOTE ] DCI: Full fetch complete (53421 records). [2025-10-20T18:15:28.045489] [DEBUG] [VALIDATOR ] DCI: Validating primary data. [2025-10-20T18:15:28.046143] [FATAL] [VALIDATOR ] DCI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:15:28.046236] [FATAL] [MAIN ] DCI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:15:28.046288] [WARN ] [MAIN ] DCI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:15:28.046366] [INFO ] [LOCAL ] DCI: Starting deletion of primary data. [2025-10-20T18:15:28.046443] [DEBUG] [LOCAL ] DCI: No primary data file found to delete. [2025-10-20T18:15:28.046490] [INFO ] [LOCAL ] DCI: Starting deletion of derivative data. [2025-10-20T18:15:28.046704] [DEBUG] [LOCAL ] DCI: Deleted derivative data file. [2025-10-20T18:15:28.046812] [INFO ] [MAIN ] Update Progress: 595/2367 symbols completed. DCI took 31.9s. Estimated remaining: 3h 44m 32s. Rate: 7.9 symbols/min [2025-10-20T18:15:28.232099] [INFO ] [MAIN ] DDL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:28.251827] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-02. [2025-10-20T18:15:28.269765] [DEBUG] [LOCAL ] DDL: Loaded 37618 existing records from primary storage. [2025-10-20T18:15:28.270121] [INFO ] [MAIN ] DDL: Loaded 37618 existing records. [2025-10-20T18:15:28.464086] [WARN ] [MAIN ] DDL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:28.464449] [INFO ] [MAIN ] DDL: Wiping existing primary data before full refresh. [2025-10-20T18:15:28.464559] [INFO ] [LOCAL ] DDL: Starting deletion of primary data. [2025-10-20T18:15:28.472338] [DEBUG] [LOCAL ] DDL: Deleted primary data file. [2025-10-20T18:15:28.472480] [INFO ] [MAIN ] DDL: Starting full primary refresh. [2025-10-20T18:15:28.493783] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T18:15:28.561884] [INFO ] [REMOTE ] DCO: Full fetch complete (19547 records). [2025-10-20T18:15:28.589487] [DEBUG] [VALIDATOR ] DCO: Validating primary data. [2025-10-20T18:15:29.067856] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-03. [2025-10-20T18:15:29.158037] [INFO ] [REMOTE ] DDL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:29.257940] [DEBUG] [VALIDATOR ] DCO: All expected trading days are present. [2025-10-20T18:15:29.258034] [DEBUG] [VALIDATOR ] DCO: Primary data validation passed. [2025-10-20T18:15:29.258454] [INFO ] [MAIN ] DCO: Fetched primary data is valid. [2025-10-20T18:15:29.258494] [DEBUG] [LOCAL ] DCO: Saving primary data. [2025-10-20T18:15:29.259352] [DEBUG] [LOCAL ] DCO: Trimmed 19547 records to 14765 (keeping 200 days of data) [2025-10-20T18:15:29.346300] [DEBUG] [LOCAL ] DCO: Primary data saved (14765 records). [2025-10-20T18:15:29.346451] [INFO ] [MAIN ] DCO: Full history saved. [2025-10-20T18:15:29.346530] [INFO ] [MAIN ] Update Progress: 596/2367 symbols completed. DCO took 22.3s. Estimated remaining: 3h 44m 6s. Rate: 7.9 symbols/min [2025-10-20T18:15:29.527054] [INFO ] [MAIN ] DDS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:29.541622] [DEBUG] [LOCAL ] DDS: Loaded 12846 existing records from primary storage. [2025-10-20T18:15:29.541874] [INFO ] [MAIN ] DDS: Loaded 12846 existing records. [2025-10-20T18:15:29.732594] [WARN ] [MAIN ] DDS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:29.733293] [INFO ] [MAIN ] DDS: Wiping existing primary data before full refresh. [2025-10-20T18:15:29.733347] [INFO ] [LOCAL ] DDS: Starting deletion of primary data. [2025-10-20T18:15:29.733493] [DEBUG] [LOCAL ] DDS: Deleted primary data file. [2025-10-20T18:15:29.733530] [INFO ] [MAIN ] DDS: Starting full primary refresh. [2025-10-20T18:15:29.854622] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-01. [2025-10-20T18:15:30.136506] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T18:15:30.141829] [INFO ] [REMOTE ] DDS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:30.651388] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-03. [2025-10-20T18:15:30.955567] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:15:31.319084] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T18:15:31.451445] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-02. [2025-10-20T18:15:32.251348] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-04. [2025-10-20T18:15:32.376326] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T18:15:32.709490] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T18:15:33.051407] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-01. [2025-10-20T18:15:33.851384] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-03. [2025-10-20T18:15:34.632524] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-04 (2.38 seconds). [2025-10-20T18:15:34.651332] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-05. [2025-10-20T18:15:35.451375] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-04. [2025-10-20T18:15:36.063594] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-01 (3.01 seconds). [2025-10-20T18:15:36.251426] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-02. [2025-10-20T18:15:36.648853] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-03 (2.80 seconds). [2025-10-20T18:15:37.006542] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T18:15:37.051435] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-03. [2025-10-20T18:15:37.285378] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-05 (2.63 seconds). [2025-10-20T18:15:37.785015] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T18:15:37.858697] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-04. [2025-10-20T18:15:37.878271] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-04 (2.43 seconds). [2025-10-20T18:15:38.651372] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-05. [2025-10-20T18:15:39.451350] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-04. [2025-10-20T18:15:39.484929] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:15:40.149081] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T18:15:40.251339] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-06. [2025-10-20T18:15:40.299724] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T18:15:41.051399] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-05. [2025-10-20T18:15:41.851406] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-06. [2025-10-20T18:15:41.871259] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T18:15:41.987808] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T18:15:42.651365] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-06. [2025-10-20T18:15:43.309457] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T18:15:43.457555] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-07. [2025-10-20T18:15:43.478506] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T18:15:44.108794] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T18:15:44.251392] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-07. [2025-10-20T18:15:45.051376] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-08. [2025-10-20T18:15:45.157206] [DEBUG] [REMOTE ] DDS: Error fetching primary data for 2025-08: 'API call returned an empty response.' [2025-10-20T18:15:45.157464] [WARN ] [REMOTE ] DDS: No data returned for month 2025-08 [2025-10-20T18:15:45.157550] [FATAL] [REMOTE ] DDS: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:15:45.157629] [FATAL] [MAIN ] DDS: Failed to fetch primary data. [2025-10-20T18:15:45.157712] [WARN ] [MAIN ] DDS: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:15:45.157785] [INFO ] [LOCAL ] DDS: Starting deletion of primary data. [2025-10-20T18:15:45.157846] [DEBUG] [LOCAL ] DDS: No primary data file found to delete. [2025-10-20T18:15:45.157882] [INFO ] [LOCAL ] DDS: Starting deletion of derivative data. [2025-10-20T18:15:45.158027] [DEBUG] [LOCAL ] DDS: Deleted derivative data file. [2025-10-20T18:15:45.158107] [INFO ] [MAIN ] Update Progress: 597/2367 symbols completed. DDS took 15.8s. Estimated remaining: 3h 44m 23s. Rate: 7.9 symbols/min [2025-10-20T18:15:45.851372] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-05. [2025-10-20T18:15:46.289053] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:15:46.651386] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-08. [2025-10-20T18:15:47.451443] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-07. [2025-10-20T18:15:47.975423] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T18:15:48.251409] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-06. [2025-10-20T18:15:49.093433] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-08 (2.44 seconds). [2025-10-20T18:15:49.104836] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-09. [2025-10-20T18:15:49.744254] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T18:15:49.851364] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-07. [2025-10-20T18:15:50.381721] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-07 (2.93 seconds). [2025-10-20T18:15:50.651362] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-08. [2025-10-20T18:15:50.829450] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:15:51.078295] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T18:15:51.451436] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-10. [2025-10-20T18:15:52.262605] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-08. [2025-10-20T18:15:52.265218] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T18:15:52.274096] [INFO ] [REMOTE ] DD: Full fetch complete (76619 records). [2025-10-20T18:15:52.274255] [DEBUG] [VALIDATOR ] DD: Validating primary data. [2025-10-20T18:15:52.607192] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T18:15:52.758517] [DEBUG] [VALIDATOR ] DD: All expected trading days are present. [2025-10-20T18:15:52.758668] [DEBUG] [VALIDATOR ] DD: Primary data validation passed. [2025-10-20T18:15:52.759717] [INFO ] [MAIN ] DD: Fetched primary data is valid. [2025-10-20T18:15:52.759785] [DEBUG] [LOCAL ] DD: Saving primary data. [2025-10-20T18:15:52.761354] [DEBUG] [LOCAL ] DD: Trimmed 76619 records to 52934 (keeping 200 days of data) [2025-10-20T18:15:53.061988] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-09. [2025-10-20T18:15:53.112980] [DEBUG] [LOCAL ] DD: Primary data saved (52934 records). [2025-10-20T18:15:53.113135] [INFO ] [MAIN ] DD: Full history saved. [2025-10-20T18:15:53.113512] [INFO ] [MAIN ] Update Progress: 598/2367 symbols completed. DD took 32.5s. Estimated remaining: 3h 44m 16s. Rate: 7.9 symbols/min [2025-10-20T18:15:53.535850] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:15:53.851418] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-09. [2025-10-20T18:15:54.781400] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:15:54.792636] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-10. [2025-10-20T18:15:55.364626] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:15:55.451343] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-10. [2025-10-20T18:15:56.806553] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T18:15:56.814411] [INFO ] [REMOTE ] DDD: Full fetch complete (73946 records). [2025-10-20T18:15:56.814557] [DEBUG] [VALIDATOR ] DDD: Validating primary data. [2025-10-20T18:15:57.131192] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T18:15:57.177936] [INFO ] [REMOTE ] DDL: Full fetch complete (53607 records). [2025-10-20T18:15:57.193553] [DEBUG] [VALIDATOR ] DDL: Validating primary data. [2025-10-20T18:15:57.381392] [DEBUG] [VALIDATOR ] DDD: All expected trading days are present. [2025-10-20T18:15:57.384016] [DEBUG] [VALIDATOR ] DDD: Primary data validation passed. [2025-10-20T18:15:57.385201] [INFO ] [MAIN ] DDD: Fetched primary data is valid. [2025-10-20T18:15:57.395850] [DEBUG] [LOCAL ] DDD: Saving primary data. [2025-10-20T18:15:57.451921] [DEBUG] [LOCAL ] DDD: Trimmed 73946 records to 50939 (keeping 200 days of data) [2025-10-20T18:15:57.772605] [DEBUG] [VALIDATOR ] DDL: All expected trading days are present. [2025-10-20T18:15:57.774337] [DEBUG] [VALIDATOR ] DDL: Primary data validation passed. [2025-10-20T18:15:57.775702] [INFO ] [MAIN ] DDL: Fetched primary data is valid. [2025-10-20T18:15:57.775773] [DEBUG] [LOCAL ] DDL: Saving primary data. [2025-10-20T18:15:57.783049] [DEBUG] [LOCAL ] DDL: Trimmed 53607 records to 37211 (keeping 200 days of data) [2025-10-20T18:15:58.165867] [DEBUG] [LOCAL ] DDL: Primary data saved (37211 records). [2025-10-20T18:15:58.167735] [INFO ] [MAIN ] DDL: Full history saved. [2025-10-20T18:15:58.173486] [INFO ] [MAIN ] Update Progress: 599/2367 symbols completed. DDL took 30.1s. Estimated remaining: 3h 44m 1s. Rate: 7.9 symbols/min [2025-10-20T18:15:58.186967] [DEBUG] [LOCAL ] DDD: Primary data saved (50939 records). [2025-10-20T18:15:58.187151] [INFO ] [MAIN ] DDD: Full history saved. [2025-10-20T18:15:58.187306] [INFO ] [MAIN ] Update Progress: 600/2367 symbols completed. DDD took 36.5s. Estimated remaining: 3h 43m 31s. Rate: 7.9 symbols/min [2025-10-20T18:15:58.814283] [INFO ] [MAIN ] DE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:58.891750] [DEBUG] [LOCAL ] DE: No existing data file found in primary storage. [2025-10-20T18:15:58.861716] [INFO ] [MAIN ] DEA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:58.912330] [WARN ] [MAIN ] DE: No existing data found, performing full refresh instead. [2025-10-20T18:15:58.929965] [INFO ] [MAIN ] DDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:58.933747] [INFO ] [MAIN ] DEC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:58.930609] [INFO ] [MAIN ] DE: Wiping existing primary data before full refresh. [2025-10-20T18:15:58.934088] [INFO ] [LOCAL ] DE: Starting deletion of primary data. [2025-10-20T18:15:58.934767] [DEBUG] [LOCAL ] DE: No primary data file found to delete. [2025-10-20T18:15:58.935050] [INFO ] [MAIN ] DE: Starting full primary refresh. [2025-10-20T18:15:59.044397] [DEBUG] [LOCAL ] DDT: Loaded 2106 existing records from primary storage. [2025-10-20T18:15:59.057336] [INFO ] [MAIN ] DDT: Loaded 2106 existing records. [2025-10-20T18:15:59.286962] [DEBUG] [LOCAL ] DEC: Loaded 30778 existing records from primary storage. [2025-10-20T18:15:59.389162] [DEBUG] [LOCAL ] DEA: Loaded 41023 existing records from primary storage. [2025-10-20T18:15:59.364996] [INFO ] [MAIN ] DEC: Loaded 30778 existing records. [2025-10-20T18:15:59.415054] [INFO ] [MAIN ] DEA: Loaded 41023 existing records. [2025-10-20T18:15:59.497723] [WARN ] [MAIN ] DDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:59.621681] [INFO ] [MAIN ] DDT: Wiping existing primary data before full refresh. [2025-10-20T18:15:59.726230] [INFO ] [LOCAL ] DDT: Starting deletion of primary data. [2025-10-20T18:15:59.808380] [WARN ] [MAIN ] DEC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:59.813746] [DEBUG] [LOCAL ] DDT: Deleted primary data file. [2025-10-20T18:15:59.845817] [INFO ] [MAIN ] DEC: Wiping existing primary data before full refresh. [2025-10-20T18:15:59.853845] [INFO ] [REMOTE ] DE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:59.854016] [INFO ] [LOCAL ] DEC: Starting deletion of primary data. [2025-10-20T18:15:59.859483] [INFO ] [MAIN ] DDT: Starting full primary refresh. [2025-10-20T18:15:59.859407] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-01. [2025-10-20T18:15:59.901832] [DEBUG] [LOCAL ] DEC: Deleted primary data file. [2025-10-20T18:15:59.923393] [INFO ] [MAIN ] DEC: Starting full primary refresh. [2025-10-20T18:16:00.026243] [WARN ] [MAIN ] DEA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:00.092504] [INFO ] [MAIN ] DEA: Wiping existing primary data before full refresh. [2025-10-20T18:16:00.096320] [INFO ] [LOCAL ] DEA: Starting deletion of primary data. [2025-10-20T18:16:00.127044] [DEBUG] [LOCAL ] DEA: Deleted primary data file. [2025-10-20T18:16:00.136375] [INFO ] [MAIN ] DEA: Starting full primary refresh. [2025-10-20T18:16:00.815509] [INFO ] [REMOTE ] DDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:00.835693] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-01. [2025-10-20T18:16:00.963412] [INFO ] [REMOTE ] DEC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:00.992615] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-01. [2025-10-20T18:16:01.033956] [INFO ] [REMOTE ] DEA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:01.034635] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-01. [2025-10-20T18:16:01.637075] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T18:16:01.638156] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-02. [2025-10-20T18:16:01.936882] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-10-20T18:16:01.938086] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-03. [2025-10-20T18:16:01.974617] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-01 (2.12 seconds). [2025-10-20T18:16:01.984944] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-02. [2025-10-20T18:16:02.123996] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T18:16:02.129126] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-02. [2025-10-20T18:16:02.433951] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T18:16:02.467597] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:16:02.651408] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-04. [2025-10-20T18:16:03.257421] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:16:03.285387] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T18:16:03.451400] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-05. [2025-10-20T18:16:03.904973] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T18:16:03.994494] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T18:16:04.251391] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-03. [2025-10-20T18:16:05.051559] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-03. [2025-10-20T18:16:05.763536] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T18:16:05.851381] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-04. [2025-10-20T18:16:06.651373] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-02. [2025-10-20T18:16:06.981454] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T18:16:07.143626] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T18:16:07.451386] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-06. [2025-10-20T18:16:08.238705] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T18:16:08.251344] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-07. [2025-10-20T18:16:08.822096] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-02 (2.17 seconds). [2025-10-20T18:16:08.891018] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T18:16:09.051377] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-08. [2025-10-20T18:16:09.393966] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:16:09.851409] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-03. [2025-10-20T18:16:10.651388] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-04. [2025-10-20T18:16:11.451412] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-09. [2025-10-20T18:16:11.515422] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T18:16:11.978195] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T18:16:12.251399] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-10. [2025-10-20T18:16:12.488758] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T18:16:12.692241] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T18:16:12.693360] [INFO ] [REMOTE ] DDT: Full fetch complete (2816 records). [2025-10-20T18:16:12.693438] [DEBUG] [VALIDATOR ] DDT: Validating primary data. [2025-10-20T18:16:13.054227] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-04. [2025-10-20T18:16:13.060700] [DEBUG] [VALIDATOR ] DDT: All expected trading days are present. [2025-10-20T18:16:13.060811] [DEBUG] [VALIDATOR ] DDT: Primary data validation passed. [2025-10-20T18:16:13.060942] [INFO ] [MAIN ] DDT: Fetched primary data is valid. [2025-10-20T18:16:13.060984] [DEBUG] [LOCAL ] DDT: Saving primary data. [2025-10-20T18:16:13.061585] [DEBUG] [LOCAL ] DDT: Trimmed 2816 records to 1978 (keeping 200 days of data) [2025-10-20T18:16:13.074667] [DEBUG] [LOCAL ] DDT: Primary data saved (1978 records). [2025-10-20T18:16:13.074823] [INFO ] [MAIN ] DDT: Full history saved. [2025-10-20T18:16:13.074919] [INFO ] [MAIN ] Update Progress: 601/2367 symbols completed. DDT took 14.9s. Estimated remaining: 3h 43m 45s. Rate: 7.9 symbols/min [2025-10-20T18:16:13.246589] [INFO ] [MAIN ] DECK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:13.301355] [DEBUG] [LOCAL ] DECK: Loaded 54077 existing records from primary storage. [2025-10-20T18:16:13.301752] [INFO ] [MAIN ] DECK: Loaded 54077 existing records. [2025-10-20T18:16:13.479957] [WARN ] [MAIN ] DECK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:13.480175] [INFO ] [MAIN ] DECK: Wiping existing primary data before full refresh. [2025-10-20T18:16:13.480223] [INFO ] [LOCAL ] DECK: Starting deletion of primary data. [2025-10-20T18:16:13.480781] [DEBUG] [LOCAL ] DECK: Deleted primary data file. [2025-10-20T18:16:13.480845] [INFO ] [MAIN ] DECK: Starting full primary refresh. [2025-10-20T18:16:13.836768] [INFO ] [REMOTE ] DECK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:13.851338] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-01. [2025-10-20T18:16:14.651462] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-05. [2025-10-20T18:16:14.900737] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:16:15.205507] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T18:16:15.451475] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-05. [2025-10-20T18:16:15.924512] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:16:16.251503] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-06. [2025-10-20T18:16:17.051389] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-05. [2025-10-20T18:16:17.245578] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T18:16:17.303870] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:16:17.851405] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-06. [2025-10-20T18:16:18.558040] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T18:16:18.651454] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-06. [2025-10-20T18:16:19.451366] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-07. [2025-10-20T18:16:19.567089] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:16:19.876006] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T18:16:20.251468] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-07. [2025-10-20T18:16:20.740935] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T18:16:21.051472] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-07. [2025-10-20T18:16:21.746967] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T18:16:21.851465] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-08. [2025-10-20T18:16:22.618512] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T18:16:22.651367] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-02. [2025-10-20T18:16:23.012198] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T18:16:23.451475] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-09. [2025-10-20T18:16:24.251467] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-08. [2025-10-20T18:16:24.874272] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-02 (2.22 seconds). [2025-10-20T18:16:24.973697] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T18:16:25.051399] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-10. [2025-10-20T18:16:25.851436] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-03. [2025-10-20T18:16:25.969931] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:16:26.469803] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:16:26.476434] [INFO ] [REMOTE ] DEA: Full fetch complete (61600 records). [2025-10-20T18:16:26.476615] [DEBUG] [VALIDATOR ] DEA: Validating primary data. [2025-10-20T18:16:26.477761] [FATAL] [VALIDATOR ] DEA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:16:26.477869] [FATAL] [MAIN ] DEA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:16:26.477952] [WARN ] [MAIN ] DEA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:16:26.478044] [INFO ] [LOCAL ] DEA: Starting deletion of primary data. [2025-10-20T18:16:26.478141] [DEBUG] [LOCAL ] DEA: No primary data file found to delete. [2025-10-20T18:16:26.478200] [INFO ] [LOCAL ] DEA: Starting deletion of derivative data. [2025-10-20T18:16:26.478410] [DEBUG] [LOCAL ] DEA: Deleted derivative data file. [2025-10-20T18:16:26.478539] [INFO ] [MAIN ] Update Progress: 602/2367 symbols completed. DEA took 28.3s. Estimated remaining: 3h 43m 54s. Rate: 7.9 symbols/min [2025-10-20T18:16:26.658089] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-09. [2025-10-20T18:16:26.677158] [INFO ] [MAIN ] DEI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:26.736800] [DEBUG] [LOCAL ] DEI: Loaded 49027 existing records from primary storage. [2025-10-20T18:16:26.737461] [INFO ] [MAIN ] DEI: Loaded 49027 existing records. [2025-10-20T18:16:26.936490] [WARN ] [MAIN ] DEI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:26.936853] [INFO ] [MAIN ] DEI: Wiping existing primary data before full refresh. [2025-10-20T18:16:26.936913] [INFO ] [LOCAL ] DEI: Starting deletion of primary data. [2025-10-20T18:16:26.937165] [DEBUG] [LOCAL ] DEI: Deleted primary data file. [2025-10-20T18:16:26.937223] [INFO ] [MAIN ] DEI: Starting full primary refresh. [2025-10-20T18:16:27.315081] [INFO ] [REMOTE ] DEI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:27.451368] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-08. [2025-10-20T18:16:27.626723] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T18:16:28.256600] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-04. [2025-10-20T18:16:28.275702] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:16:28.677583] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T18:16:29.051481] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-10. [2025-10-20T18:16:29.851505] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-09. [2025-10-20T18:16:30.432957] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T18:16:30.651428] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-05. [2025-10-20T18:16:30.805859] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T18:16:30.814801] [INFO ] [REMOTE ] DE: Full fetch complete (69566 records). [2025-10-20T18:16:30.815175] [DEBUG] [VALIDATOR ] DE: Validating primary data. [2025-10-20T18:16:30.816356] [FATAL] [VALIDATOR ] DE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:16:30.816476] [FATAL] [MAIN ] DE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:16:30.816537] [WARN ] [MAIN ] DE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:16:30.816613] [INFO ] [LOCAL ] DE: Starting deletion of primary data. [2025-10-20T18:16:30.816724] [DEBUG] [LOCAL ] DE: No primary data file found to delete. [2025-10-20T18:16:30.816774] [INFO ] [LOCAL ] DE: Starting deletion of derivative data. [2025-10-20T18:16:30.816818] [DEBUG] [LOCAL ] DE: No derivative data file found to delete. [2025-10-20T18:16:30.816890] [INFO ] [MAIN ] Update Progress: 603/2367 symbols completed. DE took 32.6s. Estimated remaining: 3h 43m 37s. Rate: 7.9 symbols/min [2025-10-20T18:16:31.020077] [INFO ] [MAIN ] DELL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:31.023594] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:16:31.093139] [DEBUG] [LOCAL ] DELL: Loaded 54516 existing records from primary storage. [2025-10-20T18:16:31.093568] [INFO ] [MAIN ] DELL: Loaded 54516 existing records. [2025-10-20T18:16:31.282566] [WARN ] [MAIN ] DELL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:31.282928] [INFO ] [MAIN ] DELL: Wiping existing primary data before full refresh. [2025-10-20T18:16:31.282982] [INFO ] [LOCAL ] DELL: Starting deletion of primary data. [2025-10-20T18:16:31.283827] [DEBUG] [LOCAL ] DELL: Deleted primary data file. [2025-10-20T18:16:31.283887] [INFO ] [MAIN ] DELL: Starting full primary refresh. [2025-10-20T18:16:31.465546] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-10. [2025-10-20T18:16:31.684687] [INFO ] [REMOTE ] DELL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:32.251417] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-01. [2025-10-20T18:16:32.320824] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:16:32.845441] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T18:16:32.851027] [INFO ] [REMOTE ] DEC: Full fetch complete (44243 records). [2025-10-20T18:16:32.851177] [DEBUG] [VALIDATOR ] DEC: Validating primary data. [2025-10-20T18:16:32.852096] [FATAL] [VALIDATOR ] DEC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:16:32.852224] [FATAL] [MAIN ] DEC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:16:32.852296] [WARN ] [MAIN ] DEC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:16:32.852359] [INFO ] [LOCAL ] DEC: Starting deletion of primary data. [2025-10-20T18:16:32.852464] [DEBUG] [LOCAL ] DEC: No primary data file found to delete. [2025-10-20T18:16:32.852536] [INFO ] [LOCAL ] DEC: Starting deletion of derivative data. [2025-10-20T18:16:32.852739] [DEBUG] [LOCAL ] DEC: Deleted derivative data file. [2025-10-20T18:16:32.852893] [INFO ] [MAIN ] Update Progress: 604/2367 symbols completed. DEC took 34.7s. Estimated remaining: 3h 43m 13s. Rate: 7.9 symbols/min [2025-10-20T18:16:33.046495] [INFO ] [MAIN ] DEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:33.051426] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-06. [2025-10-20T18:16:33.102059] [DEBUG] [LOCAL ] DEO: Loaded 49969 existing records from primary storage. [2025-10-20T18:16:33.102507] [INFO ] [MAIN ] DEO: Loaded 49969 existing records. [2025-10-20T18:16:33.288308] [WARN ] [MAIN ] DEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:33.288616] [INFO ] [MAIN ] DEO: Wiping existing primary data before full refresh. [2025-10-20T18:16:33.288702] [INFO ] [LOCAL ] DEO: Starting deletion of primary data. [2025-10-20T18:16:33.289320] [DEBUG] [LOCAL ] DEO: Deleted primary data file. [2025-10-20T18:16:33.289377] [INFO ] [MAIN ] DEO: Starting full primary refresh. [2025-10-20T18:16:33.674395] [INFO ] [REMOTE ] DEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:33.851484] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-01. [2025-10-20T18:16:34.651410] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-01. [2025-10-20T18:16:34.767984] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-01 (2.52 seconds). [2025-10-20T18:16:34.817174] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:16:35.451357] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-07. [2025-10-20T18:16:35.689943] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:16:36.251485] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-02. [2025-10-20T18:16:36.615366] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T18:16:37.051503] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-02. [2025-10-20T18:16:37.558134] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T18:16:37.784899] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:16:37.851391] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-08. [2025-10-20T18:16:38.560207] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T18:16:38.651461] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-03. [2025-10-20T18:16:39.305274] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:16:39.451450] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-03. [2025-10-20T18:16:40.142718] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:16:40.251419] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-09. [2025-10-20T18:16:40.944187] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:16:41.051431] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-02. [2025-10-20T18:16:41.851391] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-04. [2025-10-20T18:16:42.043902] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T18:16:42.651372] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-10. [2025-10-20T18:16:42.983264] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T18:16:43.451350] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-03. [2025-10-20T18:16:43.709392] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T18:16:44.251386] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-04. [2025-10-20T18:16:44.842206] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-10 (2.19 seconds). [2025-10-20T18:16:44.849779] [INFO ] [REMOTE ] DECK: Full fetch complete (76733 records). [2025-10-20T18:16:44.849956] [DEBUG] [VALIDATOR ] DECK: Validating primary data. [2025-10-20T18:16:44.850813] [FATAL] [VALIDATOR ] DECK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:16:44.850905] [FATAL] [MAIN ] DECK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:16:44.850949] [WARN ] [MAIN ] DECK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:16:44.851009] [INFO ] [LOCAL ] DECK: Starting deletion of primary data. [2025-10-20T18:16:44.851106] [DEBUG] [LOCAL ] DECK: No primary data file found to delete. [2025-10-20T18:16:44.851162] [INFO ] [LOCAL ] DECK: Starting deletion of derivative data. [2025-10-20T18:16:44.851323] [DEBUG] [LOCAL ] DECK: Deleted derivative data file. [2025-10-20T18:16:44.851422] [INFO ] [MAIN ] Update Progress: 605/2367 symbols completed. DECK took 31.8s. Estimated remaining: 3h 43m 18s. Rate: 7.9 symbols/min [2025-10-20T18:16:45.021531] [INFO ] [MAIN ] DFH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:45.052007] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-05. [2025-10-20T18:16:45.052605] [DEBUG] [LOCAL ] DFH: Loaded 32567 existing records from primary storage. [2025-10-20T18:16:45.053003] [INFO ] [MAIN ] DFH: Loaded 32567 existing records. [2025-10-20T18:16:45.234893] [WARN ] [MAIN ] DFH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:45.235175] [INFO ] [MAIN ] DFH: Wiping existing primary data before full refresh. [2025-10-20T18:16:45.235227] [INFO ] [LOCAL ] DFH: Starting deletion of primary data. [2025-10-20T18:16:45.235612] [DEBUG] [LOCAL ] DFH: Deleted primary data file. [2025-10-20T18:16:45.235686] [INFO ] [MAIN ] DFH: Starting full primary refresh. [2025-10-20T18:16:45.610245] [INFO ] [REMOTE ] DFH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:45.719508] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-03 (2.27 seconds). [2025-10-20T18:16:45.851424] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-04. [2025-10-20T18:16:46.062858] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T18:16:46.651470] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-01. [2025-10-20T18:16:46.853324] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T18:16:47.451409] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-06. [2025-10-20T18:16:47.660502] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T18:16:48.095913] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:16:48.251393] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-05. [2025-10-20T18:16:48.900956] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T18:16:49.051403] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-02. [2025-10-20T18:16:49.851378] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-07. [2025-10-20T18:16:49.999622] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T18:16:50.220466] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T18:16:50.651375] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-06. [2025-10-20T18:16:51.451382] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-05. [2025-10-20T18:16:51.652796] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:16:52.251433] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-08. [2025-10-20T18:16:52.955704] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-06 (2.30 seconds). [2025-10-20T18:16:53.028042] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:16:53.051354] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-06. [2025-10-20T18:16:53.851375] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-07. [2025-10-20T18:16:54.065900] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:16:54.256840] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T18:16:54.651378] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-03. [2025-10-20T18:16:55.451380] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-09. [2025-10-20T18:16:56.117559] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:16:56.228384] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-07 (2.38 seconds). [2025-10-20T18:16:56.251334] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-08. [2025-10-20T18:16:57.051346] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-04. [2025-10-20T18:16:57.083604] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:16:57.851384] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-10. [2025-10-20T18:16:58.152276] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T18:16:58.556615] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-08 (2.31 seconds). [2025-10-20T18:16:58.651385] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-05. [2025-10-20T18:16:59.451380] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-07. [2025-10-20T18:16:59.771879] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T18:16:59.779715] [INFO ] [REMOTE ] DEO: Full fetch complete (71759 records). [2025-10-20T18:16:59.779861] [DEBUG] [VALIDATOR ] DEO: Validating primary data. [2025-10-20T18:17:00.121419] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T18:17:00.224966] [DEBUG] [VALIDATOR ] DEO: All expected trading days are present. [2025-10-20T18:17:00.225055] [DEBUG] [VALIDATOR ] DEO: Primary data validation passed. [2025-10-20T18:17:00.225879] [INFO ] [MAIN ] DEO: Fetched primary data is valid. [2025-10-20T18:17:00.225941] [DEBUG] [LOCAL ] DEO: Saving primary data. [2025-10-20T18:17:00.227294] [DEBUG] [LOCAL ] DEO: Trimmed 71759 records to 49256 (keeping 200 days of data) [2025-10-20T18:17:00.283416] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-06. [2025-10-20T18:17:00.534310] [DEBUG] [LOCAL ] DEO: Primary data saved (49256 records). [2025-10-20T18:17:00.534438] [INFO ] [MAIN ] DEO: Full history saved. [2025-10-20T18:17:00.534510] [INFO ] [MAIN ] Update Progress: 606/2367 symbols completed. DEO took 27.7s. Estimated remaining: 3h 43m 34s. Rate: 7.9 symbols/min [2025-10-20T18:17:00.706109] [INFO ] [MAIN ] DFIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:00.728293] [DEBUG] [LOCAL ] DFIN: Loaded 22974 existing records from primary storage. [2025-10-20T18:17:00.728551] [INFO ] [MAIN ] DFIN: Loaded 22974 existing records. [2025-10-20T18:17:00.906276] [WARN ] [MAIN ] DFIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:00.906463] [INFO ] [MAIN ] DFIN: Wiping existing primary data before full refresh. [2025-10-20T18:17:00.906511] [INFO ] [LOCAL ] DFIN: Starting deletion of primary data. [2025-10-20T18:17:00.906719] [DEBUG] [LOCAL ] DFIN: Deleted primary data file. [2025-10-20T18:17:00.906779] [INFO ] [MAIN ] DFIN: Starting full primary refresh. [2025-10-20T18:17:01.056506] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-09. [2025-10-20T18:17:01.291831] [INFO ] [REMOTE ] DFIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:01.319390] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:17:01.851419] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T18:17:01.851494] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-08. [2025-10-20T18:17:02.651409] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-07. [2025-10-20T18:17:03.224920] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T18:17:03.445604] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:17:03.455348] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-01. [2025-10-20T18:17:03.707990] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T18:17:04.251362] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-09. [2025-10-20T18:17:04.493296] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:17:05.051416] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-10. [2025-10-20T18:17:05.851380] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-08. [2025-10-20T18:17:05.914113] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:17:06.651467] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-10. [2025-10-20T18:17:07.300395] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:17:07.451363] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-09. [2025-10-20T18:17:07.635090] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-10 (2.58 seconds). [2025-10-20T18:17:07.643283] [INFO ] [REMOTE ] DELL: Full fetch complete (77915 records). [2025-10-20T18:17:07.643438] [DEBUG] [VALIDATOR ] DELL: Validating primary data. [2025-10-20T18:17:08.048999] [DEBUG] [VALIDATOR ] DELL: All expected trading days are present. [2025-10-20T18:17:08.049135] [DEBUG] [VALIDATOR ] DELL: Primary data validation passed. [2025-10-20T18:17:08.050277] [INFO ] [MAIN ] DELL: Fetched primary data is valid. [2025-10-20T18:17:08.050464] [DEBUG] [LOCAL ] DELL: Saving primary data. [2025-10-20T18:17:08.052088] [DEBUG] [LOCAL ] DELL: Trimmed 77915 records to 53736 (keeping 200 days of data) [2025-10-20T18:17:08.210786] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T18:17:08.220957] [INFO ] [REMOTE ] DEI: Full fetch complete (69906 records). [2025-10-20T18:17:08.226479] [DEBUG] [VALIDATOR ] DEI: Validating primary data. [2025-10-20T18:17:08.332844] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-02. [2025-10-20T18:17:08.838888] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T18:17:08.867405] [DEBUG] [LOCAL ] DELL: Primary data saved (53736 records). [2025-10-20T18:17:08.872970] [INFO ] [MAIN ] DELL: Full history saved. [2025-10-20T18:17:08.888305] [INFO ] [MAIN ] Update Progress: 607/2367 symbols completed. DELL took 38.1s. Estimated remaining: 3h 43m 29s. Rate: 7.9 symbols/min [2025-10-20T18:17:09.055759] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-10. [2025-10-20T18:17:09.056032] [DEBUG] [VALIDATOR ] DEI: All expected trading days are present. [2025-10-20T18:17:09.059615] [DEBUG] [VALIDATOR ] DEI: Primary data validation passed. [2025-10-20T18:17:09.066434] [INFO ] [MAIN ] DEI: Fetched primary data is valid. [2025-10-20T18:17:09.077300] [DEBUG] [LOCAL ] DEI: Saving primary data. [2025-10-20T18:17:09.135563] [INFO ] [MAIN ] DFP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:09.136236] [DEBUG] [LOCAL ] DFP: No existing data file found in primary storage. [2025-10-20T18:17:09.136404] [WARN ] [MAIN ] DFP: No existing data found, performing full refresh instead. [2025-10-20T18:17:09.136710] [INFO ] [MAIN ] DFP: Wiping existing primary data before full refresh. [2025-10-20T18:17:09.136936] [INFO ] [LOCAL ] DFP: Starting deletion of primary data. [2025-10-20T18:17:09.136998] [DEBUG] [LOCAL ] DFP: No primary data file found to delete. [2025-10-20T18:17:09.137033] [INFO ] [MAIN ] DFP: Starting full primary refresh. [2025-10-20T18:17:09.168690] [DEBUG] [LOCAL ] DEI: Trimmed 69906 records to 48313 (keeping 200 days of data) [2025-10-20T18:17:09.624947] [INFO ] [REMOTE ] DFP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:09.741828] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T18:17:09.821529] [DEBUG] [LOCAL ] DEI: Primary data saved (48313 records). [2025-10-20T18:17:09.821721] [INFO ] [MAIN ] DEI: Full history saved. [2025-10-20T18:17:09.821816] [INFO ] [MAIN ] Update Progress: 608/2367 symbols completed. DEI took 43.3s. Estimated remaining: 3h 43m 2s. Rate: 7.9 symbols/min [2025-10-20T18:17:09.852889] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-03. [2025-10-20T18:17:09.991838] [INFO ] [MAIN ] DG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:10.047300] [DEBUG] [LOCAL ] DG: Loaded 54049 existing records from primary storage. [2025-10-20T18:17:10.047701] [INFO ] [MAIN ] DG: Loaded 54049 existing records. [2025-10-20T18:17:10.226294] [WARN ] [MAIN ] DG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:10.226557] [INFO ] [MAIN ] DG: Wiping existing primary data before full refresh. [2025-10-20T18:17:10.226828] [INFO ] [LOCAL ] DG: Starting deletion of primary data. [2025-10-20T18:17:10.227694] [DEBUG] [LOCAL ] DG: Deleted primary data file. [2025-10-20T18:17:10.227750] [INFO ] [MAIN ] DG: Starting full primary refresh. [2025-10-20T18:17:10.340179] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:17:10.365565] [INFO ] [REMOTE ] DFH: Full fetch complete (47556 records). [2025-10-20T18:17:10.382751] [DEBUG] [VALIDATOR ] DFH: Validating primary data. [2025-10-20T18:17:10.419260] [FATAL] [VALIDATOR ] DFH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:17:10.486449] [FATAL] [MAIN ] DFH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:17:10.521143] [WARN ] [MAIN ] DFH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:17:10.523591] [INFO ] [LOCAL ] DFH: Starting deletion of primary data. [2025-10-20T18:17:10.534456] [DEBUG] [LOCAL ] DFH: No primary data file found to delete. [2025-10-20T18:17:10.544909] [INFO ] [LOCAL ] DFH: Starting deletion of derivative data. [2025-10-20T18:17:10.575457] [DEBUG] [LOCAL ] DFH: Deleted derivative data file. [2025-10-20T18:17:10.581679] [INFO ] [MAIN ] Update Progress: 609/2367 symbols completed. DFH took 25.7s. Estimated remaining: 3h 42m 34s. Rate: 7.9 symbols/min [2025-10-20T18:17:10.607817] [INFO ] [REMOTE ] DG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:10.658009] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-01. [2025-10-20T18:17:10.781584] [INFO ] [MAIN ] DGX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:10.833686] [DEBUG] [LOCAL ] DGX: Loaded 45306 existing records from primary storage. [2025-10-20T18:17:10.834349] [INFO ] [MAIN ] DGX: Loaded 45306 existing records. [2025-10-20T18:17:11.017752] [WARN ] [MAIN ] DGX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:11.017954] [INFO ] [MAIN ] DGX: Wiping existing primary data before full refresh. [2025-10-20T18:17:11.018029] [INFO ] [LOCAL ] DGX: Starting deletion of primary data. [2025-10-20T18:17:11.018340] [DEBUG] [LOCAL ] DGX: Deleted primary data file. [2025-10-20T18:17:11.018397] [INFO ] [MAIN ] DGX: Starting full primary refresh. [2025-10-20T18:17:11.339934] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:17:11.409904] [INFO ] [REMOTE ] DGX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:11.451342] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-04. [2025-10-20T18:17:11.602973] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:17:12.251461] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-01. [2025-10-20T18:17:12.632803] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T18:17:13.051397] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-02. [2025-10-20T18:17:13.418841] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:17:13.579337] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:17:13.851424] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-05. [2025-10-20T18:17:14.651423] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-02. [2025-10-20T18:17:15.211461] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T18:17:15.451485] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-06. [2025-10-20T18:17:16.251485] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-01. [2025-10-20T18:17:16.679997] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T18:17:16.717272] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-10-20T18:17:17.051443] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-03. [2025-10-20T18:17:17.851374] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-03. [2025-10-20T18:17:17.884835] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:17:18.110931] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T18:17:18.651453] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-04. [2025-10-20T18:17:19.203762] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T18:17:19.451404] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-02. [2025-10-20T18:17:19.593746] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T18:17:20.251454] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-05. [2025-10-20T18:17:20.778045] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T18:17:21.051418] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-04. [2025-10-20T18:17:21.202098] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:17:21.851391] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-07. [2025-10-20T18:17:22.651353] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-03. [2025-10-20T18:17:22.844498] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T18:17:22.950208] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:17:23.451418] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-05. [2025-10-20T18:17:24.251385] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-08. [2025-10-20T18:17:24.509744] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T18:17:25.051346] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-04. [2025-10-20T18:17:25.088606] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T18:17:25.284954] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T18:17:25.851415] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-06. [2025-10-20T18:17:26.554071] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T18:17:26.651358] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-07. [2025-10-20T18:17:26.926020] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T18:17:27.451410] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-06. [2025-10-20T18:17:27.467543] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T18:17:28.251372] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-09. [2025-10-20T18:17:29.051786] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-05. [2025-10-20T18:17:29.068604] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:17:29.227365] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T18:17:29.851396] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-07. [2025-10-20T18:17:30.651601] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-10. [2025-10-20T18:17:31.009879] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T18:17:31.442627] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T18:17:31.452209] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-06. [2025-10-20T18:17:31.781167] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T18:17:31.785305] [INFO ] [REMOTE ] DFIN: Full fetch complete (33166 records). [2025-10-20T18:17:31.785481] [DEBUG] [VALIDATOR ] DFIN: Validating primary data. [2025-10-20T18:17:31.786229] [FATAL] [VALIDATOR ] DFIN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:17:31.786322] [FATAL] [MAIN ] DFIN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:17:31.786365] [WARN ] [MAIN ] DFIN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:17:31.786402] [INFO ] [LOCAL ] DFIN: Starting deletion of primary data. [2025-10-20T18:17:31.786458] [DEBUG] [LOCAL ] DFIN: No primary data file found to delete. [2025-10-20T18:17:31.786493] [INFO ] [LOCAL ] DFIN: Starting deletion of derivative data. [2025-10-20T18:17:31.786664] [DEBUG] [LOCAL ] DFIN: Deleted derivative data file. [2025-10-20T18:17:31.786757] [INFO ] [MAIN ] Update Progress: 610/2367 symbols completed. DFIN took 31.3s. Estimated remaining: 3h 43m 6s. Rate: 7.9 symbols/min [2025-10-20T18:17:31.960985] [INFO ] [MAIN ] DHF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:31.982977] [DEBUG] [LOCAL ] DHF: Loaded 21251 existing records from primary storage. [2025-10-20T18:17:31.983301] [INFO ] [MAIN ] DHF: Loaded 21251 existing records. [2025-10-20T18:17:32.160814] [WARN ] [MAIN ] DHF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:32.160956] [INFO ] [MAIN ] DHF: Wiping existing primary data before full refresh. [2025-10-20T18:17:32.160998] [INFO ] [LOCAL ] DHF: Starting deletion of primary data. [2025-10-20T18:17:32.161146] [DEBUG] [LOCAL ] DHF: Deleted primary data file. [2025-10-20T18:17:32.161185] [INFO ] [MAIN ] DHF: Starting full primary refresh. [2025-10-20T18:17:32.257854] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-08. [2025-10-20T18:17:32.518154] [INFO ] [REMOTE ] DHF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:33.051357] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-08. [2025-10-20T18:17:33.142844] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T18:17:33.222303] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T18:17:33.851424] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-09. [2025-10-20T18:17:34.494131] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T18:17:34.651379] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-09. [2025-10-20T18:17:34.676630] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:17:35.451399] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-10. [2025-10-20T18:17:36.005545] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T18:17:36.007581] [INFO ] [REMOTE ] DFP: Full fetch complete (15719 records). [2025-10-20T18:17:36.007724] [DEBUG] [VALIDATOR ] DFP: Validating primary data. [2025-10-20T18:17:36.157004] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:17:36.256576] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-10. [2025-10-20T18:17:36.446583] [DEBUG] [VALIDATOR ] DFP: All expected trading days are present. [2025-10-20T18:17:36.446688] [DEBUG] [VALIDATOR ] DFP: Primary data validation passed. [2025-10-20T18:17:36.447065] [INFO ] [MAIN ] DFP: Fetched primary data is valid. [2025-10-20T18:17:36.447108] [DEBUG] [LOCAL ] DFP: Saving primary data. [2025-10-20T18:17:36.447607] [DEBUG] [LOCAL ] DFP: Trimmed 15719 records to 11121 (keeping 200 days of data) [2025-10-20T18:17:36.515918] [DEBUG] [LOCAL ] DFP: Primary data saved (11121 records). [2025-10-20T18:17:36.516050] [INFO ] [MAIN ] DFP: Full history saved. [2025-10-20T18:17:36.516119] [INFO ] [MAIN ] Update Progress: 611/2367 symbols completed. DFP took 27.6s. Estimated remaining: 3h 42m 50s. Rate: 7.9 symbols/min [2025-10-20T18:17:36.698265] [INFO ] [MAIN ] DHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:36.698459] [DEBUG] [LOCAL ] DHI: No existing data file found in primary storage. [2025-10-20T18:17:36.698497] [WARN ] [MAIN ] DHI: No existing data found, performing full refresh instead. [2025-10-20T18:17:36.698607] [INFO ] [MAIN ] DHI: Wiping existing primary data before full refresh. [2025-10-20T18:17:36.698674] [INFO ] [LOCAL ] DHI: Starting deletion of primary data. [2025-10-20T18:17:36.698739] [DEBUG] [LOCAL ] DHI: No primary data file found to delete. [2025-10-20T18:17:36.698773] [INFO ] [MAIN ] DHI: Starting full primary refresh. [2025-10-20T18:17:37.059734] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-07. [2025-10-20T18:17:37.064627] [INFO ] [REMOTE ] DHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:37.851347] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-01. [2025-10-20T18:17:37.950148] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T18:17:37.957130] [INFO ] [REMOTE ] DGX: Full fetch complete (65083 records). [2025-10-20T18:17:37.957342] [DEBUG] [VALIDATOR ] DGX: Validating primary data. [2025-10-20T18:17:37.958174] [FATAL] [VALIDATOR ] DGX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:17:37.958261] [FATAL] [MAIN ] DGX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:17:37.958303] [WARN ] [MAIN ] DGX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:17:37.958342] [INFO ] [LOCAL ] DGX: Starting deletion of primary data. [2025-10-20T18:17:37.958398] [DEBUG] [LOCAL ] DGX: No primary data file found to delete. [2025-10-20T18:17:37.958432] [INFO ] [LOCAL ] DGX: Starting deletion of derivative data. [2025-10-20T18:17:37.958583] [DEBUG] [LOCAL ] DGX: Deleted derivative data file. [2025-10-20T18:17:37.959404] [INFO ] [MAIN ] Update Progress: 612/2367 symbols completed. DGX took 27.4s. Estimated remaining: 3h 42m 25s. Rate: 7.9 symbols/min [2025-10-20T18:17:38.134697] [INFO ] [MAIN ] DHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:38.134865] [DEBUG] [LOCAL ] DHR: No existing data file found in primary storage. [2025-10-20T18:17:38.134905] [WARN ] [MAIN ] DHR: No existing data found, performing full refresh instead. [2025-10-20T18:17:38.135013] [INFO ] [MAIN ] DHR: Wiping existing primary data before full refresh. [2025-10-20T18:17:38.135056] [INFO ] [LOCAL ] DHR: Starting deletion of primary data. [2025-10-20T18:17:38.135100] [DEBUG] [LOCAL ] DHR: No primary data file found to delete. [2025-10-20T18:17:38.135134] [INFO ] [MAIN ] DHR: Starting full primary refresh. [2025-10-20T18:17:38.531526] [INFO ] [REMOTE ] DHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:38.596477] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T18:17:38.651384] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-08. [2025-10-20T18:17:39.451432] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-01. [2025-10-20T18:17:39.532462] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:17:40.251450] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-02. [2025-10-20T18:17:40.476094] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:17:40.518102] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T18:17:41.051407] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-01. [2025-10-20T18:17:41.851354] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-09. [2025-10-20T18:17:42.129134] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T18:17:42.651400] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-02. [2025-10-20T18:17:42.784896] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T18:17:43.451349] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-03. [2025-10-20T18:17:43.690131] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:17:43.776698] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T18:17:44.251348] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-03. [2025-10-20T18:17:44.973167] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:17:45.051353] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-02. [2025-10-20T18:17:45.096732] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T18:17:45.851454] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-10. [2025-10-20T18:17:46.651365] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-04. [2025-10-20T18:17:46.714023] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:17:46.751591] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T18:17:46.759836] [DEBUG] [REMOTE ] DHF: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T18:17:46.760985] [WARN ] [REMOTE ] DHF: No data returned for month 2025-04 [2025-10-20T18:17:46.761186] [INFO ] [REMOTE ] DG: Full fetch complete (77343 records). [2025-10-20T18:17:46.761263] [FATAL] [REMOTE ] DHF: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:17:46.761595] [DEBUG] [VALIDATOR ] DG: Validating primary data. [2025-10-20T18:17:46.761754] [FATAL] [MAIN ] DHF: Failed to fetch primary data. [2025-10-20T18:17:46.761938] [WARN ] [MAIN ] DHF: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:17:46.762184] [INFO ] [LOCAL ] DHF: Starting deletion of primary data. [2025-10-20T18:17:46.762335] [DEBUG] [LOCAL ] DHF: No primary data file found to delete. [2025-10-20T18:17:46.762383] [INFO ] [LOCAL ] DHF: Starting deletion of derivative data. [2025-10-20T18:17:46.794882] [DEBUG] [LOCAL ] DHF: Deleted derivative data file. [2025-10-20T18:17:46.839275] [INFO ] [MAIN ] Update Progress: 613/2367 symbols completed. DHF took 15.1s. Estimated remaining: 3h 42m 21s. Rate: 7.9 symbols/min [2025-10-20T18:17:47.263367] [INFO ] [MAIN ] DHT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:47.424837] [DEBUG] [VALIDATOR ] DG: All expected trading days are present. [2025-10-20T18:17:47.424960] [DEBUG] [VALIDATOR ] DG: Primary data validation passed. [2025-10-20T18:17:47.425898] [INFO ] [MAIN ] DG: Fetched primary data is valid. [2025-10-20T18:17:47.426047] [DEBUG] [LOCAL ] DG: Saving primary data. [2025-10-20T18:17:47.438449] [DEBUG] [LOCAL ] DG: Trimmed 77343 records to 53269 (keeping 200 days of data) [2025-10-20T18:17:47.499695] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-04. [2025-10-20T18:17:47.583771] [DEBUG] [LOCAL ] DHT: Loaded 48665 existing records from primary storage. [2025-10-20T18:17:47.601801] [INFO ] [MAIN ] DHT: Loaded 48665 existing records. [2025-10-20T18:17:47.931309] [WARN ] [MAIN ] DHT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:47.936313] [INFO ] [MAIN ] DHT: Wiping existing primary data before full refresh. [2025-10-20T18:17:47.936398] [INFO ] [LOCAL ] DHT: Starting deletion of primary data. [2025-10-20T18:17:47.943822] [DEBUG] [LOCAL ] DHT: Deleted primary data file. [2025-10-20T18:17:47.943895] [INFO ] [MAIN ] DHT: Starting full primary refresh. [2025-10-20T18:17:48.304148] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-03. [2025-10-20T18:17:48.487730] [INFO ] [REMOTE ] DHT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:48.487857] [DEBUG] [LOCAL ] DG: Primary data saved (53269 records). [2025-10-20T18:17:48.488476] [INFO ] [MAIN ] DG: Full history saved. [2025-10-20T18:17:48.488616] [INFO ] [MAIN ] Update Progress: 614/2367 symbols completed. DG took 38.7s. Estimated remaining: 3h 41m 56s. Rate: 7.9 symbols/min [2025-10-20T18:17:48.674504] [INFO ] [MAIN ] DHX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:48.674759] [DEBUG] [LOCAL ] DHX: No existing data file found in primary storage. [2025-10-20T18:17:48.674810] [WARN ] [MAIN ] DHX: No existing data found, performing full refresh instead. [2025-10-20T18:17:48.674932] [INFO ] [MAIN ] DHX: Wiping existing primary data before full refresh. [2025-10-20T18:17:48.674975] [INFO ] [LOCAL ] DHX: Starting deletion of primary data. [2025-10-20T18:17:48.675017] [DEBUG] [LOCAL ] DHX: No primary data file found to delete. [2025-10-20T18:17:48.675050] [INFO ] [MAIN ] DHX: Starting full primary refresh. [2025-10-20T18:17:49.052897] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-01. [2025-10-20T18:17:49.052760] [INFO ] [REMOTE ] DHX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:49.827532] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-04 (2.33 seconds). [2025-10-20T18:17:49.851377] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-05. [2025-10-20T18:17:49.922433] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:17:50.555308] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:17:50.651336] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-02. [2025-10-20T18:17:51.451378] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-01. [2025-10-20T18:17:51.526393] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:17:52.251554] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-06. [2025-10-20T18:17:52.284660] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:17:52.336338] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T18:17:53.051368] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-03. [2025-10-20T18:17:53.851346] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-04. [2025-10-20T18:17:53.992410] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T18:17:54.651349] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-07. [2025-10-20T18:17:54.694522] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T18:17:55.451438] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-04. [2025-10-20T18:17:55.950281] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-10-20T18:17:56.251362] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-02. [2025-10-20T18:17:56.667552] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T18:17:57.051423] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-05. [2025-10-20T18:17:57.123181] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:17:57.284924] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T18:17:57.851369] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-03. [2025-10-20T18:17:58.607358] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T18:17:58.651342] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-06. [2025-10-20T18:17:58.797010] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T18:17:59.451408] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-08. [2025-10-20T18:18:00.013391] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T18:18:00.251391] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-07. [2025-10-20T18:18:01.051385] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-04. [2025-10-20T18:18:01.851355] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-05. [2025-10-20T18:18:01.949935] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-08 (2.50 seconds). [2025-10-20T18:18:02.122916] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:18:02.145930] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T18:18:02.651394] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-05. [2025-10-20T18:18:03.456638] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-08. [2025-10-20T18:18:03.475385] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:18:03.826430] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:18:04.251385] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-06. [2025-10-20T18:18:04.809421] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T18:18:05.051374] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-07. [2025-10-20T18:18:05.334618] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T18:18:05.851363] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-09. [2025-10-20T18:18:05.886288] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T18:18:06.651426] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-06. [2025-10-20T18:18:07.451365] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-09. [2025-10-20T18:18:07.974208] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-09 (2.12 seconds). [2025-10-20T18:18:08.128614] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:18:08.251379] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-10. [2025-10-20T18:18:09.051362] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-07. [2025-10-20T18:18:09.137580] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T18:18:09.146050] [INFO ] [REMOTE ] DHR: Full fetch complete (77441 records). [2025-10-20T18:18:09.146244] [DEBUG] [VALIDATOR ] DHR: Validating primary data. [2025-10-20T18:18:09.558175] [DEBUG] [VALIDATOR ] DHR: All expected trading days are present. [2025-10-20T18:18:09.558384] [DEBUG] [VALIDATOR ] DHR: Primary data validation passed. [2025-10-20T18:18:09.559424] [INFO ] [MAIN ] DHR: Fetched primary data is valid. [2025-10-20T18:18:09.559536] [DEBUG] [LOCAL ] DHR: Saving primary data. [2025-10-20T18:18:09.561312] [DEBUG] [LOCAL ] DHR: Trimmed 77441 records to 53381 (keeping 200 days of data) [2025-10-20T18:18:09.685600] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-10-20T18:18:09.851428] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-10. [2025-10-20T18:18:09.937620] [DEBUG] [LOCAL ] DHR: Primary data saved (53381 records). [2025-10-20T18:18:09.937782] [INFO ] [MAIN ] DHR: Full history saved. [2025-10-20T18:18:09.937855] [INFO ] [MAIN ] Update Progress: 615/2367 symbols completed. DHR took 32.0s. Estimated remaining: 3h 42m 28s. Rate: 7.9 symbols/min [2025-10-20T18:18:10.108498] [INFO ] [MAIN ] DIAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:10.123285] [DEBUG] [LOCAL ] DIAX: Loaded 13192 existing records from primary storage. [2025-10-20T18:18:10.123609] [INFO ] [MAIN ] DIAX: Loaded 13192 existing records. [2025-10-20T18:18:10.317841] [WARN ] [MAIN ] DIAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:10.318174] [INFO ] [MAIN ] DIAX: Wiping existing primary data before full refresh. [2025-10-20T18:18:10.318251] [INFO ] [LOCAL ] DIAX: Starting deletion of primary data. [2025-10-20T18:18:10.318591] [DEBUG] [LOCAL ] DIAX: Deleted primary data file. [2025-10-20T18:18:10.318865] [INFO ] [MAIN ] DIAX: Starting full primary refresh. [2025-10-20T18:18:10.507545] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:18:10.656522] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-08. [2025-10-20T18:18:10.725362] [INFO ] [REMOTE ] DIAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:11.267702] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T18:18:11.451369] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-08. [2025-10-20T18:18:12.040738] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-10 (2.19 seconds). [2025-10-20T18:18:12.049110] [INFO ] [REMOTE ] DHI: Full fetch complete (77394 records). [2025-10-20T18:18:12.049330] [DEBUG] [VALIDATOR ] DHI: Validating primary data. [2025-10-20T18:18:12.262342] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-09. [2025-10-20T18:18:12.447871] [DEBUG] [VALIDATOR ] DHI: All expected trading days are present. [2025-10-20T18:18:12.447961] [DEBUG] [VALIDATOR ] DHI: Primary data validation passed. [2025-10-20T18:18:12.448912] [INFO ] [MAIN ] DHI: Fetched primary data is valid. [2025-10-20T18:18:12.448984] [DEBUG] [LOCAL ] DHI: Saving primary data. [2025-10-20T18:18:12.450518] [DEBUG] [LOCAL ] DHI: Trimmed 77394 records to 53365 (keeping 200 days of data) [2025-10-20T18:18:12.774869] [DEBUG] [LOCAL ] DHI: Primary data saved (53365 records). [2025-10-20T18:18:12.775000] [INFO ] [MAIN ] DHI: Full history saved. [2025-10-20T18:18:12.775067] [INFO ] [MAIN ] Update Progress: 616/2367 symbols completed. DHI took 36.3s. Estimated remaining: 3h 42m 7s. Rate: 7.9 symbols/min [2025-10-20T18:18:12.942961] [INFO ] [MAIN ] DIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:12.977974] [DEBUG] [LOCAL ] DIN: Loaded 37698 existing records from primary storage. [2025-10-20T18:18:12.978246] [INFO ] [MAIN ] DIN: Loaded 37698 existing records. [2025-10-20T18:18:13.053595] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-01. [2025-10-20T18:18:13.188037] [WARN ] [MAIN ] DIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:13.200687] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T18:18:13.201112] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T18:18:13.210133] [INFO ] [MAIN ] DIN: Wiping existing primary data before full refresh. [2025-10-20T18:18:13.210666] [INFO ] [LOCAL ] DIN: Starting deletion of primary data. [2025-10-20T18:18:13.214610] [DEBUG] [LOCAL ] DIN: Deleted primary data file. [2025-10-20T18:18:13.214900] [INFO ] [MAIN ] DIN: Starting full primary refresh. [2025-10-20T18:18:13.564413] [INFO ] [REMOTE ] DIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:13.851362] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-01. [2025-10-20T18:18:13.985433] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T18:18:14.651396] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-10. [2025-10-20T18:18:15.451439] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-09. [2025-10-20T18:18:15.484413] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:18:15.759411] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:18:15.762555] [INFO ] [REMOTE ] DHX: Full fetch complete (21449 records). [2025-10-20T18:18:15.762704] [DEBUG] [VALIDATOR ] DHX: Validating primary data. [2025-10-20T18:18:16.145361] [DEBUG] [VALIDATOR ] DHX: All expected trading days are present. [2025-10-20T18:18:16.145460] [DEBUG] [VALIDATOR ] DHX: Primary data validation passed. [2025-10-20T18:18:16.145912] [INFO ] [MAIN ] DHX: Fetched primary data is valid. [2025-10-20T18:18:16.145974] [DEBUG] [LOCAL ] DHX: Saving primary data. [2025-10-20T18:18:16.146585] [DEBUG] [LOCAL ] DHX: Trimmed 21449 records to 13199 (keeping 200 days of data) [2025-10-20T18:18:16.218371] [DEBUG] [LOCAL ] DHX: Primary data saved (13199 records). [2025-10-20T18:18:16.218515] [INFO ] [MAIN ] DHX: Full history saved. [2025-10-20T18:18:16.218589] [INFO ] [MAIN ] Update Progress: 617/2367 symbols completed. DHX took 27.7s. Estimated remaining: 3h 41m 47s. Rate: 7.9 symbols/min [2025-10-20T18:18:16.260611] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-02. [2025-10-20T18:18:16.387381] [INFO ] [MAIN ] DINO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:16.433979] [DEBUG] [LOCAL ] DINO: Loaded 53587 existing records from primary storage. [2025-10-20T18:18:16.434352] [INFO ] [MAIN ] DINO: Loaded 53587 existing records. [2025-10-20T18:18:16.607174] [WARN ] [MAIN ] DINO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:16.607377] [INFO ] [MAIN ] DINO: Wiping existing primary data before full refresh. [2025-10-20T18:18:16.607423] [INFO ] [LOCAL ] DINO: Starting deletion of primary data. [2025-10-20T18:18:16.607991] [DEBUG] [LOCAL ] DINO: Deleted primary data file. [2025-10-20T18:18:16.608047] [INFO ] [MAIN ] DINO: Starting full primary refresh. [2025-10-20T18:18:16.965768] [INFO ] [REMOTE ] DINO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:17.051403] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-01. [2025-10-20T18:18:17.076459] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T18:18:17.268362] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:18:17.851377] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-10. [2025-10-20T18:18:18.651368] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-03. [2025-10-20T18:18:19.033921] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T18:18:19.146454] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T18:18:19.451397] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-02. [2025-10-20T18:18:19.664852] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T18:18:19.672505] [INFO ] [REMOTE ] DHT: Full fetch complete (70233 records). [2025-10-20T18:18:19.672694] [DEBUG] [VALIDATOR ] DHT: Validating primary data. [2025-10-20T18:18:20.075702] [DEBUG] [VALIDATOR ] DHT: All expected trading days are present. [2025-10-20T18:18:20.075796] [DEBUG] [VALIDATOR ] DHT: Primary data validation passed. [2025-10-20T18:18:20.076609] [INFO ] [MAIN ] DHT: Fetched primary data is valid. [2025-10-20T18:18:20.076685] [DEBUG] [LOCAL ] DHT: Saving primary data. [2025-10-20T18:18:20.078069] [DEBUG] [LOCAL ] DHT: Trimmed 70233 records to 47915 (keeping 200 days of data) [2025-10-20T18:18:20.269621] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-02. [2025-10-20T18:18:20.391522] [DEBUG] [LOCAL ] DHT: Primary data saved (47915 records). [2025-10-20T18:18:20.391696] [INFO ] [MAIN ] DHT: Full history saved. [2025-10-20T18:18:20.391789] [INFO ] [MAIN ] Update Progress: 618/2367 symbols completed. DHT took 33.5s. Estimated remaining: 3h 41m 30s. Rate: 7.9 symbols/min [2025-10-20T18:18:20.577060] [INFO ] [MAIN ] DIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:20.631005] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T18:18:20.659801] [DEBUG] [LOCAL ] DIS: Loaded 54568 existing records from primary storage. [2025-10-20T18:18:20.660157] [INFO ] [MAIN ] DIS: Loaded 54568 existing records. [2025-10-20T18:18:20.859876] [WARN ] [MAIN ] DIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:20.860160] [INFO ] [MAIN ] DIS: Wiping existing primary data before full refresh. [2025-10-20T18:18:20.860219] [INFO ] [LOCAL ] DIS: Starting deletion of primary data. [2025-10-20T18:18:20.860715] [DEBUG] [LOCAL ] DIS: Deleted primary data file. [2025-10-20T18:18:20.860772] [INFO ] [MAIN ] DIS: Starting full primary refresh. [2025-10-20T18:18:21.056516] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-03. [2025-10-20T18:18:21.257769] [INFO ] [REMOTE ] DIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:21.668530] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T18:18:21.851371] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-03. [2025-10-20T18:18:22.651352] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-01. [2025-10-20T18:18:22.984301] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T18:18:23.060005] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T18:18:23.451431] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-04. [2025-10-20T18:18:24.251375] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-04. [2025-10-20T18:18:24.708507] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-01 (2.06 seconds). [2025-10-20T18:18:25.040627] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:18:25.051325] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-05. [2025-10-20T18:18:25.851410] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-02. [2025-10-20T18:18:25.920094] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:18:26.509250] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T18:18:26.651386] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-04. [2025-10-20T18:18:27.451433] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-06. [2025-10-20T18:18:27.630114] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T18:18:27.879809] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-10-20T18:18:28.251402] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-05. [2025-10-20T18:18:28.645438] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T18:18:29.051394] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-07. [2025-10-20T18:18:29.851358] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-05. [2025-10-20T18:18:30.033322] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T18:18:30.490144] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:18:30.602172] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T18:18:30.651426] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-06. [2025-10-20T18:18:31.101055] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T18:18:31.451382] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-06. [2025-10-20T18:18:32.251406] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-08. [2025-10-20T18:18:33.051462] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-03. [2025-10-20T18:18:33.252372] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T18:18:33.733634] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:18:33.851369] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-09. [2025-10-20T18:18:34.651401] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-07. [2025-10-20T18:18:35.009979] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T18:18:35.456573] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-10. [2025-10-20T18:18:35.477594] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-03 (2.43 seconds). [2025-10-20T18:18:35.576978] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T18:18:36.251427] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-07. [2025-10-20T18:18:37.007949] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T18:18:37.013893] [INFO ] [REMOTE ] DIN: Full fetch complete (54925 records). [2025-10-20T18:18:37.014024] [DEBUG] [VALIDATOR ] DIN: Validating primary data. [2025-10-20T18:18:37.056597] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-08. [2025-10-20T18:18:37.443350] [DEBUG] [VALIDATOR ] DIN: All expected trading days are present. [2025-10-20T18:18:37.443449] [DEBUG] [VALIDATOR ] DIN: Primary data validation passed. [2025-10-20T18:18:37.444226] [INFO ] [MAIN ] DIN: Fetched primary data is valid. [2025-10-20T18:18:37.444290] [DEBUG] [LOCAL ] DIN: Saving primary data. [2025-10-20T18:18:37.445421] [DEBUG] [LOCAL ] DIN: Trimmed 54925 records to 37006 (keeping 200 days of data) [2025-10-20T18:18:37.674622] [DEBUG] [LOCAL ] DIN: Primary data saved (37006 records). [2025-10-20T18:18:37.674799] [INFO ] [MAIN ] DIN: Full history saved. [2025-10-20T18:18:37.675136] [INFO ] [MAIN ] Update Progress: 619/2367 symbols completed. DIN took 24.9s. Estimated remaining: 3h 41m 50s. Rate: 7.9 symbols/min [2025-10-20T18:18:37.854743] [INFO ] [MAIN ] DIVY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:37.853707] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-04. [2025-10-20T18:18:37.855119] [DEBUG] [LOCAL ] DIVY: No existing data file found in primary storage. [2025-10-20T18:18:37.863161] [WARN ] [MAIN ] DIVY: No existing data found, performing full refresh instead. [2025-10-20T18:18:37.893408] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T18:18:37.892809] [INFO ] [MAIN ] DIVY: Wiping existing primary data before full refresh. [2025-10-20T18:18:37.896900] [INFO ] [LOCAL ] DIVY: Starting deletion of primary data. [2025-10-20T18:18:37.900498] [DEBUG] [LOCAL ] DIVY: No primary data file found to delete. [2025-10-20T18:18:37.902948] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:18:37.901426] [INFO ] [MAIN ] DIVY: Starting full primary refresh. [2025-10-20T18:18:38.309393] [INFO ] [REMOTE ] DIVY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:38.651351] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-08. [2025-10-20T18:18:39.451413] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-09. [2025-10-20T18:18:40.251362] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-01. [2025-10-20T18:18:40.281923] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:18:40.407179] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:18:40.546394] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-04 (2.69 seconds). [2025-10-20T18:18:40.623751] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-01 (0.37 seconds). [2025-10-20T18:18:41.051391] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-09. [2025-10-20T18:18:41.851400] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-02. [2025-10-20T18:18:42.207101] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:18:42.651353] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-05. [2025-10-20T18:18:42.900348] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T18:18:43.451420] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-10. [2025-10-20T18:18:44.251430] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-10. [2025-10-20T18:18:45.051392] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-03. [2025-10-20T18:18:45.195234] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-05 (2.54 seconds). [2025-10-20T18:18:45.229268] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T18:18:45.238018] [INFO ] [REMOTE ] DINO: Full fetch complete (76708 records). [2025-10-20T18:18:45.238137] [DEBUG] [VALIDATOR ] DINO: Validating primary data. [2025-10-20T18:18:45.345699] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-10-20T18:18:45.375128] [INFO ] [REMOTE ] DIAX: Full fetch complete (17679 records). [2025-10-20T18:18:45.422912] [DEBUG] [VALIDATOR ] DIAX: Validating primary data. [2025-10-20T18:18:45.558720] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-10-20T18:18:45.857085] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-06. [2025-10-20T18:18:45.861245] [DEBUG] [VALIDATOR ] DINO: All expected trading days are present. [2025-10-20T18:18:45.899797] [DEBUG] [VALIDATOR ] DINO: Primary data validation passed. [2025-10-20T18:18:45.933420] [INFO ] [MAIN ] DINO: Fetched primary data is valid. [2025-10-20T18:18:45.944866] [DEBUG] [LOCAL ] DINO: Saving primary data. [2025-10-20T18:18:46.007569] [DEBUG] [LOCAL ] DINO: Trimmed 76708 records to 52842 (keeping 200 days of data) [2025-10-20T18:18:46.129458] [DEBUG] [VALIDATOR ] DIAX: All expected trading days are present. [2025-10-20T18:18:46.129560] [DEBUG] [VALIDATOR ] DIAX: Primary data validation passed. [2025-10-20T18:18:46.157363] [INFO ] [MAIN ] DIAX: Fetched primary data is valid. [2025-10-20T18:18:46.160066] [DEBUG] [LOCAL ] DIAX: Saving primary data. [2025-10-20T18:18:46.176755] [DEBUG] [LOCAL ] DIAX: Trimmed 17679 records to 12903 (keeping 200 days of data) [2025-10-20T18:18:46.426408] [DEBUG] [LOCAL ] DIAX: Primary data saved (12903 records). [2025-10-20T18:18:46.432214] [INFO ] [MAIN ] DIAX: Full history saved. [2025-10-20T18:18:46.432349] [INFO ] [MAIN ] Update Progress: 620/2367 symbols completed. DIAX took 36.5s. Estimated remaining: 3h 41m 45s. Rate: 7.9 symbols/min [2025-10-20T18:18:46.548907] [DEBUG] [LOCAL ] DINO: Primary data saved (52842 records). [2025-10-20T18:18:46.565788] [INFO ] [MAIN ] DINO: Full history saved. [2025-10-20T18:18:46.573749] [INFO ] [MAIN ] Update Progress: 621/2367 symbols completed. DINO took 30.4s. Estimated remaining: 3h 41m 17s. Rate: 7.9 symbols/min [2025-10-20T18:18:46.656502] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-04. [2025-10-20T18:18:46.791686] [INFO ] [MAIN ] DK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:46.804505] [DEBUG] [LOCAL ] DK: No existing data file found in primary storage. [2025-10-20T18:18:46.804792] [WARN ] [MAIN ] DK: No existing data found, performing full refresh instead. [2025-10-20T18:18:46.806218] [INFO ] [MAIN ] DK: Wiping existing primary data before full refresh. [2025-10-20T18:18:46.806608] [INFO ] [LOCAL ] DK: Starting deletion of primary data. [2025-10-20T18:18:46.827383] [DEBUG] [LOCAL ] DK: No primary data file found to delete. [2025-10-20T18:18:46.829587] [INFO ] [MAIN ] DKL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:46.829820] [INFO ] [MAIN ] DK: Starting full primary refresh. [2025-10-20T18:18:47.023490] [DEBUG] [LOCAL ] DKL: Loaded 17652 existing records from primary storage. [2025-10-20T18:18:47.081501] [INFO ] [MAIN ] DKL: Loaded 17652 existing records. [2025-10-20T18:18:47.237624] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T18:18:47.397563] [INFO ] [REMOTE ] DK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:47.417353] [WARN ] [MAIN ] DKL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:47.418007] [INFO ] [MAIN ] DKL: Wiping existing primary data before full refresh. [2025-10-20T18:18:47.418056] [INFO ] [LOCAL ] DKL: Starting deletion of primary data. [2025-10-20T18:18:47.418361] [DEBUG] [LOCAL ] DKL: Deleted primary data file. [2025-10-20T18:18:47.418417] [INFO ] [MAIN ] DKL: Starting full primary refresh. [2025-10-20T18:18:47.460322] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-01. [2025-10-20T18:18:47.776948] [INFO ] [REMOTE ] DKL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:48.172468] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-06 (2.32 seconds). [2025-10-20T18:18:48.251338] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-07. [2025-10-20T18:18:48.897052] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:18:49.051402] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-05. [2025-10-20T18:18:49.401287] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T18:18:49.851392] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-06. [2025-10-20T18:18:50.162993] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-10-20T18:18:50.641825] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-07 (2.39 seconds). [2025-10-20T18:18:50.653259] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-02. [2025-10-20T18:18:51.451398] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-07. [2025-10-20T18:18:51.908748] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T18:18:52.042224] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T18:18:52.251382] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-03. [2025-10-20T18:18:53.051396] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-01. [2025-10-20T18:18:53.851569] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-08. [2025-10-20T18:18:53.982313] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T18:18:53.993895] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:18:54.114423] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-08 (0.26 seconds). [2025-10-20T18:18:54.651367] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-08. [2025-10-20T18:18:55.451384] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-04. [2025-10-20T18:18:56.251367] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-09. [2025-10-20T18:18:56.967628] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T18:18:56.996989] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-08 (2.35 seconds). [2025-10-20T18:18:57.039412] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:18:57.051322] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-05. [2025-10-20T18:18:57.851408] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-10. [2025-10-20T18:18:58.198142] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-10-20T18:18:58.198965] [INFO ] [REMOTE ] DIVY: Full fetch complete (1701 records). [2025-10-20T18:18:58.199033] [DEBUG] [VALIDATOR ] DIVY: Validating primary data. [2025-10-20T18:18:58.407496] [WARN ] [VALIDATOR ] DIVY: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T18:18:58.407801] [FATAL] [MAIN ] DIVY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:18:58.408067] [WARN ] [MAIN ] DIVY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:18:58.408142] [INFO ] [LOCAL ] DIVY: Starting deletion of primary data. [2025-10-20T18:18:58.408384] [DEBUG] [LOCAL ] DIVY: No primary data file found to delete. [2025-10-20T18:18:58.408450] [INFO ] [LOCAL ] DIVY: Starting deletion of derivative data. [2025-10-20T18:18:58.408582] [DEBUG] [LOCAL ] DIVY: No derivative data file found to delete. [2025-10-20T18:18:58.408743] [INFO ] [MAIN ] Update Progress: 622/2367 symbols completed. DIVY took 20.7s. Estimated remaining: 3h 41m 21s. Rate: 7.9 symbols/min [2025-10-20T18:18:58.604075] [INFO ] [MAIN ] DKS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:58.634823] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:18:58.651366] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-06. [2025-10-20T18:18:58.697709] [DEBUG] [LOCAL ] DKS: Loaded 50372 existing records from primary storage. [2025-10-20T18:18:58.698114] [INFO ] [MAIN ] DKS: Loaded 50372 existing records. [2025-10-20T18:18:58.883108] [WARN ] [MAIN ] DKS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:58.883343] [INFO ] [MAIN ] DKS: Wiping existing primary data before full refresh. [2025-10-20T18:18:58.883398] [INFO ] [LOCAL ] DKS: Starting deletion of primary data. [2025-10-20T18:18:58.883582] [DEBUG] [LOCAL ] DKS: Deleted primary data file. [2025-10-20T18:18:58.883621] [INFO ] [MAIN ] DKS: Starting full primary refresh. [2025-10-20T18:18:59.253134] [INFO ] [REMOTE ] DKS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:59.451368] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-01. [2025-10-20T18:19:00.251353] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-09. [2025-10-20T18:19:00.515858] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T18:19:01.051355] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-02. [2025-10-20T18:19:01.407534] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T18:19:01.851354] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-07. [2025-10-20T18:19:02.180920] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:19:02.651346] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-03. [2025-10-20T18:19:02.688750] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-09 (2.44 seconds). [2025-10-20T18:19:03.322960] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T18:19:03.451357] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-02. [2025-10-20T18:19:04.167706] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:19:04.251370] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-08. [2025-10-20T18:19:04.926879] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T18:19:05.051375] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-04. [2025-10-20T18:19:05.851348] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-10. [2025-10-20T18:19:06.093974] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T18:19:06.563166] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:19:06.651352] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-09. [2025-10-20T18:19:06.675147] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T18:19:06.683951] [INFO ] [REMOTE ] DIS: Full fetch complete (77967 records). [2025-10-20T18:19:06.684100] [DEBUG] [VALIDATOR ] DIS: Validating primary data. [2025-10-20T18:19:07.076932] [DEBUG] [VALIDATOR ] DIS: All expected trading days are present. [2025-10-20T18:19:07.077028] [DEBUG] [VALIDATOR ] DIS: Primary data validation passed. [2025-10-20T18:19:07.078008] [INFO ] [MAIN ] DIS: Fetched primary data is valid. [2025-10-20T18:19:07.078070] [DEBUG] [LOCAL ] DIS: Saving primary data. [2025-10-20T18:19:07.079490] [DEBUG] [LOCAL ] DIS: Trimmed 77967 records to 53788 (keeping 200 days of data) [2025-10-20T18:19:07.403102] [DEBUG] [LOCAL ] DIS: Primary data saved (53788 records). [2025-10-20T18:19:07.403233] [INFO ] [MAIN ] DIS: Full history saved. [2025-10-20T18:19:07.403311] [INFO ] [MAIN ] Update Progress: 623/2367 symbols completed. DIS took 47.0s. Estimated remaining: 3h 41m 17s. Rate: 7.9 symbols/min [2025-10-20T18:19:07.458023] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-05. [2025-10-20T18:19:07.573173] [INFO ] [MAIN ] DLB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:07.603314] [DEBUG] [LOCAL ] DLB: Loaded 32746 existing records from primary storage. [2025-10-20T18:19:07.603586] [INFO ] [MAIN ] DLB: Loaded 32746 existing records. [2025-10-20T18:19:07.784515] [WARN ] [MAIN ] DLB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:07.784745] [INFO ] [MAIN ] DLB: Wiping existing primary data before full refresh. [2025-10-20T18:19:07.784797] [INFO ] [LOCAL ] DLB: Starting deletion of primary data. [2025-10-20T18:19:07.785042] [DEBUG] [LOCAL ] DLB: Deleted primary data file. [2025-10-20T18:19:07.785080] [INFO ] [MAIN ] DLB: Starting full primary refresh. [2025-10-20T18:19:08.140789] [INFO ] [REMOTE ] DLB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:08.251389] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-01. [2025-10-20T18:19:08.610916] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T18:19:08.635001] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:19:09.051373] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-10. [2025-10-20T18:19:09.851368] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-03. [2025-10-20T18:19:10.027171] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T18:19:10.651383] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-06. [2025-10-20T18:19:11.207522] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-10 (2.16 seconds). [2025-10-20T18:19:11.214724] [INFO ] [REMOTE ] DK: Full fetch complete (70837 records). [2025-10-20T18:19:11.214887] [DEBUG] [VALIDATOR ] DK: Validating primary data. [2025-10-20T18:19:11.460906] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-02. [2025-10-20T18:19:11.617427] [DEBUG] [VALIDATOR ] DK: All expected trading days are present. [2025-10-20T18:19:11.625326] [DEBUG] [VALIDATOR ] DK: Primary data validation passed. [2025-10-20T18:19:11.627566] [INFO ] [MAIN ] DK: Fetched primary data is valid. [2025-10-20T18:19:11.627698] [DEBUG] [LOCAL ] DK: Saving primary data. [2025-10-20T18:19:11.628992] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T18:19:11.634058] [DEBUG] [LOCAL ] DK: Trimmed 70837 records to 49346 (keeping 200 days of data) [2025-10-20T18:19:11.963144] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T18:19:11.980219] [DEBUG] [LOCAL ] DK: Primary data saved (49346 records). [2025-10-20T18:19:11.980743] [INFO ] [MAIN ] DK: Full history saved. [2025-10-20T18:19:11.980829] [INFO ] [MAIN ] Update Progress: 624/2367 symbols completed. DK took 25.5s. Estimated remaining: 3h 41m 1s. Rate: 7.9 symbols/min [2025-10-20T18:19:12.146080] [INFO ] [MAIN ] DLNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:12.153469] [DEBUG] [LOCAL ] DLNG: Loaded 6256 existing records from primary storage. [2025-10-20T18:19:12.153728] [INFO ] [MAIN ] DLNG: Loaded 6256 existing records. [2025-10-20T18:19:12.251428] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-07. [2025-10-20T18:19:12.330613] [WARN ] [MAIN ] DLNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:12.330835] [INFO ] [MAIN ] DLNG: Wiping existing primary data before full refresh. [2025-10-20T18:19:12.330883] [INFO ] [LOCAL ] DLNG: Starting deletion of primary data. [2025-10-20T18:19:12.331037] [DEBUG] [LOCAL ] DLNG: Deleted primary data file. [2025-10-20T18:19:12.331092] [INFO ] [MAIN ] DLNG: Starting full primary refresh. [2025-10-20T18:19:12.689549] [INFO ] [REMOTE ] DLNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:12.707490] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T18:19:13.020166] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T18:19:13.051375] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-01. [2025-10-20T18:19:13.851386] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-08. [2025-10-20T18:19:14.071688] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:19:14.651384] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-03. [2025-10-20T18:19:14.893887] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:19:15.451408] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-02. [2025-10-20T18:19:15.930139] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T18:19:16.251372] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-09. [2025-10-20T18:19:16.288251] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T18:19:17.051396] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-03. [2025-10-20T18:19:17.168974] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T18:19:17.708533] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T18:19:17.851409] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-10. [2025-10-20T18:19:18.651356] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-04. [2025-10-20T18:19:18.802792] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T18:19:18.804960] [INFO ] [REMOTE ] DKL: Full fetch complete (27662 records). [2025-10-20T18:19:18.805035] [DEBUG] [VALIDATOR ] DKL: Validating primary data. [2025-10-20T18:19:19.164268] [DEBUG] [VALIDATOR ] DKL: All expected trading days are present. [2025-10-20T18:19:19.164343] [DEBUG] [VALIDATOR ] DKL: Primary data validation passed. [2025-10-20T18:19:19.164853] [INFO ] [MAIN ] DKL: Fetched primary data is valid. [2025-10-20T18:19:19.164916] [DEBUG] [LOCAL ] DKL: Saving primary data. [2025-10-20T18:19:19.165534] [DEBUG] [LOCAL ] DKL: Trimmed 27662 records to 17203 (keeping 200 days of data) [2025-10-20T18:19:19.268973] [DEBUG] [LOCAL ] DKL: Primary data saved (17203 records). [2025-10-20T18:19:19.269104] [INFO ] [MAIN ] DKL: Full history saved. [2025-10-20T18:19:19.269179] [INFO ] [MAIN ] Update Progress: 625/2367 symbols completed. DKL took 32.7s. Estimated remaining: 3h 40m 53s. Rate: 7.9 symbols/min [2025-10-20T18:19:19.439404] [INFO ] [MAIN ] DLR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:19.451369] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-04. [2025-10-20T18:19:19.490429] [DEBUG] [LOCAL ] DLR: Loaded 51423 existing records from primary storage. [2025-10-20T18:19:19.490775] [INFO ] [MAIN ] DLR: Loaded 51423 existing records. [2025-10-20T18:19:19.673506] [WARN ] [MAIN ] DLR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:19.673819] [INFO ] [MAIN ] DLR: Wiping existing primary data before full refresh. [2025-10-20T18:19:19.673873] [INFO ] [LOCAL ] DLR: Starting deletion of primary data. [2025-10-20T18:19:19.674139] [DEBUG] [LOCAL ] DLR: Deleted primary data file. [2025-10-20T18:19:19.674207] [INFO ] [MAIN ] DLR: Starting full primary refresh. [2025-10-20T18:19:20.047711] [INFO ] [REMOTE ] DLR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:20.066872] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T18:19:20.101976] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T18:19:20.251344] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-05. [2025-10-20T18:19:21.051369] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-04. [2025-10-20T18:19:21.551428] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T18:19:21.851377] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-01. [2025-10-20T18:19:22.651355] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-06. [2025-10-20T18:19:22.985153] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T18:19:23.451369] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-05. [2025-10-20T18:19:23.655680] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T18:19:24.073277] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:19:24.198881] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T18:19:24.251395] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-06. [2025-10-20T18:19:25.051443] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-02. [2025-10-20T18:19:25.104375] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T18:19:25.851448] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-07. [2025-10-20T18:19:26.304665] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T18:19:26.651392] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-08. [2025-10-20T18:19:26.961434] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T18:19:27.324926] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T18:19:27.451382] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-07. [2025-10-20T18:19:28.251428] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-03. [2025-10-20T18:19:28.827600] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T18:19:29.051375] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-05. [2025-10-20T18:19:29.796663] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T18:19:29.851421] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-04. [2025-10-20T18:19:30.651354] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-09. [2025-10-20T18:19:30.942124] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:19:31.248334] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T18:19:31.451352] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-10. [2025-10-20T18:19:31.751392] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T18:19:32.040718] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T18:19:32.042131] [INFO ] [REMOTE ] DLNG: Full fetch complete (11617 records). [2025-10-20T18:19:32.042202] [DEBUG] [VALIDATOR ] DLNG: Validating primary data. [2025-10-20T18:19:32.256591] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-05. [2025-10-20T18:19:32.412143] [DEBUG] [VALIDATOR ] DLNG: All expected trading days are present. [2025-10-20T18:19:32.412428] [DEBUG] [VALIDATOR ] DLNG: Primary data validation passed. [2025-10-20T18:19:32.412743] [INFO ] [MAIN ] DLNG: Fetched primary data is valid. [2025-10-20T18:19:32.412802] [DEBUG] [LOCAL ] DLNG: Saving primary data. [2025-10-20T18:19:32.413290] [DEBUG] [LOCAL ] DLNG: Trimmed 11617 records to 6143 (keeping 200 days of data) [2025-10-20T18:19:32.448825] [DEBUG] [LOCAL ] DLNG: Primary data saved (6143 records). [2025-10-20T18:19:32.448925] [INFO ] [MAIN ] DLNG: Full history saved. [2025-10-20T18:19:32.449001] [INFO ] [MAIN ] Update Progress: 626/2367 symbols completed. DLNG took 20.5s. Estimated remaining: 3h 41m 1s. Rate: 7.9 symbols/min [2025-10-20T18:19:32.618123] [INFO ] [MAIN ] DLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:32.646465] [DEBUG] [LOCAL ] DLX: Loaded 28685 existing records from primary storage. [2025-10-20T18:19:32.646746] [INFO ] [MAIN ] DLX: Loaded 28685 existing records. [2025-10-20T18:19:32.822224] [WARN ] [MAIN ] DLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:32.822436] [INFO ] [MAIN ] DLX: Wiping existing primary data before full refresh. [2025-10-20T18:19:32.822484] [INFO ] [LOCAL ] DLX: Starting deletion of primary data. [2025-10-20T18:19:32.822755] [DEBUG] [LOCAL ] DLX: Deleted primary data file. [2025-10-20T18:19:32.822814] [INFO ] [MAIN ] DLX: Starting full primary refresh. [2025-10-20T18:19:33.058625] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-06. [2025-10-20T18:19:33.184372] [INFO ] [REMOTE ] DLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:33.851594] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-08. [2025-10-20T18:19:34.107138] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:19:34.651364] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-06. [2025-10-20T18:19:34.688445] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T18:19:35.026253] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T18:19:35.451352] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-09. [2025-10-20T18:19:36.251440] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-01. [2025-10-20T18:19:36.326009] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:19:36.798317] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T18:19:37.051374] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-10. [2025-10-20T18:19:37.414834] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:19:37.851450] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-07. [2025-10-20T18:19:38.327912] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:19:38.335836] [INFO ] [REMOTE ] DLB: Full fetch complete (47353 records). [2025-10-20T18:19:38.336008] [DEBUG] [VALIDATOR ] DLB: Validating primary data. [2025-10-20T18:19:38.656687] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-07. [2025-10-20T18:19:38.721439] [DEBUG] [VALIDATOR ] DLB: All expected trading days are present. [2025-10-20T18:19:38.721522] [DEBUG] [VALIDATOR ] DLB: Primary data validation passed. [2025-10-20T18:19:38.722231] [INFO ] [MAIN ] DLB: Fetched primary data is valid. [2025-10-20T18:19:38.722295] [DEBUG] [LOCAL ] DLB: Saving primary data. [2025-10-20T18:19:38.723342] [DEBUG] [LOCAL ] DLB: Trimmed 47353 records to 32311 (keeping 200 days of data) [2025-10-20T18:19:38.924529] [DEBUG] [LOCAL ] DLB: Primary data saved (32311 records). [2025-10-20T18:19:38.924920] [INFO ] [MAIN ] DLB: Full history saved. [2025-10-20T18:19:38.925017] [INFO ] [MAIN ] Update Progress: 627/2367 symbols completed. DLB took 31.5s. Estimated remaining: 3h 40m 50s. Rate: 7.9 symbols/min [2025-10-20T18:19:39.096670] [INFO ] [MAIN ] DLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:39.116087] [DEBUG] [LOCAL ] DLY: Loaded 20947 existing records from primary storage. [2025-10-20T18:19:39.116360] [INFO ] [MAIN ] DLY: Loaded 20947 existing records. [2025-10-20T18:19:39.298409] [WARN ] [MAIN ] DLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:39.298758] [INFO ] [MAIN ] DLY: Wiping existing primary data before full refresh. [2025-10-20T18:19:39.298816] [INFO ] [LOCAL ] DLY: Starting deletion of primary data. [2025-10-20T18:19:39.299047] [DEBUG] [LOCAL ] DLY: Deleted primary data file. [2025-10-20T18:19:39.299102] [INFO ] [MAIN ] DLY: Starting full primary refresh. [2025-10-20T18:19:39.466700] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-02. [2025-10-20T18:19:39.669151] [INFO ] [REMOTE ] DLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:39.699552] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:19:40.251404] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-08. [2025-10-20T18:19:40.501693] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:19:40.820408] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T18:19:41.051378] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-08. [2025-10-20T18:19:41.851410] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-03. [2025-10-20T18:19:41.953618] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T18:19:42.463872] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T18:19:42.651408] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-01. [2025-10-20T18:19:43.018343] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T18:19:43.442869] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T18:19:43.451333] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-02. [2025-10-20T18:19:44.062306] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T18:19:44.251435] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-04. [2025-10-20T18:19:45.051430] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-03. [2025-10-20T18:19:45.593367] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:19:45.851409] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-05. [2025-10-20T18:19:46.037185] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T18:19:46.651432] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-04. [2025-10-20T18:19:46.910769] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:19:47.451382] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-09. [2025-10-20T18:19:47.721118] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:19:48.251429] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-05. [2025-10-20T18:19:49.051376] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-09. [2025-10-20T18:19:49.136932] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:19:49.179383] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:19:49.851436] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-06. [2025-10-20T18:19:50.651398] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-10. [2025-10-20T18:19:50.826910] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T18:19:50.878457] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T18:19:51.356770] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T18:19:51.371149] [INFO ] [REMOTE ] DLR: Full fetch complete (74165 records). [2025-10-20T18:19:51.371323] [DEBUG] [VALIDATOR ] DLR: Validating primary data. [2025-10-20T18:19:51.458964] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-07. [2025-10-20T18:19:51.815094] [DEBUG] [VALIDATOR ] DLR: All expected trading days are present. [2025-10-20T18:19:51.815172] [DEBUG] [VALIDATOR ] DLR: Primary data validation passed. [2025-10-20T18:19:51.816234] [INFO ] [MAIN ] DLR: Fetched primary data is valid. [2025-10-20T18:19:51.816316] [DEBUG] [LOCAL ] DLR: Saving primary data. [2025-10-20T18:19:51.817698] [DEBUG] [LOCAL ] DLR: Trimmed 74165 records to 50645 (keeping 200 days of data) [2025-10-20T18:19:52.123102] [DEBUG] [LOCAL ] DLR: Primary data saved (50645 records). [2025-10-20T18:19:52.123233] [INFO ] [MAIN ] DLR: Full history saved. [2025-10-20T18:19:52.123310] [INFO ] [MAIN ] Update Progress: 628/2367 symbols completed. DLR took 32.9s. Estimated remaining: 3h 40m 58s. Rate: 7.9 symbols/min [2025-10-20T18:19:52.263466] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-06. [2025-10-20T18:19:52.293934] [INFO ] [MAIN ] DMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:52.294939] [DEBUG] [LOCAL ] DMA: No existing data file found in primary storage. [2025-10-20T18:19:52.294996] [WARN ] [MAIN ] DMA: No existing data found, performing full refresh instead. [2025-10-20T18:19:52.295104] [INFO ] [MAIN ] DMA: Wiping existing primary data before full refresh. [2025-10-20T18:19:52.295145] [INFO ] [LOCAL ] DMA: Starting deletion of primary data. [2025-10-20T18:19:52.295186] [DEBUG] [LOCAL ] DMA: No primary data file found to delete. [2025-10-20T18:19:52.295217] [INFO ] [MAIN ] DMA: Starting full primary refresh. [2025-10-20T18:19:52.683358] [INFO ] [REMOTE ] DMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:52.685799] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T18:19:53.051415] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-01. [2025-10-20T18:19:53.562421] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T18:19:53.742838] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:19:53.851390] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-07. [2025-10-20T18:19:54.651411] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-10. [2025-10-20T18:19:55.161943] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T18:19:55.451384] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-08. [2025-10-20T18:19:56.251369] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-08. [2025-10-20T18:19:56.316281] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T18:19:56.323537] [INFO ] [REMOTE ] DKS: Full fetch complete (71728 records). [2025-10-20T18:19:56.323716] [DEBUG] [VALIDATOR ] DKS: Validating primary data. [2025-10-20T18:19:56.649681] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T18:19:56.751119] [DEBUG] [VALIDATOR ] DKS: All expected trading days are present. [2025-10-20T18:19:56.751198] [DEBUG] [VALIDATOR ] DKS: Primary data validation passed. [2025-10-20T18:19:56.752095] [INFO ] [MAIN ] DKS: Fetched primary data is valid. [2025-10-20T18:19:56.752160] [DEBUG] [LOCAL ] DKS: Saving primary data. [2025-10-20T18:19:56.753525] [DEBUG] [LOCAL ] DKS: Trimmed 71728 records to 49551 (keeping 200 days of data) [2025-10-20T18:19:57.053290] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-09. [2025-10-20T18:19:57.072281] [DEBUG] [LOCAL ] DKS: Primary data saved (49551 records). [2025-10-20T18:19:57.072398] [INFO ] [MAIN ] DKS: Full history saved. [2025-10-20T18:19:57.072694] [INFO ] [MAIN ] Update Progress: 629/2367 symbols completed. DKS took 58.7s. Estimated remaining: 3h 40m 43s. Rate: 7.9 symbols/min [2025-10-20T18:19:57.242421] [INFO ] [MAIN ] DMB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:57.242684] [DEBUG] [LOCAL ] DMB: No existing data file found in primary storage. [2025-10-20T18:19:57.242742] [WARN ] [MAIN ] DMB: No existing data found, performing full refresh instead. [2025-10-20T18:19:57.242851] [INFO ] [MAIN ] DMB: Wiping existing primary data before full refresh. [2025-10-20T18:19:57.242914] [INFO ] [LOCAL ] DMB: Starting deletion of primary data. [2025-10-20T18:19:57.242959] [DEBUG] [LOCAL ] DMB: No primary data file found to delete. [2025-10-20T18:19:57.242993] [INFO ] [MAIN ] DMB: Starting full primary refresh. [2025-10-20T18:19:57.604367] [INFO ] [REMOTE ] DMB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:57.621450] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T18:19:57.851379] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-01. [2025-10-20T18:19:58.150942] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T18:19:58.651345] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-02. [2025-10-20T18:19:58.842385] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T18:19:59.132952] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T18:19:59.451385] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-02. [2025-10-20T18:20:00.163485] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T18:20:00.251344] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-03. [2025-10-20T18:20:00.907255] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T18:20:01.051376] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-10. [2025-10-20T18:20:01.851358] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-03. [2025-10-20T18:20:02.397922] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T18:20:02.407695] [INFO ] [REMOTE ] DLY: Full fetch complete (28491 records). [2025-10-20T18:20:02.407907] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T18:20:02.408093] [DEBUG] [VALIDATOR ] DLY: Validating primary data. [2025-10-20T18:20:02.660370] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-09. [2025-10-20T18:20:02.795830] [DEBUG] [VALIDATOR ] DLY: All expected trading days are present. [2025-10-20T18:20:02.795910] [DEBUG] [VALIDATOR ] DLY: Primary data validation passed. [2025-10-20T18:20:02.796396] [INFO ] [MAIN ] DLY: Fetched primary data is valid. [2025-10-20T18:20:02.796434] [DEBUG] [LOCAL ] DLY: Saving primary data. [2025-10-20T18:20:02.797216] [DEBUG] [LOCAL ] DLY: Trimmed 28491 records to 20770 (keeping 200 days of data) [2025-10-20T18:20:02.920887] [DEBUG] [LOCAL ] DLY: Primary data saved (20770 records). [2025-10-20T18:20:02.921013] [INFO ] [MAIN ] DLY: Full history saved. [2025-10-20T18:20:02.921090] [INFO ] [MAIN ] Update Progress: 630/2367 symbols completed. DLY took 24.0s. Estimated remaining: 3h 40m 30s. Rate: 7.9 symbols/min [2025-10-20T18:20:03.094460] [INFO ] [MAIN ] DMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:03.094630] [DEBUG] [LOCAL ] DMO: No existing data file found in primary storage. [2025-10-20T18:20:03.094707] [WARN ] [MAIN ] DMO: No existing data found, performing full refresh instead. [2025-10-20T18:20:03.094835] [INFO ] [MAIN ] DMO: Wiping existing primary data before full refresh. [2025-10-20T18:20:03.094878] [INFO ] [LOCAL ] DMO: Starting deletion of primary data. [2025-10-20T18:20:03.094920] [DEBUG] [LOCAL ] DMO: No primary data file found to delete. [2025-10-20T18:20:03.094955] [INFO ] [MAIN ] DMO: Starting full primary refresh. [2025-10-20T18:20:03.457047] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-04. [2025-10-20T18:20:03.463270] [INFO ] [REMOTE ] DMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:04.102329] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T18:20:04.218491] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T18:20:04.251334] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-01. [2025-10-20T18:20:05.051376] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-10. [2025-10-20T18:20:05.154351] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T18:20:05.851369] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-05. [2025-10-20T18:20:06.419402] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:20:06.423228] [INFO ] [REMOTE ] DLX: Full fetch complete (41309 records). [2025-10-20T18:20:06.423328] [DEBUG] [VALIDATOR ] DLX: Validating primary data. [2025-10-20T18:20:06.540213] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T18:20:06.660598] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-02. [2025-10-20T18:20:06.823385] [DEBUG] [VALIDATOR ] DLX: All expected trading days are present. [2025-10-20T18:20:06.823470] [DEBUG] [VALIDATOR ] DLX: Primary data validation passed. [2025-10-20T18:20:06.824088] [INFO ] [MAIN ] DLX: Fetched primary data is valid. [2025-10-20T18:20:06.824151] [DEBUG] [LOCAL ] DLX: Saving primary data. [2025-10-20T18:20:06.825159] [DEBUG] [LOCAL ] DLX: Trimmed 41309 records to 28318 (keeping 200 days of data) [2025-10-20T18:20:07.000751] [DEBUG] [LOCAL ] DLX: Primary data saved (28318 records). [2025-10-20T18:20:07.000891] [INFO ] [MAIN ] DLX: Full history saved. [2025-10-20T18:20:07.000970] [INFO ] [MAIN ] Update Progress: 631/2367 symbols completed. DLX took 34.6s. Estimated remaining: 3h 40m 13s. Rate: 7.9 symbols/min [2025-10-20T18:20:07.188437] [INFO ] [MAIN ] DNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:07.231542] [DEBUG] [LOCAL ] DNA: Loaded 49343 existing records from primary storage. [2025-10-20T18:20:07.231871] [INFO ] [MAIN ] DNA: Loaded 49343 existing records. [2025-10-20T18:20:07.413788] [WARN ] [MAIN ] DNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:20:07.414200] [INFO ] [MAIN ] DNA: Wiping existing primary data before full refresh. [2025-10-20T18:20:07.414262] [INFO ] [LOCAL ] DNA: Starting deletion of primary data. [2025-10-20T18:20:07.414560] [DEBUG] [LOCAL ] DNA: Deleted primary data file. [2025-10-20T18:20:07.414619] [INFO ] [MAIN ] DNA: Starting full primary refresh. [2025-10-20T18:20:07.457486] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-04. [2025-10-20T18:20:07.536173] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T18:20:07.799435] [INFO ] [REMOTE ] DNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:08.251442] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-06. [2025-10-20T18:20:08.311132] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T18:20:08.787972] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:20:09.051393] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-05. [2025-10-20T18:20:09.851388] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-03. [2025-10-20T18:20:09.978399] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:20:10.651379] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-01. [2025-10-20T18:20:10.678340] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:20:11.451432] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-07. [2025-10-20T18:20:12.026100] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T18:20:12.251410] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-08. [2025-10-20T18:20:12.613470] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T18:20:12.913496] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T18:20:13.051453] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-04. [2025-10-20T18:20:13.824223] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T18:20:13.851335] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-05. [2025-10-20T18:20:14.544104] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T18:20:14.651411] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-02. [2025-10-20T18:20:15.451398] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-09. [2025-10-20T18:20:16.225601] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T18:20:16.251349] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-10. [2025-10-20T18:20:16.655514] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-10-20T18:20:16.967979] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T18:20:16.969249] [INFO ] [REMOTE ] DMA: Full fetch complete (6602 records). [2025-10-20T18:20:16.969318] [DEBUG] [VALIDATOR ] DMA: Validating primary data. [2025-10-20T18:20:16.969699] [FATAL] [VALIDATOR ] DMA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:20:16.969795] [FATAL] [MAIN ] DMA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:20:16.969871] [WARN ] [MAIN ] DMA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:20:16.969969] [INFO ] [LOCAL ] DMA: Starting deletion of primary data. [2025-10-20T18:20:16.970067] [DEBUG] [LOCAL ] DMA: No primary data file found to delete. [2025-10-20T18:20:16.970125] [INFO ] [LOCAL ] DMA: Starting deletion of derivative data. [2025-10-20T18:20:16.970171] [DEBUG] [LOCAL ] DMA: No derivative data file found to delete. [2025-10-20T18:20:16.970224] [INFO ] [MAIN ] Update Progress: 632/2367 symbols completed. DMA took 24.8s. Estimated remaining: 3h 40m 12s. Rate: 7.9 symbols/min [2025-10-20T18:20:17.056456] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-03. [2025-10-20T18:20:17.165689] [INFO ] [MAIN ] DNOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:17.206410] [DEBUG] [LOCAL ] DNOW: Loaded 44485 existing records from primary storage. [2025-10-20T18:20:17.206718] [INFO ] [MAIN ] DNOW: Loaded 44485 existing records. [2025-10-20T18:20:17.385112] [WARN ] [MAIN ] DNOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:20:17.385386] [INFO ] [MAIN ] DNOW: Wiping existing primary data before full refresh. [2025-10-20T18:20:17.385438] [INFO ] [LOCAL ] DNOW: Starting deletion of primary data. [2025-10-20T18:20:17.385714] [DEBUG] [LOCAL ] DNOW: Deleted primary data file. [2025-10-20T18:20:17.385773] [INFO ] [MAIN ] DNOW: Starting full primary refresh. [2025-10-20T18:20:17.749400] [INFO ] [REMOTE ] DNOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:17.851380] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-01. [2025-10-20T18:20:18.651360] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-06. [2025-10-20T18:20:19.038169] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T18:20:19.385492] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:20:19.420200] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:20:19.451438] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-07. [2025-10-20T18:20:20.251428] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-06. [2025-10-20T18:20:20.367346] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T18:20:20.944270] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T18:20:21.051409] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-08. [2025-10-20T18:20:21.811599] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T18:20:21.851382] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-09. [2025-10-20T18:20:22.640861] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:20:22.651344] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-10. [2025-10-20T18:20:23.441823] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T18:20:23.443628] [INFO ] [REMOTE ] DMB: Full fetch complete (14670 records). [2025-10-20T18:20:23.443846] [DEBUG] [VALIDATOR ] DMB: Validating primary data. [2025-10-20T18:20:23.456492] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-02. [2025-10-20T18:20:23.874466] [DEBUG] [VALIDATOR ] DMB: All expected trading days are present. [2025-10-20T18:20:23.874558] [DEBUG] [VALIDATOR ] DMB: Primary data validation passed. [2025-10-20T18:20:23.874937] [INFO ] [MAIN ] DMB: Fetched primary data is valid. [2025-10-20T18:20:23.874998] [DEBUG] [LOCAL ] DMB: Saving primary data. [2025-10-20T18:20:23.875688] [DEBUG] [LOCAL ] DMB: Trimmed 14670 records to 10378 (keeping 200 days of data) [2025-10-20T18:20:23.939295] [DEBUG] [LOCAL ] DMB: Primary data saved (10378 records). [2025-10-20T18:20:23.939448] [INFO ] [MAIN ] DMB: Full history saved. [2025-10-20T18:20:23.939521] [INFO ] [MAIN ] Update Progress: 633/2367 symbols completed. DMB took 26.9s. Estimated remaining: 3h 40m 2s. Rate: 7.9 symbols/min [2025-10-20T18:20:24.116335] [INFO ] [MAIN ] DNP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:24.117002] [DEBUG] [LOCAL ] DNP: No existing data file found in primary storage. [2025-10-20T18:20:24.117056] [WARN ] [MAIN ] DNP: No existing data found, performing full refresh instead. [2025-10-20T18:20:24.117240] [INFO ] [MAIN ] DNP: Wiping existing primary data before full refresh. [2025-10-20T18:20:24.117294] [INFO ] [LOCAL ] DNP: Starting deletion of primary data. [2025-10-20T18:20:24.117339] [DEBUG] [LOCAL ] DNP: No primary data file found to delete. [2025-10-20T18:20:24.117392] [INFO ] [MAIN ] DNP: Starting full primary refresh. [2025-10-20T18:20:24.258890] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-04. [2025-10-20T18:20:24.544010] [INFO ] [REMOTE ] DNP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:25.022064] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:20:25.051405] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-03. [2025-10-20T18:20:25.851399] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-01. [2025-10-20T18:20:26.307183] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T18:20:26.651395] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-07. [2025-10-20T18:20:26.704621] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:20:27.452210] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-04. [2025-10-20T18:20:27.454992] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:20:27.497804] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T18:20:28.251396] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-05. [2025-10-20T18:20:29.035024] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T18:20:29.051347] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-05. [2025-10-20T18:20:29.851367] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-08. [2025-10-20T18:20:29.980521] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T18:20:30.579029] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T18:20:30.651355] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-02. [2025-10-20T18:20:30.742959] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T18:20:31.451421] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-09. [2025-10-20T18:20:31.785129] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:20:32.251365] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-06. [2025-10-20T18:20:32.390519] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T18:20:33.051439] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-03. [2025-10-20T18:20:33.712121] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T18:20:33.851331] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-07. [2025-10-20T18:20:34.455344] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:20:34.651399] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-06. [2025-10-20T18:20:35.435547] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T18:20:35.451359] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-08. [2025-10-20T18:20:36.245513] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T18:20:36.255155] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-04. [2025-10-20T18:20:37.057831] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-07. [2025-10-20T18:20:37.078005] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T18:20:37.851430] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-09. [2025-10-20T18:20:37.949962] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:20:38.651418] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-05. [2025-10-20T18:20:39.090839] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T18:20:39.153481] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T18:20:39.451436] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-10. [2025-10-20T18:20:39.565066] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T18:20:40.251390] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-06. [2025-10-20T18:20:41.051398] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-08. [2025-10-20T18:20:41.194142] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T18:20:41.195676] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:20:41.214350] [INFO ] [REMOTE ] DNOW: Full fetch complete (63496 records). [2025-10-20T18:20:41.214572] [DEBUG] [VALIDATOR ] DNOW: Validating primary data. [2025-10-20T18:20:41.607664] [DEBUG] [VALIDATOR ] DNOW: All expected trading days are present. [2025-10-20T18:20:41.607756] [DEBUG] [VALIDATOR ] DNOW: Primary data validation passed. [2025-10-20T18:20:41.608581] [INFO ] [MAIN ] DNOW: Fetched primary data is valid. [2025-10-20T18:20:41.608632] [DEBUG] [LOCAL ] DNOW: Saving primary data. [2025-10-20T18:20:41.610132] [DEBUG] [LOCAL ] DNOW: Trimmed 63496 records to 43858 (keeping 200 days of data) [2025-10-20T18:20:41.861705] [DEBUG] [LOCAL ] DNOW: Primary data saved (43858 records). [2025-10-20T18:20:41.862060] [INFO ] [MAIN ] DNOW: Full history saved. [2025-10-20T18:20:41.861815] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-07. [2025-10-20T18:20:41.862259] [INFO ] [MAIN ] Update Progress: 634/2367 symbols completed. DNOW took 24.9s. Estimated remaining: 3h 40m 23s. Rate: 7.9 symbols/min [2025-10-20T18:20:42.042903] [INFO ] [MAIN ] DOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:42.096714] [DEBUG] [LOCAL ] DOC: Loaded 54334 existing records from primary storage. [2025-10-20T18:20:42.097053] [INFO ] [MAIN ] DOC: Loaded 54334 existing records. [2025-10-20T18:20:42.278816] [WARN ] [MAIN ] DOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:20:42.279049] [INFO ] [MAIN ] DOC: Wiping existing primary data before full refresh. [2025-10-20T18:20:42.279097] [INFO ] [LOCAL ] DOC: Starting deletion of primary data. [2025-10-20T18:20:42.279531] [DEBUG] [LOCAL ] DOC: Deleted primary data file. [2025-10-20T18:20:42.279588] [INFO ] [MAIN ] DOC: Starting full primary refresh. [2025-10-20T18:20:42.635771] [INFO ] [REMOTE ] DOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:42.651330] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-01. [2025-10-20T18:20:42.917009] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T18:20:43.356518] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:20:43.451453] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-09. [2025-10-20T18:20:44.251393] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-10. [2025-10-20T18:20:44.450693] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T18:20:44.939959] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T18:20:44.941807] [INFO ] [REMOTE ] DMO: Full fetch complete (14559 records). [2025-10-20T18:20:44.941878] [DEBUG] [VALIDATOR ] DMO: Validating primary data. [2025-10-20T18:20:45.054877] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-08. [2025-10-20T18:20:45.338004] [DEBUG] [VALIDATOR ] DMO: All expected trading days are present. [2025-10-20T18:20:45.338130] [DEBUG] [VALIDATOR ] DMO: Primary data validation passed. [2025-10-20T18:20:45.338709] [INFO ] [MAIN ] DMO: Fetched primary data is valid. [2025-10-20T18:20:45.338830] [DEBUG] [LOCAL ] DMO: Saving primary data. [2025-10-20T18:20:45.339956] [DEBUG] [LOCAL ] DMO: Trimmed 14559 records to 9868 (keeping 200 days of data) [2025-10-20T18:20:45.424461] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T18:20:45.440999] [DEBUG] [LOCAL ] DMO: Primary data saved (9868 records). [2025-10-20T18:20:45.441127] [INFO ] [MAIN ] DMO: Full history saved. [2025-10-20T18:20:45.441194] [INFO ] [MAIN ] Update Progress: 635/2367 symbols completed. DMO took 42.5s. Estimated remaining: 3h 40m 4s. Rate: 7.9 symbols/min [2025-10-20T18:20:45.620724] [INFO ] [MAIN ] DOCN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:45.666794] [DEBUG] [LOCAL ] DOCN: Loaded 51798 existing records from primary storage. [2025-10-20T18:20:45.667104] [INFO ] [MAIN ] DOCN: Loaded 51798 existing records. [2025-10-20T18:20:45.841845] [WARN ] [MAIN ] DOCN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:20:45.842065] [INFO ] [MAIN ] DOCN: Wiping existing primary data before full refresh. [2025-10-20T18:20:45.842110] [INFO ] [LOCAL ] DOCN: Starting deletion of primary data. [2025-10-20T18:20:45.842415] [DEBUG] [LOCAL ] DOCN: Deleted primary data file. [2025-10-20T18:20:45.842452] [INFO ] [MAIN ] DOCN: Starting full primary refresh. [2025-10-20T18:20:45.856458] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-10. [2025-10-20T18:20:46.197816] [INFO ] [REMOTE ] DOCN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:46.436608] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T18:20:46.651373] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-01. [2025-10-20T18:20:47.451394] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-02. [2025-10-20T18:20:48.144591] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-10 (2.29 seconds). [2025-10-20T18:20:48.152546] [INFO ] [REMOTE ] DNA: Full fetch complete (71299 records). [2025-10-20T18:20:48.152762] [DEBUG] [VALIDATOR ] DNA: Validating primary data. [2025-10-20T18:20:48.259052] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-09. [2025-10-20T18:20:48.566407] [DEBUG] [VALIDATOR ] DNA: All expected trading days are present. [2025-10-20T18:20:48.566528] [DEBUG] [VALIDATOR ] DNA: Primary data validation passed. [2025-10-20T18:20:48.567581] [INFO ] [MAIN ] DNA: Fetched primary data is valid. [2025-10-20T18:20:48.567760] [DEBUG] [LOCAL ] DNA: Saving primary data. [2025-10-20T18:20:48.569511] [DEBUG] [LOCAL ] DNA: Trimmed 71299 records to 48575 (keeping 200 days of data) [2025-10-20T18:20:48.686110] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-10-20T18:20:48.938188] [DEBUG] [LOCAL ] DNA: Primary data saved (48575 records). [2025-10-20T18:20:48.938484] [INFO ] [MAIN ] DNA: Full history saved. [2025-10-20T18:20:48.938699] [INFO ] [MAIN ] Update Progress: 636/2367 symbols completed. DNA took 41.9s. Estimated remaining: 3h 39m 45s. Rate: 7.9 symbols/min [2025-10-20T18:20:49.059840] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-02. [2025-10-20T18:20:49.157188] [INFO ] [MAIN ] DOCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:49.163116] [DEBUG] [LOCAL ] DOCS: No existing data file found in primary storage. [2025-10-20T18:20:49.170378] [WARN ] [MAIN ] DOCS: No existing data found, performing full refresh instead. [2025-10-20T18:20:49.173516] [INFO ] [MAIN ] DOCS: Wiping existing primary data before full refresh. [2025-10-20T18:20:49.173990] [INFO ] [LOCAL ] DOCS: Starting deletion of primary data. [2025-10-20T18:20:49.175021] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T18:20:49.175229] [DEBUG] [LOCAL ] DOCS: No primary data file found to delete. [2025-10-20T18:20:49.185163] [INFO ] [MAIN ] DOCS: Starting full primary refresh. [2025-10-20T18:20:49.607313] [INFO ] [REMOTE ] DOCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:49.628310] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T18:20:49.851388] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-01. [2025-10-20T18:20:50.651363] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-03. [2025-10-20T18:20:50.738746] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T18:20:51.451369] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-10. [2025-10-20T18:20:51.770140] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T18:20:52.251444] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-02. [2025-10-20T18:20:52.350245] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:20:53.051353] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-04. [2025-10-20T18:20:53.147314] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T18:20:53.154751] [INFO ] [REMOTE ] DNP: Full fetch complete (48751 records). [2025-10-20T18:20:53.154892] [DEBUG] [VALIDATOR ] DNP: Validating primary data. [2025-10-20T18:20:53.586023] [DEBUG] [VALIDATOR ] DNP: All expected trading days are present. [2025-10-20T18:20:53.586110] [DEBUG] [VALIDATOR ] DNP: Primary data validation passed. [2025-10-20T18:20:53.586778] [INFO ] [MAIN ] DNP: Fetched primary data is valid. [2025-10-20T18:20:53.586838] [DEBUG] [LOCAL ] DNP: Saving primary data. [2025-10-20T18:20:53.587863] [DEBUG] [LOCAL ] DNP: Trimmed 48751 records to 33154 (keeping 200 days of data) [2025-10-20T18:20:53.782777] [DEBUG] [LOCAL ] DNP: Primary data saved (33154 records). [2025-10-20T18:20:53.782936] [INFO ] [MAIN ] DNP: Full history saved. [2025-10-20T18:20:53.783020] [INFO ] [MAIN ] Update Progress: 637/2367 symbols completed. DNP took 29.8s. Estimated remaining: 3h 39m 30s. Rate: 7.9 symbols/min [2025-10-20T18:20:53.851454] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-03. [2025-10-20T18:20:54.808982] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-02 (2.56 seconds). [2025-10-20T18:20:54.818892] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-03. [2025-10-20T18:20:55.341032] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T18:20:55.451336] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-05. [2025-10-20T18:20:55.818008] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T18:20:56.251415] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-04. [2025-10-20T18:20:56.622671] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T18:20:57.051347] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-04. [2025-10-20T18:20:57.189810] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:20:57.835589] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T18:20:57.851335] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-05. [2025-10-20T18:20:58.657814] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-06. [2025-10-20T18:20:58.683877] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:20:59.278595] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T18:20:59.451435] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-05. [2025-10-20T18:21:00.251377] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-06. [2025-10-20T18:21:00.408285] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T18:21:01.051367] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-07. [2025-10-20T18:21:01.557158] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-05 (2.11 seconds). [2025-10-20T18:21:01.851409] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-06. [2025-10-20T18:21:02.016179] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T18:21:02.651380] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-07. [2025-10-20T18:21:03.079352] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T18:21:03.451371] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-08. [2025-10-20T18:21:03.582301] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T18:21:04.251432] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-07. [2025-10-20T18:21:04.414096] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T18:21:04.994063] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:21:05.051455] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-08. [2025-10-20T18:21:05.795872] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T18:21:05.851371] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-08. [2025-10-20T18:21:06.651419] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-09. [2025-10-20T18:21:07.014015] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T18:21:07.451345] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-09. [2025-10-20T18:21:07.641590] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T18:21:08.251415] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-09. [2025-10-20T18:21:09.771739] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-09 (3.12 seconds). [2025-10-20T18:21:09.783362] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-10. [2025-10-20T18:21:09.841132] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-09 (2.39 seconds). [2025-10-20T18:21:09.852589] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-10. [2025-10-20T18:21:10.430395] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-10-20T18:21:10.651395] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-10. [2025-10-20T18:21:11.762529] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:21:11.770134] [INFO ] [REMOTE ] DOCN: Full fetch complete (72423 records). [2025-10-20T18:21:11.770304] [DEBUG] [VALIDATOR ] DOCN: Validating primary data. [2025-10-20T18:21:11.771179] [FATAL] [VALIDATOR ] DOCN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:21:11.771267] [FATAL] [MAIN ] DOCN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:21:11.771310] [WARN ] [MAIN ] DOCN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:21:11.771347] [INFO ] [LOCAL ] DOCN: Starting deletion of primary data. [2025-10-20T18:21:11.771409] [DEBUG] [LOCAL ] DOCN: No primary data file found to delete. [2025-10-20T18:21:11.771452] [INFO ] [LOCAL ] DOCN: Starting deletion of derivative data. [2025-10-20T18:21:11.771628] [DEBUG] [LOCAL ] DOCN: Deleted derivative data file. [2025-10-20T18:21:11.771773] [INFO ] [MAIN ] Update Progress: 638/2367 symbols completed. DOCN took 26.3s. Estimated remaining: 3h 39m 51s. Rate: 7.9 symbols/min [2025-10-20T18:21:11.834579] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T18:21:11.842895] [INFO ] [REMOTE ] DOC: Full fetch complete (77570 records). [2025-10-20T18:21:11.843014] [DEBUG] [VALIDATOR ] DOC: Validating primary data. [2025-10-20T18:21:12.233556] [DEBUG] [VALIDATOR ] DOC: All expected trading days are present. [2025-10-20T18:21:12.233701] [DEBUG] [VALIDATOR ] DOC: Primary data validation passed. [2025-10-20T18:21:12.234730] [INFO ] [MAIN ] DOC: Fetched primary data is valid. [2025-10-20T18:21:12.234910] [DEBUG] [LOCAL ] DOC: Saving primary data. [2025-10-20T18:21:12.261106] [DEBUG] [LOCAL ] DOC: Trimmed 77570 records to 53556 (keeping 200 days of data) [2025-10-20T18:21:12.262168] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T18:21:12.270083] [INFO ] [REMOTE ] DOCS: Full fetch complete (73598 records). [2025-10-20T18:21:12.270280] [DEBUG] [VALIDATOR ] DOCS: Validating primary data. [2025-10-20T18:21:12.321759] [FATAL] [VALIDATOR ] DOCS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:21:12.330820] [FATAL] [MAIN ] DOCS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:21:12.357203] [WARN ] [MAIN ] DOCS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:21:12.368535] [INFO ] [LOCAL ] DOCS: Starting deletion of primary data. [2025-10-20T18:21:12.391760] [DEBUG] [LOCAL ] DOCS: No primary data file found to delete. [2025-10-20T18:21:12.391852] [INFO ] [LOCAL ] DOCS: Starting deletion of derivative data. [2025-10-20T18:21:12.391896] [DEBUG] [LOCAL ] DOCS: No derivative data file found to delete. [2025-10-20T18:21:12.391984] [INFO ] [MAIN ] Update Progress: 639/2367 symbols completed. DOCS took 23.5s. Estimated remaining: 3h 39m 24s. Rate: 7.9 symbols/min [2025-10-20T18:21:12.588146] [DEBUG] [LOCAL ] DOC: Primary data saved (53556 records). [2025-10-20T18:21:12.588274] [INFO ] [MAIN ] DOC: Full history saved. [2025-10-20T18:21:12.588357] [INFO ] [MAIN ] Update Progress: 640/2367 symbols completed. DOC took 30.7s. Estimated remaining: 3h 38m 56s. Rate: 7.9 symbols/min [2025-10-20T18:21:13.162009] [INFO ] [MAIN ] DOLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:13.299181] [INFO ] [MAIN ] DOUG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:13.323279] [INFO ] [MAIN ] DOV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:13.340619] [INFO ] [MAIN ] DOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:13.411821] [DEBUG] [LOCAL ] DOUG: Loaded 34582 existing records from primary storage. [2025-10-20T18:21:13.412330] [INFO ] [MAIN ] DOUG: Loaded 34582 existing records. [2025-10-20T18:21:13.558775] [DEBUG] [LOCAL ] DOLE: Loaded 39732 existing records from primary storage. [2025-10-20T18:21:13.559720] [DEBUG] [LOCAL ] DOW: Loaded 54513 existing records from primary storage. [2025-10-20T18:21:13.562418] [DEBUG] [LOCAL ] DOV: Loaded 46315 existing records from primary storage. [2025-10-20T18:21:13.573131] [INFO ] [MAIN ] DOLE: Loaded 39732 existing records. [2025-10-20T18:21:13.573041] [INFO ] [MAIN ] DOV: Loaded 46315 existing records. [2025-10-20T18:21:13.573330] [INFO ] [MAIN ] DOW: Loaded 54513 existing records. [2025-10-20T18:21:13.943143] [WARN ] [MAIN ] DOUG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:13.978672] [INFO ] [MAIN ] DOUG: Wiping existing primary data before full refresh. [2025-10-20T18:21:13.990150] [INFO ] [LOCAL ] DOUG: Starting deletion of primary data. [2025-10-20T18:21:14.109459] [DEBUG] [LOCAL ] DOUG: Deleted primary data file. [2025-10-20T18:21:14.150891] [WARN ] [MAIN ] DOV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:14.159900] [INFO ] [MAIN ] DOUG: Starting full primary refresh. [2025-10-20T18:21:14.202411] [INFO ] [MAIN ] DOV: Wiping existing primary data before full refresh. [2025-10-20T18:21:14.244156] [INFO ] [LOCAL ] DOV: Starting deletion of primary data. [2025-10-20T18:21:14.287198] [WARN ] [MAIN ] DOLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:14.301056] [WARN ] [MAIN ] DOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:14.329014] [INFO ] [MAIN ] DOW: Wiping existing primary data before full refresh. [2025-10-20T18:21:14.329257] [INFO ] [MAIN ] DOLE: Wiping existing primary data before full refresh. [2025-10-20T18:21:14.315683] [DEBUG] [LOCAL ] DOV: Deleted primary data file. [2025-10-20T18:21:14.340031] [INFO ] [LOCAL ] DOW: Starting deletion of primary data. [2025-10-20T18:21:14.345241] [INFO ] [LOCAL ] DOLE: Starting deletion of primary data. [2025-10-20T18:21:14.350464] [INFO ] [MAIN ] DOV: Starting full primary refresh. [2025-10-20T18:21:14.371884] [DEBUG] [LOCAL ] DOW: Deleted primary data file. [2025-10-20T18:21:14.380533] [INFO ] [MAIN ] DOW: Starting full primary refresh. [2025-10-20T18:21:14.386006] [DEBUG] [LOCAL ] DOLE: Deleted primary data file. [2025-10-20T18:21:14.425280] [INFO ] [MAIN ] DOLE: Starting full primary refresh. [2025-10-20T18:21:15.714173] [INFO ] [REMOTE ] DOUG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:15.745573] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-01. [2025-10-20T18:21:15.786175] [INFO ] [REMOTE ] DOV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:15.815535] [INFO ] [REMOTE ] DOLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:15.835240] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-01. [2025-10-20T18:21:15.834941] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-01. [2025-10-20T18:21:15.858954] [INFO ] [REMOTE ] DOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:15.859305] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-01. [2025-10-20T18:21:17.291105] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T18:21:17.298959] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-02. [2025-10-20T18:21:17.371152] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:21:17.378384] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-02. [2025-10-20T18:21:17.419785] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T18:21:17.428500] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-02. [2025-10-20T18:21:17.908101] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T18:21:17.919117] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-02. [2025-10-20T18:21:18.243263] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:21:18.250450] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-03. [2025-10-20T18:21:18.472817] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T18:21:18.651365] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-03. [2025-10-20T18:21:18.959312] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:21:19.204609] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T18:21:19.451415] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-03. [2025-10-20T18:21:19.659295] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T18:21:19.861879] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T18:21:20.251376] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-04. [2025-10-20T18:21:20.832470] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:21:21.051388] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-03. [2025-10-20T18:21:21.851391] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-04. [2025-10-20T18:21:22.557590] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-04 (2.31 seconds). [2025-10-20T18:21:22.651395] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-05. [2025-10-20T18:21:23.451428] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-04. [2025-10-20T18:21:23.911301] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-03 (2.86 seconds). [2025-10-20T18:21:24.251383] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-04. [2025-10-20T18:21:24.318953] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:21:24.434533] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-04 (2.58 seconds). [2025-10-20T18:21:24.742973] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T18:21:25.051476] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-06. [2025-10-20T18:21:25.851480] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-05. [2025-10-20T18:21:26.357494] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:21:26.570772] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-04 (2.32 seconds). [2025-10-20T18:21:26.651335] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-05. [2025-10-20T18:21:27.456528] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-07. [2025-10-20T18:21:27.470284] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:21:28.251429] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-06. [2025-10-20T18:21:28.876407] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T18:21:28.896520] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-10-20T18:21:29.051419] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-05. [2025-10-20T18:21:29.593348] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T18:21:29.851462] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-08. [2025-10-20T18:21:30.651399] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-07. [2025-10-20T18:21:30.742993] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T18:21:30.840821] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:21:31.451436] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-06. [2025-10-20T18:21:32.251366] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-06. [2025-10-20T18:21:32.685920] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T18:21:32.917925] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T18:21:33.051422] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-08. [2025-10-20T18:21:33.851422] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-09. [2025-10-20T18:21:34.393791] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-10-20T18:21:34.574615] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T18:21:34.651410] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-07. [2025-10-20T18:21:35.264749] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T18:21:35.451431] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-09. [2025-10-20T18:21:36.038873] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:21:36.251436] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-08. [2025-10-20T18:21:36.667144] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T18:21:37.051382] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-07. [2025-10-20T18:21:37.343892] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T18:21:37.851380] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-10. [2025-10-20T18:21:38.651563] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-09. [2025-10-20T18:21:39.232971] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-07 (2.18 seconds). [2025-10-20T18:21:39.269135] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:21:39.276155] [INFO ] [REMOTE ] DOV: Full fetch complete (65938 records). [2025-10-20T18:21:39.276261] [DEBUG] [VALIDATOR ] DOV: Validating primary data. [2025-10-20T18:21:39.458559] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-10. [2025-10-20T18:21:39.676168] [DEBUG] [VALIDATOR ] DOV: All expected trading days are present. [2025-10-20T18:21:39.676262] [DEBUG] [VALIDATOR ] DOV: Primary data validation passed. [2025-10-20T18:21:39.677160] [INFO ] [MAIN ] DOV: Fetched primary data is valid. [2025-10-20T18:21:39.677226] [DEBUG] [LOCAL ] DOV: Saving primary data. [2025-10-20T18:21:39.678632] [DEBUG] [LOCAL ] DOV: Trimmed 65938 records to 45575 (keeping 200 days of data) [2025-10-20T18:21:39.956437] [DEBUG] [LOCAL ] DOV: Primary data saved (45575 records). [2025-10-20T18:21:39.956541] [INFO ] [MAIN ] DOV: Full history saved. [2025-10-20T18:21:39.956617] [INFO ] [MAIN ] Update Progress: 641/2367 symbols completed. DOV took 27.4s. Estimated remaining: 3h 39m 42s. Rate: 7.9 symbols/min [2025-10-20T18:21:40.126596] [INFO ] [MAIN ] DPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:40.126850] [DEBUG] [LOCAL ] DPG: No existing data file found in primary storage. [2025-10-20T18:21:40.126956] [WARN ] [MAIN ] DPG: No existing data found, performing full refresh instead. [2025-10-20T18:21:40.145109] [INFO ] [MAIN ] DPG: Wiping existing primary data before full refresh. [2025-10-20T18:21:40.145282] [INFO ] [LOCAL ] DPG: Starting deletion of primary data. [2025-10-20T18:21:40.145558] [DEBUG] [LOCAL ] DPG: No primary data file found to delete. [2025-10-20T18:21:40.146008] [INFO ] [MAIN ] DPG: Starting full primary refresh. [2025-10-20T18:21:40.146813] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:21:40.256756] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-08. [2025-10-20T18:21:40.508506] [INFO ] [REMOTE ] DPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:40.928073] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T18:21:40.934583] [INFO ] [REMOTE ] DOLE: Full fetch complete (54624 records). [2025-10-20T18:21:40.934762] [DEBUG] [VALIDATOR ] DOLE: Validating primary data. [2025-10-20T18:21:41.056518] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-01. [2025-10-20T18:21:41.340679] [DEBUG] [VALIDATOR ] DOLE: All expected trading days are present. [2025-10-20T18:21:41.340780] [DEBUG] [VALIDATOR ] DOLE: Primary data validation passed. [2025-10-20T18:21:41.341475] [INFO ] [MAIN ] DOLE: Fetched primary data is valid. [2025-10-20T18:21:41.341515] [DEBUG] [LOCAL ] DOLE: Saving primary data. [2025-10-20T18:21:41.342782] [DEBUG] [LOCAL ] DOLE: Trimmed 54624 records to 39134 (keeping 200 days of data) [2025-10-20T18:21:41.572890] [DEBUG] [LOCAL ] DOLE: Primary data saved (39134 records). [2025-10-20T18:21:41.573025] [INFO ] [MAIN ] DOLE: Full history saved. [2025-10-20T18:21:41.573281] [INFO ] [MAIN ] Update Progress: 642/2367 symbols completed. DOLE took 29.0s. Estimated remaining: 3h 39m 18s. Rate: 7.9 symbols/min [2025-10-20T18:21:41.742664] [INFO ] [MAIN ] DQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:41.786185] [DEBUG] [LOCAL ] DQ: Loaded 44434 existing records from primary storage. [2025-10-20T18:21:41.786528] [INFO ] [MAIN ] DQ: Loaded 44434 existing records. [2025-10-20T18:21:41.851510] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-10. [2025-10-20T18:21:41.968837] [WARN ] [MAIN ] DQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:41.969051] [INFO ] [MAIN ] DQ: Wiping existing primary data before full refresh. [2025-10-20T18:21:41.969120] [INFO ] [LOCAL ] DQ: Starting deletion of primary data. [2025-10-20T18:21:41.969507] [DEBUG] [LOCAL ] DQ: Deleted primary data file. [2025-10-20T18:21:41.969572] [INFO ] [MAIN ] DQ: Starting full primary refresh. [2025-10-20T18:21:42.233969] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T18:21:42.347760] [INFO ] [REMOTE ] DQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:42.453842] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-10-20T18:21:42.651383] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-01. [2025-10-20T18:21:43.310973] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:21:43.315629] [INFO ] [REMOTE ] DOUG: Full fetch complete (50225 records). [2025-10-20T18:21:43.315799] [DEBUG] [VALIDATOR ] DOUG: Validating primary data. [2025-10-20T18:21:43.466458] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-02. [2025-10-20T18:21:43.708818] [DEBUG] [VALIDATOR ] DOUG: All expected trading days are present. [2025-10-20T18:21:43.708945] [DEBUG] [VALIDATOR ] DOUG: Primary data validation passed. [2025-10-20T18:21:43.709640] [INFO ] [MAIN ] DOUG: Fetched primary data is valid. [2025-10-20T18:21:43.709722] [DEBUG] [LOCAL ] DOUG: Saving primary data. [2025-10-20T18:21:43.710745] [DEBUG] [LOCAL ] DOUG: Trimmed 50225 records to 33965 (keeping 200 days of data) [2025-10-20T18:21:43.903281] [DEBUG] [LOCAL ] DOUG: Primary data saved (33965 records). [2025-10-20T18:21:43.903479] [INFO ] [MAIN ] DOUG: Full history saved. [2025-10-20T18:21:43.903580] [INFO ] [MAIN ] Update Progress: 643/2367 symbols completed. DOUG took 31.3s. Estimated remaining: 3h 38m 56s. Rate: 7.9 symbols/min [2025-10-20T18:21:44.072122] [INFO ] [MAIN ] DRD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:44.108153] [DEBUG] [LOCAL ] DRD: Loaded 38427 existing records from primary storage. [2025-10-20T18:21:44.108432] [INFO ] [MAIN ] DRD: Loaded 38427 existing records. [2025-10-20T18:21:44.252487] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-09. [2025-10-20T18:21:44.289768] [WARN ] [MAIN ] DRD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:44.291080] [INFO ] [MAIN ] DRD: Wiping existing primary data before full refresh. [2025-10-20T18:21:44.291183] [INFO ] [LOCAL ] DRD: Starting deletion of primary data. [2025-10-20T18:21:44.291631] [DEBUG] [LOCAL ] DRD: Deleted primary data file. [2025-10-20T18:21:44.291809] [INFO ] [MAIN ] DRD: Starting full primary refresh. [2025-10-20T18:21:44.411782] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T18:21:44.457583] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T18:21:44.702515] [INFO ] [REMOTE ] DRD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:45.051372] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-01. [2025-10-20T18:21:45.851353] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-02. [2025-10-20T18:21:46.297852] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T18:21:46.550921] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:21:46.651372] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-02. [2025-10-20T18:21:47.275875] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T18:21:47.451387] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-10. [2025-10-20T18:21:48.033999] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:21:48.251391] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-03. [2025-10-20T18:21:49.051369] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-03. [2025-10-20T18:21:49.523799] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-10-20T18:21:49.532201] [INFO ] [REMOTE ] DOW: Full fetch complete (77910 records). [2025-10-20T18:21:49.532340] [DEBUG] [VALIDATOR ] DOW: Validating primary data. [2025-10-20T18:21:49.855119] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-03. [2025-10-20T18:21:49.983751] [DEBUG] [VALIDATOR ] DOW: All expected trading days are present. [2025-10-20T18:21:49.977156] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T18:21:49.984050] [DEBUG] [VALIDATOR ] DOW: Primary data validation passed. [2025-10-20T18:21:49.991889] [INFO ] [MAIN ] DOW: Fetched primary data is valid. [2025-10-20T18:21:49.990126] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T18:21:50.000980] [DEBUG] [LOCAL ] DOW: Saving primary data. [2025-10-20T18:21:50.002917] [DEBUG] [LOCAL ] DOW: Trimmed 77910 records to 53733 (keeping 200 days of data) [2025-10-20T18:21:50.324500] [DEBUG] [LOCAL ] DOW: Primary data saved (53733 records). [2025-10-20T18:21:50.324631] [INFO ] [MAIN ] DOW: Full history saved. [2025-10-20T18:21:50.324749] [INFO ] [MAIN ] Update Progress: 644/2367 symbols completed. DOW took 37.7s. Estimated remaining: 3h 38m 46s. Rate: 7.9 symbols/min [2025-10-20T18:21:50.492423] [INFO ] [MAIN ] DRH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:50.538389] [DEBUG] [LOCAL ] DRH: Loaded 51122 existing records from primary storage. [2025-10-20T18:21:50.538722] [INFO ] [MAIN ] DRH: Loaded 51122 existing records. [2025-10-20T18:21:50.656538] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-04. [2025-10-20T18:21:50.717739] [WARN ] [MAIN ] DRH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:50.717934] [INFO ] [MAIN ] DRH: Wiping existing primary data before full refresh. [2025-10-20T18:21:50.717980] [INFO ] [LOCAL ] DRH: Starting deletion of primary data. [2025-10-20T18:21:50.718472] [DEBUG] [LOCAL ] DRH: Deleted primary data file. [2025-10-20T18:21:50.718528] [INFO ] [MAIN ] DRH: Starting full primary refresh. [2025-10-20T18:21:51.079010] [INFO ] [REMOTE ] DRH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:51.100520] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T18:21:51.413268] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T18:21:51.451356] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-04. [2025-10-20T18:21:52.251394] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-04. [2025-10-20T18:21:52.932370] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:21:53.051391] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-05. [2025-10-20T18:21:53.851376] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-01. [2025-10-20T18:21:53.879074] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:21:54.132486] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T18:21:54.651362] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-06. [2025-10-20T18:21:55.352056] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:21:55.451368] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-05. [2025-10-20T18:21:56.070057] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:21:56.251383] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-07. [2025-10-20T18:21:56.930890] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:21:57.051364] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-06. [2025-10-20T18:21:57.804464] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T18:21:57.851378] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-08. [2025-10-20T18:21:57.991674] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:21:58.651417] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-07. [2025-10-20T18:21:59.392253] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:21:59.451406] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-09. [2025-10-20T18:22:00.195284] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T18:22:00.251349] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-08. [2025-10-20T18:22:01.051368] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-05. [2025-10-20T18:22:01.549053] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-10-20T18:22:01.851383] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-10. [2025-10-20T18:22:02.070888] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T18:22:02.205387] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T18:22:02.651419] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-06. [2025-10-20T18:22:03.451402] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-02. [2025-10-20T18:22:03.484220] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T18:22:03.751993] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T18:22:03.758923] [INFO ] [REMOTE ] DRD: Full fetch complete (51831 records). [2025-10-20T18:22:03.759078] [DEBUG] [VALIDATOR ] DRD: Validating primary data. [2025-10-20T18:22:04.143938] [DEBUG] [VALIDATOR ] DRD: All expected trading days are present. [2025-10-20T18:22:04.144024] [DEBUG] [VALIDATOR ] DRD: Primary data validation passed. [2025-10-20T18:22:04.144785] [INFO ] [MAIN ] DRD: Fetched primary data is valid. [2025-10-20T18:22:04.144848] [DEBUG] [LOCAL ] DRD: Saving primary data. [2025-10-20T18:22:04.146068] [DEBUG] [LOCAL ] DRD: Trimmed 51831 records to 37805 (keeping 200 days of data) [2025-10-20T18:22:04.251366] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-07. [2025-10-20T18:22:04.374911] [DEBUG] [LOCAL ] DRD: Primary data saved (37805 records). [2025-10-20T18:22:04.375039] [INFO ] [MAIN ] DRD: Full history saved. [2025-10-20T18:22:04.375114] [INFO ] [MAIN ] Update Progress: 645/2367 symbols completed. DRD took 20.5s. Estimated remaining: 3h 38m 55s. Rate: 7.9 symbols/min [2025-10-20T18:22:04.539895] [INFO ] [MAIN ] DRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:04.582770] [DEBUG] [LOCAL ] DRI: Loaded 48386 existing records from primary storage. [2025-10-20T18:22:04.583061] [INFO ] [MAIN ] DRI: Loaded 48386 existing records. [2025-10-20T18:22:04.761045] [WARN ] [MAIN ] DRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:04.761353] [INFO ] [MAIN ] DRI: Wiping existing primary data before full refresh. [2025-10-20T18:22:04.761411] [INFO ] [LOCAL ] DRI: Starting deletion of primary data. [2025-10-20T18:22:04.769783] [DEBUG] [LOCAL ] DRI: Deleted primary data file. [2025-10-20T18:22:04.783271] [INFO ] [MAIN ] DRI: Starting full primary refresh. [2025-10-20T18:22:04.816232] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T18:22:05.059024] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-03. [2025-10-20T18:22:05.148499] [INFO ] [REMOTE ] DRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:05.245321] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T18:22:05.851412] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-08. [2025-10-20T18:22:06.621480] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T18:22:06.642913] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T18:22:06.651318] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-09. [2025-10-20T18:22:07.451377] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-01. [2025-10-20T18:22:07.577935] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T18:22:08.251487] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-10. [2025-10-20T18:22:09.051355] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-09. [2025-10-20T18:22:09.132206] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:22:09.169376] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T18:22:09.171826] [INFO ] [REMOTE ] DPG: Full fetch complete (19068 records). [2025-10-20T18:22:09.171930] [DEBUG] [VALIDATOR ] DPG: Validating primary data. [2025-10-20T18:22:09.533573] [DEBUG] [VALIDATOR ] DPG: All expected trading days are present. [2025-10-20T18:22:09.533667] [DEBUG] [VALIDATOR ] DPG: Primary data validation passed. [2025-10-20T18:22:09.534074] [INFO ] [MAIN ] DPG: Fetched primary data is valid. [2025-10-20T18:22:09.534115] [DEBUG] [LOCAL ] DPG: Saving primary data. [2025-10-20T18:22:09.534619] [DEBUG] [LOCAL ] DPG: Trimmed 19068 records to 13502 (keeping 200 days of data) [2025-10-20T18:22:09.613336] [DEBUG] [LOCAL ] DPG: Primary data saved (13502 records). [2025-10-20T18:22:09.613456] [INFO ] [MAIN ] DPG: Full history saved. [2025-10-20T18:22:09.613524] [INFO ] [MAIN ] Update Progress: 646/2367 symbols completed. DPG took 29.7s. Estimated remaining: 3h 38m 41s. Rate: 7.9 symbols/min [2025-10-20T18:22:09.777381] [INFO ] [MAIN ] DRLL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:09.786639] [DEBUG] [LOCAL ] DRLL: Loaded 9219 existing records from primary storage. [2025-10-20T18:22:09.786852] [INFO ] [MAIN ] DRLL: Loaded 9219 existing records. [2025-10-20T18:22:09.859786] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-04. [2025-10-20T18:22:09.964522] [WARN ] [MAIN ] DRLL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:09.964788] [INFO ] [MAIN ] DRLL: Wiping existing primary data before full refresh. [2025-10-20T18:22:09.964841] [INFO ] [LOCAL ] DRLL: Starting deletion of primary data. [2025-10-20T18:22:09.964999] [DEBUG] [LOCAL ] DRLL: Deleted primary data file. [2025-10-20T18:22:09.965037] [INFO ] [MAIN ] DRLL: Starting full primary refresh. [2025-10-20T18:22:10.332291] [INFO ] [REMOTE ] DRLL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:10.651427] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-02. [2025-10-20T18:22:10.870081] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:22:11.451393] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-01. [2025-10-20T18:22:11.632767] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:22:12.034833] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:22:12.085558] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T18:22:12.251420] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-02. [2025-10-20T18:22:12.875639] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T18:22:13.051378] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-05. [2025-10-20T18:22:13.851435] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-03. [2025-10-20T18:22:14.651428] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-03. [2025-10-20T18:22:14.722330] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:22:15.451365] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-10. [2025-10-20T18:22:15.505433] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:22:15.609554] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T18:22:16.251360] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-04. [2025-10-20T18:22:17.051389] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-06. [2025-10-20T18:22:17.104015] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T18:22:17.111085] [INFO ] [REMOTE ] DQ: Full fetch complete (63642 records). [2025-10-20T18:22:17.111217] [DEBUG] [VALIDATOR ] DQ: Validating primary data. [2025-10-20T18:22:17.228086] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T18:22:17.519978] [DEBUG] [VALIDATOR ] DQ: All expected trading days are present. [2025-10-20T18:22:17.520066] [DEBUG] [VALIDATOR ] DQ: Primary data validation passed. [2025-10-20T18:22:17.520843] [INFO ] [MAIN ] DQ: Fetched primary data is valid. [2025-10-20T18:22:17.520907] [DEBUG] [LOCAL ] DQ: Saving primary data. [2025-10-20T18:22:17.522180] [DEBUG] [LOCAL ] DQ: Trimmed 63642 records to 43901 (keeping 200 days of data) [2025-10-20T18:22:17.771836] [DEBUG] [LOCAL ] DQ: Primary data saved (43901 records). [2025-10-20T18:22:17.771970] [INFO ] [MAIN ] DQ: Full history saved. [2025-10-20T18:22:17.772305] [INFO ] [MAIN ] Update Progress: 647/2367 symbols completed. DQ took 36.2s. Estimated remaining: 3h 38m 35s. Rate: 7.9 symbols/min [2025-10-20T18:22:17.860467] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-04. [2025-10-20T18:22:17.940978] [INFO ] [MAIN ] DSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:17.973770] [DEBUG] [LOCAL ] DSL: Loaded 29410 existing records from primary storage. [2025-10-20T18:22:17.974055] [INFO ] [MAIN ] DSL: Loaded 29410 existing records. [2025-10-20T18:22:18.151142] [WARN ] [MAIN ] DSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:18.151411] [INFO ] [MAIN ] DSL: Wiping existing primary data before full refresh. [2025-10-20T18:22:18.151467] [INFO ] [LOCAL ] DSL: Starting deletion of primary data. [2025-10-20T18:22:18.151743] [DEBUG] [LOCAL ] DSL: Deleted primary data file. [2025-10-20T18:22:18.151816] [INFO ] [MAIN ] DSL: Starting full primary refresh. [2025-10-20T18:22:18.515216] [INFO ] [REMOTE ] DSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:18.651395] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-01. [2025-10-20T18:22:18.704358] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:22:19.451393] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-05. [2025-10-20T18:22:19.711014] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:22:19.819276] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:22:20.236709] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T18:22:20.251336] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-06. [2025-10-20T18:22:20.810905] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T18:22:21.051374] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-07. [2025-10-20T18:22:21.851397] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-07. [2025-10-20T18:22:22.651379] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-05. [2025-10-20T18:22:22.673806] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T18:22:22.725724] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T18:22:23.451416] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-02. [2025-10-20T18:22:24.251495] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-08. [2025-10-20T18:22:24.341548] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T18:22:24.652148] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:22:24.712354] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T18:22:25.051407] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-06. [2025-10-20T18:22:25.851419] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-08. [2025-10-20T18:22:26.651403] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-09. [2025-10-20T18:22:26.851086] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T18:22:27.237035] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T18:22:27.256946] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T18:22:27.451416] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-10. [2025-10-20T18:22:28.237736] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T18:22:28.239370] [INFO ] [REMOTE ] DRLL: Full fetch complete (14595 records). [2025-10-20T18:22:28.239448] [DEBUG] [VALIDATOR ] DRLL: Validating primary data. [2025-10-20T18:22:28.260060] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-07. [2025-10-20T18:22:28.633035] [DEBUG] [VALIDATOR ] DRLL: All expected trading days are present. [2025-10-20T18:22:28.633118] [DEBUG] [VALIDATOR ] DRLL: Primary data validation passed. [2025-10-20T18:22:28.633465] [INFO ] [MAIN ] DRLL: Fetched primary data is valid. [2025-10-20T18:22:28.633505] [DEBUG] [LOCAL ] DRLL: Saving primary data. [2025-10-20T18:22:28.634167] [DEBUG] [LOCAL ] DRLL: Trimmed 14595 records to 8856 (keeping 200 days of data) [2025-10-20T18:22:28.688513] [DEBUG] [LOCAL ] DRLL: Primary data saved (8856 records). [2025-10-20T18:22:28.688667] [INFO ] [MAIN ] DRLL: Full history saved. [2025-10-20T18:22:28.688765] [INFO ] [MAIN ] Update Progress: 648/2367 symbols completed. DRLL took 19.1s. Estimated remaining: 3h 38m 36s. Rate: 7.9 symbols/min [2025-10-20T18:22:28.868480] [INFO ] [MAIN ] DSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:28.868727] [DEBUG] [LOCAL ] DSM: No existing data file found in primary storage. [2025-10-20T18:22:28.868790] [WARN ] [MAIN ] DSM: No existing data found, performing full refresh instead. [2025-10-20T18:22:28.868924] [INFO ] [MAIN ] DSM: Wiping existing primary data before full refresh. [2025-10-20T18:22:28.868976] [INFO ] [LOCAL ] DSM: Starting deletion of primary data. [2025-10-20T18:22:28.869029] [DEBUG] [LOCAL ] DSM: No primary data file found to delete. [2025-10-20T18:22:28.869070] [INFO ] [MAIN ] DSM: Starting full primary refresh. [2025-10-20T18:22:29.061779] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-09. [2025-10-20T18:22:29.260085] [INFO ] [REMOTE ] DSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:29.851435] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-01. [2025-10-20T18:22:30.009159] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:22:30.651400] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-08. [2025-10-20T18:22:30.977858] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T18:22:31.040589] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T18:22:31.451368] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-03. [2025-10-20T18:22:32.229545] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T18:22:32.251333] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-09. [2025-10-20T18:22:32.296793] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T18:22:33.051380] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-02. [2025-10-20T18:22:33.851397] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-10. [2025-10-20T18:22:34.212167] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T18:22:34.279052] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T18:22:34.651399] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-03. [2025-10-20T18:22:35.274015] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:22:35.281466] [INFO ] [REMOTE ] DRH: Full fetch complete (72007 records). [2025-10-20T18:22:35.281623] [DEBUG] [VALIDATOR ] DRH: Validating primary data. [2025-10-20T18:22:35.456481] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-10. [2025-10-20T18:22:35.672295] [DEBUG] [VALIDATOR ] DRH: All expected trading days are present. [2025-10-20T18:22:35.672374] [DEBUG] [VALIDATOR ] DRH: Primary data validation passed. [2025-10-20T18:22:35.673362] [INFO ] [MAIN ] DRH: Fetched primary data is valid. [2025-10-20T18:22:35.673498] [DEBUG] [LOCAL ] DRH: Saving primary data. [2025-10-20T18:22:35.675038] [DEBUG] [LOCAL ] DRH: Trimmed 72007 records to 50315 (keeping 200 days of data) [2025-10-20T18:22:35.792042] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:22:35.993111] [DEBUG] [LOCAL ] DRH: Primary data saved (50315 records). [2025-10-20T18:22:35.993293] [INFO ] [MAIN ] DRH: Full history saved. [2025-10-20T18:22:35.993402] [INFO ] [MAIN ] Update Progress: 649/2367 symbols completed. DRH took 45.7s. Estimated remaining: 3h 38m 28s. Rate: 7.9 symbols/min [2025-10-20T18:22:36.163889] [INFO ] [MAIN ] DSMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:36.184115] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T18:22:36.193384] [DEBUG] [LOCAL ] DSMC: Loaded 1866 existing records from primary storage. [2025-10-20T18:22:36.193738] [INFO ] [MAIN ] DSMC: Loaded 1866 existing records. [2025-10-20T18:22:36.251526] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-04. [2025-10-20T18:22:36.238370] [INFO ] [REMOTE ] DRI: Full fetch complete (69569 records). [2025-10-20T18:22:36.273313] [DEBUG] [VALIDATOR ] DRI: Validating primary data. [2025-10-20T18:22:36.531059] [WARN ] [MAIN ] DSMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:36.550041] [INFO ] [MAIN ] DSMC: Wiping existing primary data before full refresh. [2025-10-20T18:22:36.550664] [INFO ] [LOCAL ] DSMC: Starting deletion of primary data. [2025-10-20T18:22:36.567301] [DEBUG] [LOCAL ] DSMC: Deleted primary data file. [2025-10-20T18:22:36.587182] [INFO ] [MAIN ] DSMC: Starting full primary refresh. [2025-10-20T18:22:37.057506] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-04. [2025-10-20T18:22:37.071846] [DEBUG] [VALIDATOR ] DRI: All expected trading days are present. [2025-10-20T18:22:37.076062] [DEBUG] [VALIDATOR ] DRI: Primary data validation passed. [2025-10-20T18:22:37.077135] [INFO ] [MAIN ] DRI: Fetched primary data is valid. [2025-10-20T18:22:37.077231] [DEBUG] [LOCAL ] DRI: Saving primary data. [2025-10-20T18:22:37.139859] [DEBUG] [LOCAL ] DRI: Trimmed 69569 records to 47643 (keeping 200 days of data) [2025-10-20T18:22:37.160602] [INFO ] [REMOTE ] DSMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:37.458538] [DEBUG] [LOCAL ] DRI: Primary data saved (47643 records). [2025-10-20T18:22:37.458695] [INFO ] [MAIN ] DRI: Full history saved. [2025-10-20T18:22:37.458796] [INFO ] [MAIN ] Update Progress: 650/2367 symbols completed. DRI took 33.1s. Estimated remaining: 3h 38m 4s. Rate: 7.9 symbols/min [2025-10-20T18:22:37.626797] [INFO ] [MAIN ] DSU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:37.650306] [DEBUG] [LOCAL ] DSU: Loaded 23392 existing records from primary storage. [2025-10-20T18:22:37.650553] [INFO ] [MAIN ] DSU: Loaded 23392 existing records. [2025-10-20T18:22:37.830962] [WARN ] [MAIN ] DSU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:37.832153] [INFO ] [MAIN ] DSU: Wiping existing primary data before full refresh. [2025-10-20T18:22:37.832301] [INFO ] [LOCAL ] DSU: Starting deletion of primary data. [2025-10-20T18:22:37.832740] [DEBUG] [LOCAL ] DSU: Deleted primary data file. [2025-10-20T18:22:37.839543] [INFO ] [MAIN ] DSU: Starting full primary refresh. [2025-10-20T18:22:37.864627] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-01. [2025-10-20T18:22:37.887569] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T18:22:38.043813] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T18:22:38.273922] [INFO ] [REMOTE ] DSU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:38.291997] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-01 (0.43 seconds). [2025-10-20T18:22:38.651378] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-05. [2025-10-20T18:22:39.451371] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-02. [2025-10-20T18:22:39.518057] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T18:22:39.943997] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T18:22:40.251369] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-03. [2025-10-20T18:22:40.722351] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T18:22:41.051366] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-04. [2025-10-20T18:22:41.617550] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T18:22:41.851398] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-05. [2025-10-20T18:22:42.320094] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T18:22:42.651385] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-05. [2025-10-20T18:22:43.451351] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-01. [2025-10-20T18:22:43.710144] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:22:44.251380] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-06. [2025-10-20T18:22:44.503995] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T18:22:44.522260] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-10-20T18:22:45.051384] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-02. [2025-10-20T18:22:45.851380] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-06. [2025-10-20T18:22:46.098014] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:22:46.651351] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-03. [2025-10-20T18:22:46.727317] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T18:22:47.451362] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-07. [2025-10-20T18:22:47.657262] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T18:22:48.251378] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-07. [2025-10-20T18:22:48.365084] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T18:22:48.905276] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T18:22:49.051366] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-08. [2025-10-20T18:22:49.851390] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-04. [2025-10-20T18:22:49.915886] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T18:22:50.651378] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-06. [2025-10-20T18:22:51.013161] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T18:22:51.451379] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-05. [2025-10-20T18:22:51.860526] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T18:22:52.251378] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-08. [2025-10-20T18:22:53.051372] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-09. [2025-10-20T18:22:53.851371] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-07. [2025-10-20T18:23:04.324247] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-08 (12.07 seconds). [2025-10-20T18:23:04.325992] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-09. [2025-10-20T18:23:05.688476] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-05 (14.24 seconds). [2025-10-20T18:23:05.692890] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-06. [2025-10-20T18:23:05.833279] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-09 (12.78 seconds). [2025-10-20T18:23:05.836876] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-10. [2025-10-20T18:23:06.400805] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T18:23:06.402073] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-10. [2025-10-20T18:23:06.510179] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-07 (12.66 seconds). [2025-10-20T18:23:06.518209] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-08. [2025-10-20T18:23:07.330632] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T18:23:07.331482] [INFO ] [REMOTE ] DSMC: Full fetch complete (2870 records). [2025-10-20T18:23:07.331549] [DEBUG] [VALIDATOR ] DSMC: Validating primary data. [2025-10-20T18:23:07.628417] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T18:23:07.652078] [INFO ] [REMOTE ] DSM: Full fetch complete (23854 records). [2025-10-20T18:23:07.682932] [DEBUG] [VALIDATOR ] DSM: Validating primary data. [2025-10-20T18:23:07.715922] [DEBUG] [VALIDATOR ] DSMC: All expected trading days are present. [2025-10-20T18:23:07.759131] [DEBUG] [VALIDATOR ] DSMC: Primary data validation passed. [2025-10-20T18:23:07.772544] [INFO ] [MAIN ] DSMC: Fetched primary data is valid. [2025-10-20T18:23:07.795966] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-10-20T18:23:07.795052] [DEBUG] [LOCAL ] DSMC: Saving primary data. [2025-10-20T18:23:07.828474] [DEBUG] [LOCAL ] DSMC: Trimmed 2870 records to 1838 (keeping 200 days of data) [2025-10-20T18:23:07.817948] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-07. [2025-10-20T18:23:07.891886] [DEBUG] [LOCAL ] DSMC: Primary data saved (1838 records). [2025-10-20T18:23:07.908939] [INFO ] [MAIN ] DSMC: Full history saved. [2025-10-20T18:23:07.914190] [INFO ] [MAIN ] Update Progress: 651/2367 symbols completed. DSMC took 31.9s. Estimated remaining: 3h 38m 56s. Rate: 7.8 symbols/min [2025-10-20T18:23:08.293362] [INFO ] [MAIN ] DSX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:08.380589] [DEBUG] [VALIDATOR ] DSM: All expected trading days are present. [2025-10-20T18:23:08.380704] [DEBUG] [VALIDATOR ] DSM: Primary data validation passed. [2025-10-20T18:23:08.381173] [INFO ] [MAIN ] DSM: Fetched primary data is valid. [2025-10-20T18:23:08.386729] [DEBUG] [LOCAL ] DSM: Saving primary data. [2025-10-20T18:23:08.387777] [DEBUG] [LOCAL ] DSM: Trimmed 23854 records to 14073 (keeping 200 days of data) [2025-10-20T18:23:08.460694] [DEBUG] [LOCAL ] DSX: Loaded 24294 existing records from primary storage. [2025-10-20T18:23:08.466388] [INFO ] [MAIN ] DSX: Loaded 24294 existing records. [2025-10-20T18:23:08.584773] [DEBUG] [LOCAL ] DSM: Primary data saved (14073 records). [2025-10-20T18:23:08.590418] [INFO ] [MAIN ] DSM: Full history saved. [2025-10-20T18:23:08.606091] [INFO ] [MAIN ] Update Progress: 652/2367 symbols completed. DSM took 39.9s. Estimated remaining: 3h 38m 30s. Rate: 7.8 symbols/min [2025-10-20T18:23:08.679677] [WARN ] [MAIN ] DSX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:08.699833] [INFO ] [MAIN ] DSX: Wiping existing primary data before full refresh. [2025-10-20T18:23:08.731683] [INFO ] [LOCAL ] DSX: Starting deletion of primary data. [2025-10-20T18:23:08.774453] [DEBUG] [LOCAL ] DSX: Deleted primary data file. [2025-10-20T18:23:08.774854] [INFO ] [MAIN ] DSX: Starting full primary refresh. [2025-10-20T18:23:08.850238] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-08 (2.33 seconds). [2025-10-20T18:23:08.982210] [INFO ] [MAIN ] DT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:08.985986] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-09. [2025-10-20T18:23:08.986253] [DEBUG] [LOCAL ] DT: No existing data file found in primary storage. [2025-10-20T18:23:08.993142] [WARN ] [MAIN ] DT: No existing data found, performing full refresh instead. [2025-10-20T18:23:09.081664] [INFO ] [MAIN ] DT: Wiping existing primary data before full refresh. [2025-10-20T18:23:09.106856] [INFO ] [LOCAL ] DT: Starting deletion of primary data. [2025-10-20T18:23:09.135006] [DEBUG] [LOCAL ] DT: No primary data file found to delete. [2025-10-20T18:23:09.146678] [INFO ] [MAIN ] DT: Starting full primary refresh. [2025-10-20T18:23:09.229326] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:23:09.314019] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-08. [2025-10-20T18:23:09.388797] [INFO ] [REMOTE ] DSX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:09.410668] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-01. [2025-10-20T18:23:09.669839] [INFO ] [REMOTE ] DT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:09.670248] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-01. [2025-10-20T18:23:10.321391] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T18:23:10.328606] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-10. [2025-10-20T18:23:10.499910] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T18:23:10.505018] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-09. [2025-10-20T18:23:10.580452] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:23:10.585935] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-02. [2025-10-20T18:23:11.375963] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T18:23:11.382023] [INFO ] [REMOTE ] DSL: Full fetch complete (41732 records). [2025-10-20T18:23:11.382177] [DEBUG] [VALIDATOR ] DSL: Validating primary data. [2025-10-20T18:23:11.576704] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:23:11.637164] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-10. [2025-10-20T18:23:11.725619] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:23:11.791520] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-03. [2025-10-20T18:23:11.917635] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-01 (2.25 seconds). [2025-10-20T18:23:11.946203] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-02. [2025-10-20T18:23:11.947153] [DEBUG] [VALIDATOR ] DSL: All expected trading days are present. [2025-10-20T18:23:11.947215] [DEBUG] [VALIDATOR ] DSL: Primary data validation passed. [2025-10-20T18:23:11.947881] [INFO ] [MAIN ] DSL: Fetched primary data is valid. [2025-10-20T18:23:11.948020] [DEBUG] [LOCAL ] DSL: Saving primary data. [2025-10-20T18:23:11.949142] [DEBUG] [LOCAL ] DSL: Trimmed 41732 records to 29017 (keeping 200 days of data) [2025-10-20T18:23:12.135051] [DEBUG] [LOCAL ] DSL: Primary data saved (29017 records). [2025-10-20T18:23:12.135195] [INFO ] [MAIN ] DSL: Full history saved. [2025-10-20T18:23:12.135628] [INFO ] [MAIN ] Update Progress: 653/2367 symbols completed. DSL took 54.4s. Estimated remaining: 3h 38m 12s. Rate: 7.9 symbols/min [2025-10-20T18:23:12.340374] [INFO ] [MAIN ] DTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:12.349544] [DEBUG] [LOCAL ] DTB: Loaded 5028 existing records from primary storage. [2025-10-20T18:23:12.349799] [INFO ] [MAIN ] DTB: Loaded 5028 existing records. [2025-10-20T18:23:12.613256] [WARN ] [MAIN ] DTB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:12.613476] [INFO ] [MAIN ] DTB: Wiping existing primary data before full refresh. [2025-10-20T18:23:12.613545] [INFO ] [LOCAL ] DTB: Starting deletion of primary data. [2025-10-20T18:23:12.613741] [DEBUG] [LOCAL ] DTB: Deleted primary data file. [2025-10-20T18:23:12.613798] [INFO ] [MAIN ] DTB: Starting full primary refresh. [2025-10-20T18:23:13.014918] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T18:23:13.028967] [INFO ] [REMOTE ] DTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:13.029341] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-01. [2025-10-20T18:23:13.034416] [INFO ] [REMOTE ] DSU: Full fetch complete (32975 records). [2025-10-20T18:23:13.034569] [DEBUG] [VALIDATOR ] DSU: Validating primary data. [2025-10-20T18:23:13.035189] [FATAL] [VALIDATOR ] DSU: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:23:13.035275] [FATAL] [MAIN ] DSU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:23:13.035340] [WARN ] [MAIN ] DSU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:23:13.035410] [INFO ] [LOCAL ] DSU: Starting deletion of primary data. [2025-10-20T18:23:13.035475] [DEBUG] [LOCAL ] DSU: No primary data file found to delete. [2025-10-20T18:23:13.035511] [INFO ] [LOCAL ] DSU: Starting deletion of derivative data. [2025-10-20T18:23:13.035691] [DEBUG] [LOCAL ] DSU: Deleted derivative data file. [2025-10-20T18:23:13.035784] [INFO ] [MAIN ] Update Progress: 654/2367 symbols completed. DSU took 35.6s. Estimated remaining: 3h 37m 47s. Rate: 7.9 symbols/min [2025-10-20T18:23:13.217026] [INFO ] [MAIN ] DTE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:13.236363] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T18:23:13.243151] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-04. [2025-10-20T18:23:13.288747] [DEBUG] [LOCAL ] DTE: Loaded 48020 existing records from primary storage. [2025-10-20T18:23:13.289111] [INFO ] [MAIN ] DTE: Loaded 48020 existing records. [2025-10-20T18:23:13.486049] [WARN ] [MAIN ] DTE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:13.486374] [INFO ] [MAIN ] DTE: Wiping existing primary data before full refresh. [2025-10-20T18:23:13.486423] [INFO ] [LOCAL ] DTE: Starting deletion of primary data. [2025-10-20T18:23:13.513084] [DEBUG] [LOCAL ] DTE: Deleted primary data file. [2025-10-20T18:23:13.513314] [INFO ] [MAIN ] DTE: Starting full primary refresh. [2025-10-20T18:23:13.563000] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T18:23:13.642835] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-03. [2025-10-20T18:23:13.796263] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:23:13.827025] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-02. [2025-10-20T18:23:13.921945] [INFO ] [REMOTE ] DTE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:13.922295] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-01. [2025-10-20T18:23:14.300371] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T18:23:14.306457] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-05. [2025-10-20T18:23:14.330863] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T18:23:14.332515] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-03. [2025-10-20T18:23:14.869272] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T18:23:14.871408] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-04. [2025-10-20T18:23:15.164528] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:23:15.176731] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-04. [2025-10-20T18:23:15.380448] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T18:23:15.390118] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-06. [2025-10-20T18:23:15.504100] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T18:23:15.506601] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-05. [2025-10-20T18:23:15.718478] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T18:23:16.114326] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T18:23:16.251346] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-06. [2025-10-20T18:23:16.329338] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:23:16.731464] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T18:23:16.983433] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T18:23:17.051420] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-05. [2025-10-20T18:23:17.851396] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-02. [2025-10-20T18:23:18.597438] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:23:18.651395] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-06. [2025-10-20T18:23:19.244394] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T18:23:19.451422] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-07. [2025-10-20T18:23:20.056281] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T18:23:20.251392] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-08. [2025-10-20T18:23:20.287640] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:23:20.900529] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T18:23:21.051410] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-03. [2025-10-20T18:23:21.851398] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-09. [2025-10-20T18:23:22.557034] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T18:23:22.559168] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T18:23:22.651377] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-10. [2025-10-20T18:23:23.258112] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T18:23:23.259406] [INFO ] [REMOTE ] DTB: Full fetch complete (6567 records). [2025-10-20T18:23:23.259476] [DEBUG] [VALIDATOR ] DTB: Validating primary data. [2025-10-20T18:23:23.456679] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-04. [2025-10-20T18:23:23.638509] [DEBUG] [VALIDATOR ] DTB: All expected trading days are present. [2025-10-20T18:23:23.638610] [DEBUG] [VALIDATOR ] DTB: Primary data validation passed. [2025-10-20T18:23:23.638843] [INFO ] [MAIN ] DTB: Fetched primary data is valid. [2025-10-20T18:23:23.638898] [DEBUG] [LOCAL ] DTB: Saving primary data. [2025-10-20T18:23:23.639364] [DEBUG] [LOCAL ] DTB: Trimmed 6567 records to 4937 (keeping 200 days of data) [2025-10-20T18:23:23.669637] [DEBUG] [LOCAL ] DTB: Primary data saved (4937 records). [2025-10-20T18:23:23.669798] [INFO ] [MAIN ] DTB: Full history saved. [2025-10-20T18:23:23.669880] [INFO ] [MAIN ] Update Progress: 655/2367 symbols completed. DTB took 11.5s. Estimated remaining: 3h 37m 47s. Rate: 7.9 symbols/min [2025-10-20T18:23:23.840441] [INFO ] [MAIN ] DTF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:23.840694] [DEBUG] [LOCAL ] DTF: No existing data file found in primary storage. [2025-10-20T18:23:23.840751] [WARN ] [MAIN ] DTF: No existing data found, performing full refresh instead. [2025-10-20T18:23:23.840832] [INFO ] [MAIN ] DTF: Wiping existing primary data before full refresh. [2025-10-20T18:23:23.840867] [INFO ] [LOCAL ] DTF: Starting deletion of primary data. [2025-10-20T18:23:23.840906] [DEBUG] [LOCAL ] DTF: No primary data file found to delete. [2025-10-20T18:23:23.840936] [INFO ] [MAIN ] DTF: Starting full primary refresh. [2025-10-20T18:23:24.238369] [INFO ] [REMOTE ] DTF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:24.251362] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-01. [2025-10-20T18:23:24.773898] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T18:23:25.051400] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-02. [2025-10-20T18:23:25.307334] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:23:25.359582] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-10-20T18:23:25.851364] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-03. [2025-10-20T18:23:26.278477] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-10-20T18:23:26.651417] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-05. [2025-10-20T18:23:27.451429] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-07. [2025-10-20T18:23:28.251352] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-07. [2025-10-20T18:23:28.349795] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:23:28.633370] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T18:23:29.051440] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-04. [2025-10-20T18:23:29.368988] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-04 (0.32 seconds). [2025-10-20T18:23:29.851406] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-08. [2025-10-20T18:23:30.221290] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T18:23:30.651424] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-05. [2025-10-20T18:23:30.956247] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-10-20T18:23:30.959312] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T18:23:31.451488] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-06. [2025-10-20T18:23:32.251426] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-06. [2025-10-20T18:23:32.655157] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-10-20T18:23:33.061446] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-09. [2025-10-20T18:23:33.067475] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:23:33.851407] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-07. [2025-10-20T18:23:33.980764] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T18:23:34.195247] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T18:23:34.651417] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-07. [2025-10-20T18:23:35.451423] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-08. [2025-10-20T18:23:36.224466] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T18:23:36.251352] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-08. [2025-10-20T18:23:36.907939] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:23:37.051400] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-10. [2025-10-20T18:23:37.663717] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T18:23:37.851396] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-09. [2025-10-20T18:23:38.332518] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:23:38.336538] [INFO ] [REMOTE ] DSX: Full fetch complete (35765 records). [2025-10-20T18:23:38.336624] [DEBUG] [VALIDATOR ] DSX: Validating primary data. [2025-10-20T18:23:38.656513] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-08. [2025-10-20T18:23:38.703163] [DEBUG] [VALIDATOR ] DSX: All expected trading days are present. [2025-10-20T18:23:38.703236] [DEBUG] [VALIDATOR ] DSX: Primary data validation passed. [2025-10-20T18:23:38.703887] [INFO ] [MAIN ] DSX: Fetched primary data is valid. [2025-10-20T18:23:38.703951] [DEBUG] [LOCAL ] DSX: Saving primary data. [2025-10-20T18:23:38.704731] [DEBUG] [LOCAL ] DSX: Trimmed 35765 records to 23744 (keeping 200 days of data) [2025-10-20T18:23:38.836451] [DEBUG] [LOCAL ] DSX: Primary data saved (23744 records). [2025-10-20T18:23:38.836572] [INFO ] [MAIN ] DSX: Full history saved. [2025-10-20T18:23:38.836666] [INFO ] [MAIN ] Update Progress: 656/2367 symbols completed. DSX took 30.9s. Estimated remaining: 3h 37m 59s. Rate: 7.8 symbols/min [2025-10-20T18:23:39.006550] [INFO ] [MAIN ] DTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:39.014561] [DEBUG] [LOCAL ] DTG: Loaded 5859 existing records from primary storage. [2025-10-20T18:23:39.014756] [INFO ] [MAIN ] DTG: Loaded 5859 existing records. [2025-10-20T18:23:39.195516] [WARN ] [MAIN ] DTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:39.195801] [INFO ] [MAIN ] DTG: Wiping existing primary data before full refresh. [2025-10-20T18:23:39.195890] [INFO ] [LOCAL ] DTG: Starting deletion of primary data. [2025-10-20T18:23:39.196130] [DEBUG] [LOCAL ] DTG: Deleted primary data file. [2025-10-20T18:23:39.196200] [INFO ] [MAIN ] DTG: Starting full primary refresh. [2025-10-20T18:23:39.295990] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T18:23:39.466560] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-09. [2025-10-20T18:23:39.585554] [INFO ] [REMOTE ] DTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:39.615527] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T18:23:40.251364] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-01. [2025-10-20T18:23:40.941988] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-10-20T18:23:41.051358] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-02. [2025-10-20T18:23:41.268672] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T18:23:41.663749] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T18:23:41.851378] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-10. [2025-10-20T18:23:42.651514] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-09. [2025-10-20T18:23:43.055016] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T18:23:43.454513] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-03. [2025-10-20T18:23:43.470246] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T18:23:43.481896] [INFO ] [REMOTE ] DT: Full fetch complete (76903 records). [2025-10-20T18:23:43.482061] [DEBUG] [VALIDATOR ] DT: Validating primary data. [2025-10-20T18:23:43.881731] [DEBUG] [VALIDATOR ] DT: All expected trading days are present. [2025-10-20T18:23:43.881825] [DEBUG] [VALIDATOR ] DT: Primary data validation passed. [2025-10-20T18:23:43.882767] [INFO ] [MAIN ] DT: Fetched primary data is valid. [2025-10-20T18:23:43.882836] [DEBUG] [LOCAL ] DT: Saving primary data. [2025-10-20T18:23:43.884291] [DEBUG] [LOCAL ] DT: Trimmed 76903 records to 52952 (keeping 200 days of data) [2025-10-20T18:23:44.188348] [DEBUG] [LOCAL ] DT: Primary data saved (52952 records). [2025-10-20T18:23:44.188498] [INFO ] [MAIN ] DT: Full history saved. [2025-10-20T18:23:44.188583] [INFO ] [MAIN ] Update Progress: 657/2367 symbols completed. DT took 35.6s. Estimated remaining: 3h 37m 45s. Rate: 7.9 symbols/min [2025-10-20T18:23:44.252287] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-10. [2025-10-20T18:23:44.302138] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T18:23:44.379079] [INFO ] [MAIN ] DTK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:44.379264] [DEBUG] [LOCAL ] DTK: No existing data file found in primary storage. [2025-10-20T18:23:44.379314] [WARN ] [MAIN ] DTK: No existing data found, performing full refresh instead. [2025-10-20T18:23:44.379423] [INFO ] [MAIN ] DTK: Wiping existing primary data before full refresh. [2025-10-20T18:23:44.379466] [INFO ] [LOCAL ] DTK: Starting deletion of primary data. [2025-10-20T18:23:44.379507] [DEBUG] [LOCAL ] DTK: No primary data file found to delete. [2025-10-20T18:23:44.379540] [INFO ] [MAIN ] DTK: Starting full primary refresh. [2025-10-20T18:23:44.729061] [INFO ] [REMOTE ] DTK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:44.752737] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T18:23:44.753854] [INFO ] [REMOTE ] DTF: Full fetch complete (1750 records). [2025-10-20T18:23:44.753947] [DEBUG] [VALIDATOR ] DTF: Validating primary data. [2025-10-20T18:23:44.948352] [WARN ] [VALIDATOR ] DTF: Validation failed - insufficient historical data. Have 197 unique days, need 200 days [2025-10-20T18:23:44.948533] [FATAL] [MAIN ] DTF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:23:44.948599] [WARN ] [MAIN ] DTF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:23:44.948691] [INFO ] [LOCAL ] DTF: Starting deletion of primary data. [2025-10-20T18:23:44.948757] [DEBUG] [LOCAL ] DTF: No primary data file found to delete. [2025-10-20T18:23:44.948792] [INFO ] [LOCAL ] DTF: Starting deletion of derivative data. [2025-10-20T18:23:44.948831] [DEBUG] [LOCAL ] DTF: No derivative data file found to delete. [2025-10-20T18:23:44.948883] [INFO ] [MAIN ] Update Progress: 658/2367 symbols completed. DTF took 21.3s. Estimated remaining: 3h 37m 20s. Rate: 7.9 symbols/min [2025-10-20T18:23:45.056533] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-04. [2025-10-20T18:23:45.121587] [INFO ] [MAIN ] DTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:45.162416] [DEBUG] [LOCAL ] DTM: Loaded 44538 existing records from primary storage. [2025-10-20T18:23:45.162764] [INFO ] [MAIN ] DTM: Loaded 44538 existing records. [2025-10-20T18:23:45.345636] [WARN ] [MAIN ] DTM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:45.345818] [INFO ] [MAIN ] DTM: Wiping existing primary data before full refresh. [2025-10-20T18:23:45.345866] [INFO ] [LOCAL ] DTM: Starting deletion of primary data. [2025-10-20T18:23:45.346331] [DEBUG] [LOCAL ] DTM: Deleted primary data file. [2025-10-20T18:23:45.346387] [INFO ] [MAIN ] DTM: Starting full primary refresh. [2025-10-20T18:23:45.715100] [INFO ] [REMOTE ] DTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:45.721045] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T18:23:45.851487] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-10. [2025-10-20T18:23:46.348819] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T18:23:46.355537] [INFO ] [REMOTE ] DTE: Full fetch complete (68927 records). [2025-10-20T18:23:46.355723] [DEBUG] [VALIDATOR ] DTE: Validating primary data. [2025-10-20T18:23:46.653100] [INFO ] [REMOTE ] DTK: Fetching primary data for 2025-01. [2025-10-20T18:23:46.746708] [DEBUG] [VALIDATOR ] DTE: All expected trading days are present. [2025-10-20T18:23:46.746804] [DEBUG] [VALIDATOR ] DTE: Primary data validation passed. [2025-10-20T18:23:46.747603] [INFO ] [MAIN ] DTE: Fetched primary data is valid. [2025-10-20T18:23:46.747665] [DEBUG] [LOCAL ] DTE: Saving primary data. [2025-10-20T18:23:46.749068] [DEBUG] [LOCAL ] DTE: Trimmed 68927 records to 47244 (keeping 200 days of data) [2025-10-20T18:23:46.927089] [DEBUG] [REMOTE ] DTK: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:23:46.936309] [WARN ] [REMOTE ] DTK: No data returned for month 2025-01 [2025-10-20T18:23:46.936449] [FATAL] [REMOTE ] DTK: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:23:46.936500] [FATAL] [MAIN ] DTK: Failed to fetch primary data. [2025-10-20T18:23:46.965733] [WARN ] [MAIN ] DTK: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:23:46.969379] [INFO ] [LOCAL ] DTK: Starting deletion of primary data. [2025-10-20T18:23:46.969741] [DEBUG] [LOCAL ] DTK: No primary data file found to delete. [2025-10-20T18:23:46.970496] [INFO ] [LOCAL ] DTK: Starting deletion of derivative data. [2025-10-20T18:23:46.973333] [DEBUG] [LOCAL ] DTK: No derivative data file found to delete. [2025-10-20T18:23:46.975772] [INFO ] [MAIN ] Update Progress: 659/2367 symbols completed. DTK took 2.8s. Estimated remaining: 3h 36m 58s. Rate: 7.9 symbols/min [2025-10-20T18:23:47.172239] [INFO ] [MAIN ] DTW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:47.228500] [DEBUG] [LOCAL ] DTW: Loaded 4872 existing records from primary storage. [2025-10-20T18:23:47.229853] [INFO ] [MAIN ] DTW: Loaded 4872 existing records. [2025-10-20T18:23:47.318371] [DEBUG] [LOCAL ] DTE: Primary data saved (47244 records). [2025-10-20T18:23:47.340398] [INFO ] [MAIN ] DTE: Full history saved. [2025-10-20T18:23:47.340576] [INFO ] [MAIN ] Update Progress: 660/2367 symbols completed. DTE took 34.3s. Estimated remaining: 3h 36m 31s. Rate: 7.9 symbols/min [2025-10-20T18:23:47.479144] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-01. [2025-10-20T18:23:47.529746] [WARN ] [MAIN ] DTW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:47.530775] [INFO ] [MAIN ] DTW: Wiping existing primary data before full refresh. [2025-10-20T18:23:47.530841] [INFO ] [LOCAL ] DTW: Starting deletion of primary data. [2025-10-20T18:23:47.541925] [DEBUG] [LOCAL ] DTW: Deleted primary data file. [2025-10-20T18:23:47.547211] [INFO ] [MAIN ] DTW: Starting full primary refresh. [2025-10-20T18:23:47.718288] [INFO ] [MAIN ] DUK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:48.000516] [INFO ] [REMOTE ] DTW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:48.017856] [DEBUG] [LOCAL ] DUK: Loaded 54217 existing records from primary storage. [2025-10-20T18:23:48.018131] [INFO ] [MAIN ] DUK: Loaded 54217 existing records. [2025-10-20T18:23:48.197807] [WARN ] [MAIN ] DUK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:48.198021] [INFO ] [MAIN ] DUK: Wiping existing primary data before full refresh. [2025-10-20T18:23:48.198077] [INFO ] [LOCAL ] DUK: Starting deletion of primary data. [2025-10-20T18:23:48.198274] [DEBUG] [LOCAL ] DUK: Deleted primary data file. [2025-10-20T18:23:48.198329] [INFO ] [MAIN ] DUK: Starting full primary refresh. [2025-10-20T18:23:48.263025] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-01. [2025-10-20T18:23:48.556442] [INFO ] [REMOTE ] DUK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:49.051387] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-05. [2025-10-20T18:23:49.075697] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T18:23:49.263579] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T18:23:49.470398] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T18:23:49.851476] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-02. [2025-10-20T18:23:49.955722] [DEBUG] [REMOTE ] DTW: Error fetching primary data for 2025-02: 'API call returned an empty response.' [2025-10-20T18:23:49.956027] [WARN ] [REMOTE ] DTW: No data returned for month 2025-02 [2025-10-20T18:23:49.956144] [FATAL] [REMOTE ] DTW: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:23:49.956262] [FATAL] [MAIN ] DTW: Failed to fetch primary data. [2025-10-20T18:23:49.956347] [WARN ] [MAIN ] DTW: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:23:49.956434] [INFO ] [LOCAL ] DTW: Starting deletion of primary data. [2025-10-20T18:23:49.956517] [DEBUG] [LOCAL ] DTW: No primary data file found to delete. [2025-10-20T18:23:49.956572] [INFO ] [LOCAL ] DTW: Starting deletion of derivative data. [2025-10-20T18:23:49.956772] [DEBUG] [LOCAL ] DTW: Deleted derivative data file. [2025-10-20T18:23:49.956856] [INFO ] [MAIN ] Update Progress: 661/2367 symbols completed. DTW took 3.0s. Estimated remaining: 3h 36m 11s. Rate: 7.9 symbols/min [2025-10-20T18:23:50.136076] [INFO ] [MAIN ] DUKB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:50.143595] [DEBUG] [LOCAL ] DUKB: Loaded 6333 existing records from primary storage. [2025-10-20T18:23:50.143851] [INFO ] [MAIN ] DUKB: Loaded 6333 existing records. [2025-10-20T18:23:50.323385] [WARN ] [MAIN ] DUKB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:50.323591] [INFO ] [MAIN ] DUKB: Wiping existing primary data before full refresh. [2025-10-20T18:23:50.323684] [INFO ] [LOCAL ] DUKB: Starting deletion of primary data. [2025-10-20T18:23:50.323863] [DEBUG] [LOCAL ] DUKB: Deleted primary data file. [2025-10-20T18:23:50.323922] [INFO ] [MAIN ] DUKB: Starting full primary refresh. [2025-10-20T18:23:50.661667] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-01. [2025-10-20T18:23:50.682371] [INFO ] [REMOTE ] DUKB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:51.451428] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-02. [2025-10-20T18:23:52.251408] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-06. [2025-10-20T18:23:52.505098] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:23:52.590039] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:23:52.723630] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T18:23:53.051364] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-02. [2025-10-20T18:23:53.851428] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-01. [2025-10-20T18:23:54.651391] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-07. [2025-10-20T18:23:54.831880] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T18:23:54.847179] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T18:23:55.003912] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T18:23:55.451416] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-03. [2025-10-20T18:23:56.251384] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-02. [2025-10-20T18:23:56.851912] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T18:23:57.051430] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-03. [2025-10-20T18:23:57.454367] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T18:23:57.851415] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-04. [2025-10-20T18:23:58.651376] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-03. [2025-10-20T18:23:58.745903] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T18:23:59.225198] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T18:23:59.451377] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-04. [2025-10-20T18:23:59.937343] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-04 (2.09 seconds). [2025-10-20T18:24:00.251364] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-05. [2025-10-20T18:24:01.051438] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-04. [2025-10-20T18:24:01.581304] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-10-20T18:24:01.851366] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-08. [2025-10-20T18:24:01.892274] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T18:24:02.145372] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:24:02.497810] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T18:24:02.651421] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-05. [2025-10-20T18:24:03.202079] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T18:24:03.451381] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-09. [2025-10-20T18:24:04.117029] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T18:24:04.251382] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-10. [2025-10-20T18:24:04.966445] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T18:24:04.968463] [INFO ] [REMOTE ] DTG: Full fetch complete (8608 records). [2025-10-20T18:24:04.968551] [DEBUG] [VALIDATOR ] DTG: Validating primary data. [2025-10-20T18:24:05.056464] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-06. [2025-10-20T18:24:05.357870] [DEBUG] [VALIDATOR ] DTG: All expected trading days are present. [2025-10-20T18:24:05.357946] [DEBUG] [VALIDATOR ] DTG: Primary data validation passed. [2025-10-20T18:24:05.358147] [INFO ] [MAIN ] DTG: Fetched primary data is valid. [2025-10-20T18:24:05.358186] [DEBUG] [LOCAL ] DTG: Saving primary data. [2025-10-20T18:24:05.358575] [DEBUG] [LOCAL ] DTG: Trimmed 8608 records to 5735 (keeping 200 days of data) [2025-10-20T18:24:05.392281] [DEBUG] [LOCAL ] DTG: Primary data saved (5735 records). [2025-10-20T18:24:05.392399] [INFO ] [MAIN ] DTG: Full history saved. [2025-10-20T18:24:05.392479] [INFO ] [MAIN ] Update Progress: 662/2367 symbols completed. DTG took 26.6s. Estimated remaining: 3h 36m 23s. Rate: 7.9 symbols/min [2025-10-20T18:24:05.562809] [INFO ] [MAIN ] DV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:05.613597] [DEBUG] [LOCAL ] DV: Loaded 52038 existing records from primary storage. [2025-10-20T18:24:05.613928] [INFO ] [MAIN ] DV: Loaded 52038 existing records. [2025-10-20T18:24:05.793450] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T18:24:05.797785] [WARN ] [MAIN ] DV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:05.800361] [INFO ] [MAIN ] DV: Wiping existing primary data before full refresh. [2025-10-20T18:24:05.800706] [INFO ] [LOCAL ] DV: Starting deletion of primary data. [2025-10-20T18:24:05.800910] [DEBUG] [LOCAL ] DV: Deleted primary data file. [2025-10-20T18:24:05.800966] [INFO ] [MAIN ] DV: Starting full primary refresh. [2025-10-20T18:24:05.856580] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-07. [2025-10-20T18:24:06.170499] [INFO ] [REMOTE ] DV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:06.390501] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T18:24:06.651419] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-08. [2025-10-20T18:24:07.332454] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T18:24:07.451378] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-05. [2025-10-20T18:24:08.251443] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-06. [2025-10-20T18:24:09.051455] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-01. [2025-10-20T18:24:09.120435] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:24:09.851350] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-09. [2025-10-20T18:24:09.967592] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:24:10.452776] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T18:24:10.651357] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-10. [2025-10-20T18:24:10.752472] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:24:11.121575] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T18:24:11.123057] [INFO ] [REMOTE ] DUKB: Full fetch complete (9060 records). [2025-10-20T18:24:11.123143] [DEBUG] [VALIDATOR ] DUKB: Validating primary data. [2025-10-20T18:24:11.456522] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-07. [2025-10-20T18:24:11.532788] [DEBUG] [VALIDATOR ] DUKB: All expected trading days are present. [2025-10-20T18:24:11.532875] [DEBUG] [VALIDATOR ] DUKB: Primary data validation passed. [2025-10-20T18:24:11.533125] [INFO ] [MAIN ] DUKB: Fetched primary data is valid. [2025-10-20T18:24:11.533168] [DEBUG] [LOCAL ] DUKB: Saving primary data. [2025-10-20T18:24:11.533725] [DEBUG] [LOCAL ] DUKB: Trimmed 9060 records to 6196 (keeping 200 days of data) [2025-10-20T18:24:11.571669] [DEBUG] [LOCAL ] DUKB: Primary data saved (6196 records). [2025-10-20T18:24:11.571816] [INFO ] [MAIN ] DUKB: Full history saved. [2025-10-20T18:24:11.571897] [INFO ] [MAIN ] Update Progress: 663/2367 symbols completed. DUKB took 21.6s. Estimated remaining: 3h 36m 12s. Rate: 7.9 symbols/min [2025-10-20T18:24:11.738999] [INFO ] [MAIN ] DVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:11.781411] [DEBUG] [LOCAL ] DVA: Loaded 41554 existing records from primary storage. [2025-10-20T18:24:11.781757] [INFO ] [MAIN ] DVA: Loaded 41554 existing records. [2025-10-20T18:24:11.961202] [WARN ] [MAIN ] DVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:11.961495] [INFO ] [MAIN ] DVA: Wiping existing primary data before full refresh. [2025-10-20T18:24:11.961557] [INFO ] [LOCAL ] DVA: Starting deletion of primary data. [2025-10-20T18:24:11.961860] [DEBUG] [LOCAL ] DVA: Deleted primary data file. [2025-10-20T18:24:11.961931] [INFO ] [MAIN ] DVA: Starting full primary refresh. [2025-10-20T18:24:12.259466] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-06. [2025-10-20T18:24:12.391101] [INFO ] [REMOTE ] DVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:13.051425] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-02. [2025-10-20T18:24:13.591504] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-10-20T18:24:13.631020] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:24:13.851401] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-07. [2025-10-20T18:24:14.599708] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T18:24:14.651379] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-03. [2025-10-20T18:24:15.367146] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T18:24:15.451374] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-08. [2025-10-20T18:24:16.250984] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T18:24:16.261903] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-08. [2025-10-20T18:24:16.912704] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:24:17.051438] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-01. [2025-10-20T18:24:17.851412] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-09. [2025-10-20T18:24:18.169136] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T18:24:18.434370] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T18:24:18.651452] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-04. [2025-10-20T18:24:19.443119] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:24:19.451672] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-02. [2025-10-20T18:24:20.253375] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-10. [2025-10-20T18:24:20.259735] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T18:24:21.032180] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:24:21.051338] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-03. [2025-10-20T18:24:21.851341] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-09. [2025-10-20T18:24:22.009674] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T18:24:22.016624] [INFO ] [REMOTE ] DTM: Full fetch complete (65167 records). [2025-10-20T18:24:22.016798] [DEBUG] [VALIDATOR ] DTM: Validating primary data. [2025-10-20T18:24:22.404813] [DEBUG] [VALIDATOR ] DTM: All expected trading days are present. [2025-10-20T18:24:22.404941] [DEBUG] [VALIDATOR ] DTM: Primary data validation passed. [2025-10-20T18:24:22.405746] [INFO ] [MAIN ] DTM: Fetched primary data is valid. [2025-10-20T18:24:22.405805] [DEBUG] [LOCAL ] DTM: Saving primary data. [2025-10-20T18:24:22.407025] [DEBUG] [LOCAL ] DTM: Trimmed 65167 records to 43821 (keeping 200 days of data) [2025-10-20T18:24:22.536824] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:24:22.657842] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-04. [2025-10-20T18:24:22.715418] [DEBUG] [LOCAL ] DTM: Primary data saved (43821 records). [2025-10-20T18:24:22.715554] [INFO ] [MAIN ] DTM: Full history saved. [2025-10-20T18:24:22.715924] [INFO ] [MAIN ] Update Progress: 664/2367 symbols completed. DTM took 37.8s. Estimated remaining: 3h 36m 13s. Rate: 7.9 symbols/min [2025-10-20T18:24:22.882392] [INFO ] [MAIN ] DVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:22.882555] [DEBUG] [LOCAL ] DVN: No existing data file found in primary storage. [2025-10-20T18:24:22.882591] [WARN ] [MAIN ] DVN: No existing data found, performing full refresh instead. [2025-10-20T18:24:22.882721] [INFO ] [MAIN ] DVN: Wiping existing primary data before full refresh. [2025-10-20T18:24:22.882766] [INFO ] [LOCAL ] DVN: Starting deletion of primary data. [2025-10-20T18:24:22.882808] [DEBUG] [LOCAL ] DVN: No primary data file found to delete. [2025-10-20T18:24:22.882879] [INFO ] [MAIN ] DVN: Starting full primary refresh. [2025-10-20T18:24:23.260536] [INFO ] [REMOTE ] DVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:23.422467] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T18:24:23.451330] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-10. [2025-10-20T18:24:24.251342] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-05. [2025-10-20T18:24:24.298281] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T18:24:24.306043] [INFO ] [REMOTE ] DUK: Full fetch complete (77567 records). [2025-10-20T18:24:24.306163] [DEBUG] [VALIDATOR ] DUK: Validating primary data. [2025-10-20T18:24:24.595538] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T18:24:24.727726] [DEBUG] [VALIDATOR ] DUK: All expected trading days are present. [2025-10-20T18:24:24.727803] [DEBUG] [VALIDATOR ] DUK: Primary data validation passed. [2025-10-20T18:24:24.728668] [INFO ] [MAIN ] DUK: Fetched primary data is valid. [2025-10-20T18:24:24.728729] [DEBUG] [LOCAL ] DUK: Saving primary data. [2025-10-20T18:24:24.730121] [DEBUG] [LOCAL ] DUK: Trimmed 77567 records to 53437 (keeping 200 days of data) [2025-10-20T18:24:25.051214] [DEBUG] [LOCAL ] DUK: Primary data saved (53437 records). [2025-10-20T18:24:25.051341] [INFO ] [MAIN ] DUK: Full history saved. [2025-10-20T18:24:25.051417] [INFO ] [MAIN ] Update Progress: 665/2367 symbols completed. DUK took 37.7s. Estimated remaining: 3h 35m 52s. Rate: 7.9 symbols/min [2025-10-20T18:24:25.051491] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-05. [2025-10-20T18:24:25.228635] [INFO ] [MAIN ] DX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:25.278393] [DEBUG] [LOCAL ] DX: Loaded 53271 existing records from primary storage. [2025-10-20T18:24:25.278762] [INFO ] [MAIN ] DX: Loaded 53271 existing records. [2025-10-20T18:24:25.458001] [WARN ] [MAIN ] DX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:25.458189] [INFO ] [MAIN ] DX: Wiping existing primary data before full refresh. [2025-10-20T18:24:25.458234] [INFO ] [LOCAL ] DX: Starting deletion of primary data. [2025-10-20T18:24:25.458517] [DEBUG] [LOCAL ] DX: Deleted primary data file. [2025-10-20T18:24:25.458553] [INFO ] [MAIN ] DX: Starting full primary refresh. [2025-10-20T18:24:25.821938] [INFO ] [REMOTE ] DX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:25.858196] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-01. [2025-10-20T18:24:25.863526] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T18:24:26.651407] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-06. [2025-10-20T18:24:26.677562] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T18:24:27.451390] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-01. [2025-10-20T18:24:27.744452] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:24:28.251380] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-02. [2025-10-20T18:24:28.296189] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:24:29.051399] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-07. [2025-10-20T18:24:29.677239] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-01 (2.23 seconds). [2025-10-20T18:24:29.851346] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-06. [2025-10-20T18:24:30.033319] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T18:24:30.651353] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-03. [2025-10-20T18:24:30.789176] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T18:24:31.281812] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T18:24:31.451406] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-07. [2025-10-20T18:24:32.251411] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-08. [2025-10-20T18:24:32.639569] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T18:24:33.012010] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T18:24:33.051384] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-08. [2025-10-20T18:24:33.851413] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-04. [2025-10-20T18:24:33.950170] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T18:24:34.450234] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T18:24:34.651370] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-09. [2025-10-20T18:24:35.451348] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-02. [2025-10-20T18:24:35.833478] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T18:24:35.886095] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T18:24:36.251376] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-05. [2025-10-20T18:24:37.051373] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-10. [2025-10-20T18:24:37.159030] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T18:24:37.851355] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-03. [2025-10-20T18:24:38.316479] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-10-20T18:24:38.371638] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T18:24:38.378136] [INFO ] [REMOTE ] DVA: Full fetch complete (59689 records). [2025-10-20T18:24:38.378271] [DEBUG] [VALIDATOR ] DVA: Validating primary data. [2025-10-20T18:24:38.662268] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-06. [2025-10-20T18:24:38.806085] [DEBUG] [VALIDATOR ] DVA: All expected trading days are present. [2025-10-20T18:24:38.806166] [DEBUG] [VALIDATOR ] DVA: Primary data validation passed. [2025-10-20T18:24:38.806916] [INFO ] [MAIN ] DVA: Fetched primary data is valid. [2025-10-20T18:24:38.806977] [DEBUG] [LOCAL ] DVA: Saving primary data. [2025-10-20T18:24:38.808126] [DEBUG] [LOCAL ] DVA: Trimmed 59689 records to 40887 (keeping 200 days of data) [2025-10-20T18:24:39.045926] [DEBUG] [LOCAL ] DVA: Primary data saved (40887 records). [2025-10-20T18:24:39.046050] [INFO ] [MAIN ] DVA: Full history saved. [2025-10-20T18:24:39.046125] [INFO ] [MAIN ] Update Progress: 666/2367 symbols completed. DVA took 27.5s. Estimated remaining: 3h 36m 1s. Rate: 7.9 symbols/min [2025-10-20T18:24:39.211698] [INFO ] [MAIN ] DXC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:39.261014] [DEBUG] [LOCAL ] DXC: Loaded 50621 existing records from primary storage. [2025-10-20T18:24:39.261383] [INFO ] [MAIN ] DXC: Loaded 50621 existing records. [2025-10-20T18:24:39.437707] [WARN ] [MAIN ] DXC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:39.437939] [INFO ] [MAIN ] DXC: Wiping existing primary data before full refresh. [2025-10-20T18:24:39.437989] [INFO ] [LOCAL ] DXC: Starting deletion of primary data. [2025-10-20T18:24:39.438175] [DEBUG] [LOCAL ] DXC: Deleted primary data file. [2025-10-20T18:24:39.438255] [INFO ] [MAIN ] DXC: Starting full primary refresh. [2025-10-20T18:24:39.472302] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-09. [2025-10-20T18:24:39.794728] [INFO ] [REMOTE ] DXC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:40.251359] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-01. [2025-10-20T18:24:40.495442] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T18:24:40.739805] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-03 (2.89 seconds). [2025-10-20T18:24:41.051377] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-07. [2025-10-20T18:24:41.473630] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T18:24:41.510393] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:24:41.851368] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-10. [2025-10-20T18:24:42.651385] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-02. [2025-10-20T18:24:43.124074] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T18:24:43.451370] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-08. [2025-10-20T18:24:43.704920] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:24:43.712713] [INFO ] [REMOTE ] DV: Full fetch complete (73860 records). [2025-10-20T18:24:43.712840] [DEBUG] [VALIDATOR ] DV: Validating primary data. [2025-10-20T18:24:44.109397] [DEBUG] [VALIDATOR ] DV: All expected trading days are present. [2025-10-20T18:24:44.109707] [DEBUG] [VALIDATOR ] DV: Primary data validation passed. [2025-10-20T18:24:44.110720] [INFO ] [MAIN ] DV: Fetched primary data is valid. [2025-10-20T18:24:44.110821] [DEBUG] [LOCAL ] DV: Saving primary data. [2025-10-20T18:24:44.121073] [DEBUG] [LOCAL ] DV: Trimmed 73860 records to 51250 (keeping 200 days of data) [2025-10-20T18:24:44.139401] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:24:44.254911] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-04. [2025-10-20T18:24:44.457725] [DEBUG] [LOCAL ] DV: Primary data saved (51250 records). [2025-10-20T18:24:44.457955] [INFO ] [MAIN ] DV: Full history saved. [2025-10-20T18:24:44.458036] [INFO ] [MAIN ] Update Progress: 667/2367 symbols completed. DV took 39.1s. Estimated remaining: 3h 35m 48s. Rate: 7.9 symbols/min [2025-10-20T18:24:44.627448] [INFO ] [MAIN ] DXYZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:44.662404] [DEBUG] [LOCAL ] DXYZ: Loaded 37760 existing records from primary storage. [2025-10-20T18:24:44.662717] [INFO ] [MAIN ] DXYZ: Loaded 37760 existing records. [2025-10-20T18:24:44.845197] [WARN ] [MAIN ] DXYZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:44.845409] [INFO ] [MAIN ] DXYZ: Wiping existing primary data before full refresh. [2025-10-20T18:24:44.845475] [INFO ] [LOCAL ] DXYZ: Starting deletion of primary data. [2025-10-20T18:24:44.845840] [DEBUG] [LOCAL ] DXYZ: Deleted primary data file. [2025-10-20T18:24:44.845914] [INFO ] [MAIN ] DXYZ: Starting full primary refresh. [2025-10-20T18:24:45.057383] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-03. [2025-10-20T18:24:45.209777] [INFO ] [REMOTE ] DXYZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:45.321709] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T18:24:45.851396] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-09. [2025-10-20T18:24:46.019303] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:24:46.651350] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-05. [2025-10-20T18:24:46.830948] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T18:24:47.451349] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-01. [2025-10-20T18:24:47.738770] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T18:24:48.251385] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-10. [2025-10-20T18:24:48.557964] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T18:24:49.051399] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-04. [2025-10-20T18:24:49.652812] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-01 (2.20 seconds). [2025-10-20T18:24:49.851422] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-06. [2025-10-20T18:24:50.398110] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T18:24:50.406339] [INFO ] [REMOTE ] DX: Full fetch complete (75864 records). [2025-10-20T18:24:50.406475] [DEBUG] [VALIDATOR ] DX: Validating primary data. [2025-10-20T18:24:50.670172] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-02. [2025-10-20T18:24:50.749740] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:24:50.847137] [DEBUG] [VALIDATOR ] DX: All expected trading days are present. [2025-10-20T18:24:50.847220] [DEBUG] [VALIDATOR ] DX: Primary data validation passed. [2025-10-20T18:24:50.848077] [INFO ] [MAIN ] DX: Fetched primary data is valid. [2025-10-20T18:24:50.848141] [DEBUG] [LOCAL ] DX: Saving primary data. [2025-10-20T18:24:50.849729] [DEBUG] [LOCAL ] DX: Trimmed 75864 records to 52526 (keeping 200 days of data) [2025-10-20T18:24:51.173395] [DEBUG] [LOCAL ] DX: Primary data saved (52526 records). [2025-10-20T18:24:51.173803] [INFO ] [MAIN ] DX: Full history saved. [2025-10-20T18:24:51.173900] [INFO ] [MAIN ] Update Progress: 668/2367 symbols completed. DX took 26.1s. Estimated remaining: 3h 35m 38s. Rate: 7.9 symbols/min [2025-10-20T18:24:51.342550] [INFO ] [MAIN ] DY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:51.369236] [DEBUG] [LOCAL ] DY: Loaded 28800 existing records from primary storage. [2025-10-20T18:24:51.369484] [INFO ] [MAIN ] DY: Loaded 28800 existing records. [2025-10-20T18:24:51.458481] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-05. [2025-10-20T18:24:51.551632] [WARN ] [MAIN ] DY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:51.551903] [INFO ] [MAIN ] DY: Wiping existing primary data before full refresh. [2025-10-20T18:24:51.551961] [INFO ] [LOCAL ] DY: Starting deletion of primary data. [2025-10-20T18:24:51.552488] [DEBUG] [LOCAL ] DY: Deleted primary data file. [2025-10-20T18:24:51.552546] [INFO ] [MAIN ] DY: Starting full primary refresh. [2025-10-20T18:24:51.920825] [INFO ] [REMOTE ] DY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:51.962950] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-06 (2.11 seconds). [2025-10-20T18:24:52.251368] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-07. [2025-10-20T18:24:52.621770] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-10-20T18:24:53.051431] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-03. [2025-10-20T18:24:53.264021] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T18:24:53.851370] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-01. [2025-10-20T18:24:54.101816] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:24:54.651354] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-08. [2025-10-20T18:24:54.848209] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T18:24:55.286605] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:24:55.451404] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-04. [2025-10-20T18:24:56.251490] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-06. [2025-10-20T18:24:56.362901] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T18:24:57.051402] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-09. [2025-10-20T18:24:57.472718] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T18:24:57.565185] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:24:57.851437] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-07. [2025-10-20T18:24:58.651401] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-02. [2025-10-20T18:24:59.096352] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T18:24:59.452296] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-10. [2025-10-20T18:24:59.529949] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T18:24:59.607818] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:25:00.251385] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-03. [2025-10-20T18:25:00.295926] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:25:00.303886] [INFO ] [REMOTE ] DVN: Full fetch complete (77879 records). [2025-10-20T18:25:00.304037] [DEBUG] [VALIDATOR ] DVN: Validating primary data. [2025-10-20T18:25:00.704394] [DEBUG] [VALIDATOR ] DVN: All expected trading days are present. [2025-10-20T18:25:00.704515] [DEBUG] [VALIDATOR ] DVN: Primary data validation passed. [2025-10-20T18:25:00.705637] [INFO ] [MAIN ] DVN: Fetched primary data is valid. [2025-10-20T18:25:00.705727] [DEBUG] [LOCAL ] DVN: Saving primary data. [2025-10-20T18:25:00.707496] [DEBUG] [LOCAL ] DVN: Trimmed 77879 records to 53699 (keeping 200 days of data) [2025-10-20T18:25:01.030591] [DEBUG] [LOCAL ] DVN: Primary data saved (53699 records). [2025-10-20T18:25:01.030761] [INFO ] [MAIN ] DVN: Full history saved. [2025-10-20T18:25:01.030838] [INFO ] [MAIN ] Update Progress: 669/2367 symbols completed. DVN took 38.3s. Estimated remaining: 3h 35m 36s. Rate: 7.9 symbols/min [2025-10-20T18:25:01.060201] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-08. [2025-10-20T18:25:01.214516] [INFO ] [MAIN ] E: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:01.252757] [DEBUG] [LOCAL ] E: Loaded 33728 existing records from primary storage. [2025-10-20T18:25:01.253107] [INFO ] [MAIN ] E: Loaded 33728 existing records. [2025-10-20T18:25:01.445502] [WARN ] [MAIN ] E: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:01.445781] [INFO ] [MAIN ] E: Wiping existing primary data before full refresh. [2025-10-20T18:25:01.445833] [INFO ] [LOCAL ] E: Starting deletion of primary data. [2025-10-20T18:25:01.446086] [DEBUG] [LOCAL ] E: Deleted primary data file. [2025-10-20T18:25:01.446145] [INFO ] [MAIN ] E: Starting full primary refresh. [2025-10-20T18:25:01.853791] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-05. [2025-10-20T18:25:01.859401] [INFO ] [REMOTE ] E: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:02.333534] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T18:25:02.651379] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-04. [2025-10-20T18:25:03.282420] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-08 (2.22 seconds). [2025-10-20T18:25:03.451428] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-09. [2025-10-20T18:25:04.029985] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-05 (2.18 seconds). [2025-10-20T18:25:04.112104] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:25:04.251370] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-05. [2025-10-20T18:25:04.788780] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T18:25:05.051385] [INFO ] [REMOTE ] E: Fetching primary data for 2025-01. [2025-10-20T18:25:05.760696] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T18:25:05.851415] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-06. [2025-10-20T18:25:06.475617] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T18:25:06.651431] [INFO ] [REMOTE ] E: Fetching primary data for 2025-02. [2025-10-20T18:25:07.451399] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-06. [2025-10-20T18:25:07.713321] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T18:25:07.884171] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:25:08.251450] [INFO ] [REMOTE ] E: Fetching primary data for 2025-03. [2025-10-20T18:25:08.752360] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T18:25:09.051397] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-07. [2025-10-20T18:25:09.218776] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T18:25:09.851387] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-10. [2025-10-20T18:25:10.186437] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T18:25:10.651373] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-07. [2025-10-20T18:25:11.451421] [INFO ] [REMOTE ] E: Fetching primary data for 2025-04. [2025-10-20T18:25:11.669823] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T18:25:11.677300] [INFO ] [REMOTE ] DXC: Full fetch complete (70224 records). [2025-10-20T18:25:11.677445] [DEBUG] [VALIDATOR ] DXC: Validating primary data. [2025-10-20T18:25:12.068522] [DEBUG] [VALIDATOR ] DXC: All expected trading days are present. [2025-10-20T18:25:12.068626] [DEBUG] [VALIDATOR ] DXC: Primary data validation passed. [2025-10-20T18:25:12.069503] [INFO ] [MAIN ] DXC: Fetched primary data is valid. [2025-10-20T18:25:12.069584] [DEBUG] [LOCAL ] DXC: Saving primary data. [2025-10-20T18:25:12.071172] [DEBUG] [LOCAL ] DXC: Trimmed 70224 records to 49946 (keeping 200 days of data) [2025-10-20T18:25:12.200344] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T18:25:12.263744] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-08. [2025-10-20T18:25:12.385894] [DEBUG] [LOCAL ] DXC: Primary data saved (49946 records). [2025-10-20T18:25:12.386022] [INFO ] [MAIN ] DXC: Full history saved. [2025-10-20T18:25:12.386100] [INFO ] [MAIN ] Update Progress: 670/2367 symbols completed. DXC took 33.3s. Estimated remaining: 3h 35m 38s. Rate: 7.9 symbols/min [2025-10-20T18:25:12.557870] [INFO ] [MAIN ] EAF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:12.558107] [DEBUG] [LOCAL ] EAF: No existing data file found in primary storage. [2025-10-20T18:25:12.558166] [WARN ] [MAIN ] EAF: No existing data found, performing full refresh instead. [2025-10-20T18:25:12.558240] [INFO ] [MAIN ] EAF: Wiping existing primary data before full refresh. [2025-10-20T18:25:12.558276] [INFO ] [LOCAL ] EAF: Starting deletion of primary data. [2025-10-20T18:25:12.558315] [DEBUG] [LOCAL ] EAF: No primary data file found to delete. [2025-10-20T18:25:12.558346] [INFO ] [MAIN ] EAF: Starting full primary refresh. [2025-10-20T18:25:12.942169] [INFO ] [REMOTE ] EAF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:12.966073] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:25:13.051390] [INFO ] [REMOTE ] E: Fetching primary data for 2025-05. [2025-10-20T18:25:13.851371] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-01. [2025-10-20T18:25:14.202858] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T18:25:14.442178] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T18:25:14.651373] [INFO ] [REMOTE ] E: Fetching primary data for 2025-06. [2025-10-20T18:25:15.426473] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T18:25:15.451325] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-02. [2025-10-20T18:25:16.016409] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:25:16.251394] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-08. [2025-10-20T18:25:17.034062] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:25:17.051326] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-03. [2025-10-20T18:25:17.351790] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:25:17.851365] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-09. [2025-10-20T18:25:18.651463] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-09. [2025-10-20T18:25:18.813058] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T18:25:19.451406] [INFO ] [REMOTE ] E: Fetching primary data for 2025-07. [2025-10-20T18:25:19.867700] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T18:25:20.024072] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T18:25:20.251386] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-10. [2025-10-20T18:25:20.878303] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:25:21.051389] [INFO ] [REMOTE ] E: Fetching primary data for 2025-08. [2025-10-20T18:25:21.320550] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T18:25:21.324055] [INFO ] [REMOTE ] DY: Full fetch complete (43351 records). [2025-10-20T18:25:21.324130] [DEBUG] [VALIDATOR ] DY: Validating primary data. [2025-10-20T18:25:21.694218] [DEBUG] [VALIDATOR ] DY: All expected trading days are present. [2025-10-20T18:25:21.694321] [DEBUG] [VALIDATOR ] DY: Primary data validation passed. [2025-10-20T18:25:21.694960] [INFO ] [MAIN ] DY: Fetched primary data is valid. [2025-10-20T18:25:21.695022] [DEBUG] [LOCAL ] DY: Saving primary data. [2025-10-20T18:25:21.695939] [DEBUG] [LOCAL ] DY: Trimmed 43351 records to 28264 (keeping 200 days of data) [2025-10-20T18:25:21.854609] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-10. [2025-10-20T18:25:21.861351] [DEBUG] [LOCAL ] DY: Primary data saved (28264 records). [2025-10-20T18:25:21.861426] [INFO ] [MAIN ] DY: Full history saved. [2025-10-20T18:25:21.861499] [INFO ] [MAIN ] Update Progress: 671/2367 symbols completed. DY took 30.7s. Estimated remaining: 3h 35m 35s. Rate: 7.9 symbols/min [2025-10-20T18:25:22.028732] [INFO ] [MAIN ] EAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:22.034515] [DEBUG] [LOCAL ] EAI: Loaded 4014 existing records from primary storage. [2025-10-20T18:25:22.034779] [INFO ] [MAIN ] EAI: Loaded 4014 existing records. [2025-10-20T18:25:22.210375] [WARN ] [MAIN ] EAI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:22.210662] [INFO ] [MAIN ] EAI: Wiping existing primary data before full refresh. [2025-10-20T18:25:22.210722] [INFO ] [LOCAL ] EAI: Starting deletion of primary data. [2025-10-20T18:25:22.210854] [DEBUG] [LOCAL ] EAI: Deleted primary data file. [2025-10-20T18:25:22.210892] [INFO ] [MAIN ] EAI: Starting full primary refresh. [2025-10-20T18:25:22.536848] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T18:25:22.625262] [INFO ] [REMOTE ] EAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:22.651333] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-01. [2025-10-20T18:25:23.152235] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T18:25:23.451368] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-02. [2025-10-20T18:25:23.792709] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T18:25:23.802101] [INFO ] [REMOTE ] DXYZ: Full fetch complete (57034 records). [2025-10-20T18:25:23.802252] [DEBUG] [VALIDATOR ] DXYZ: Validating primary data. [2025-10-20T18:25:24.047623] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T18:25:24.219140] [DEBUG] [VALIDATOR ] DXYZ: All expected trading days are present. [2025-10-20T18:25:24.219266] [DEBUG] [VALIDATOR ] DXYZ: Primary data validation passed. [2025-10-20T18:25:24.220163] [INFO ] [MAIN ] DXYZ: Fetched primary data is valid. [2025-10-20T18:25:24.220230] [DEBUG] [LOCAL ] DXYZ: Saving primary data. [2025-10-20T18:25:24.221566] [DEBUG] [LOCAL ] DXYZ: Trimmed 57034 records to 37229 (keeping 200 days of data) [2025-10-20T18:25:24.273693] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-03. [2025-10-20T18:25:24.454347] [DEBUG] [LOCAL ] DXYZ: Primary data saved (37229 records). [2025-10-20T18:25:24.454502] [INFO ] [MAIN ] DXYZ: Full history saved. [2025-10-20T18:25:24.454594] [INFO ] [MAIN ] Update Progress: 672/2367 symbols completed. DXYZ took 40.0s. Estimated remaining: 3h 35m 14s. Rate: 7.9 symbols/min [2025-10-20T18:25:24.647826] [INFO ] [MAIN ] EARN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:24.652719] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-10-20T18:25:24.680577] [DEBUG] [LOCAL ] EARN: Loaded 28956 existing records from primary storage. [2025-10-20T18:25:24.680883] [INFO ] [MAIN ] EARN: Loaded 28956 existing records. [2025-10-20T18:25:24.887400] [WARN ] [MAIN ] EARN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:24.887692] [INFO ] [MAIN ] EARN: Wiping existing primary data before full refresh. [2025-10-20T18:25:24.887752] [INFO ] [LOCAL ] EARN: Starting deletion of primary data. [2025-10-20T18:25:24.887936] [DEBUG] [LOCAL ] EARN: Deleted primary data file. [2025-10-20T18:25:24.887973] [INFO ] [MAIN ] EARN: Starting full primary refresh. [2025-10-20T18:25:25.058758] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-04. [2025-10-20T18:25:25.292532] [INFO ] [REMOTE ] EARN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:25.714401] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T18:25:25.851379] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-05. [2025-10-20T18:25:26.277589] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T18:25:26.651348] [INFO ] [REMOTE ] E: Fetching primary data for 2025-09. [2025-10-20T18:25:27.451376] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-06. [2025-10-20T18:25:27.942870] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T18:25:27.968716] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T18:25:28.251371] [INFO ] [REMOTE ] E: Fetching primary data for 2025-10. [2025-10-20T18:25:29.051381] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-07. [2025-10-20T18:25:29.622831] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:25:29.627947] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T18:25:29.633028] [INFO ] [REMOTE ] E: Full fetch complete (47718 records). [2025-10-20T18:25:29.633120] [DEBUG] [VALIDATOR ] E: Validating primary data. [2025-10-20T18:25:29.861017] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-04. [2025-10-20T18:25:30.028299] [DEBUG] [VALIDATOR ] E: All expected trading days are present. [2025-10-20T18:25:30.028385] [DEBUG] [VALIDATOR ] E: Primary data validation passed. [2025-10-20T18:25:30.029046] [INFO ] [MAIN ] E: Fetched primary data is valid. [2025-10-20T18:25:30.029107] [DEBUG] [LOCAL ] E: Saving primary data. [2025-10-20T18:25:30.030203] [DEBUG] [LOCAL ] E: Trimmed 47718 records to 33355 (keeping 200 days of data) [2025-10-20T18:25:30.231147] [DEBUG] [LOCAL ] E: Primary data saved (33355 records). [2025-10-20T18:25:30.231294] [INFO ] [MAIN ] E: Full history saved. [2025-10-20T18:25:30.231669] [INFO ] [MAIN ] Update Progress: 673/2367 symbols completed. E took 29.2s. Estimated remaining: 3h 35m 2s. Rate: 7.9 symbols/min [2025-10-20T18:25:30.407784] [INFO ] [MAIN ] EAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:30.460348] [DEBUG] [LOCAL ] EAT: Loaded 50582 existing records from primary storage. [2025-10-20T18:25:30.460689] [INFO ] [MAIN ] EAT: Loaded 50582 existing records. [2025-10-20T18:25:30.648881] [WARN ] [MAIN ] EAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:30.649093] [INFO ] [MAIN ] EAT: Wiping existing primary data before full refresh. [2025-10-20T18:25:30.649156] [INFO ] [LOCAL ] EAT: Starting deletion of primary data. [2025-10-20T18:25:30.649448] [DEBUG] [LOCAL ] EAT: Deleted primary data file. [2025-10-20T18:25:30.649519] [INFO ] [MAIN ] EAT: Starting full primary refresh. [2025-10-20T18:25:30.661489] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-08. [2025-10-20T18:25:31.038599] [INFO ] [REMOTE ] EAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:31.293328] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T18:25:31.438917] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T18:25:31.451329] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-05. [2025-10-20T18:25:32.251398] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-01. [2025-10-20T18:25:33.004812] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:25:33.051333] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-06. [2025-10-20T18:25:33.851382] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-01. [2025-10-20T18:25:33.939844] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:25:34.457578] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:25:34.651358] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-07. [2025-10-20T18:25:35.309769] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T18:25:35.451355] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-02. [2025-10-20T18:25:36.251360] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-02. [2025-10-20T18:25:36.455388] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:25:36.784332] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:25:37.051443] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-08. [2025-10-20T18:25:37.851351] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-09. [2025-10-20T18:25:38.095565] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T18:25:38.323924] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T18:25:38.593520] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:25:38.651377] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-09. [2025-10-20T18:25:39.451404] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-03. [2025-10-20T18:25:39.754789] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T18:25:40.251403] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-10. [2025-10-20T18:25:41.051383] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-03. [2025-10-20T18:25:41.157263] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T18:25:41.361380] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:25:41.366517] [INFO ] [REMOTE ] EAF: Full fetch complete (62351 records). [2025-10-20T18:25:41.366768] [DEBUG] [VALIDATOR ] EAF: Validating primary data. [2025-10-20T18:25:41.770726] [DEBUG] [VALIDATOR ] EAF: All expected trading days are present. [2025-10-20T18:25:41.770819] [DEBUG] [VALIDATOR ] EAF: Primary data validation passed. [2025-10-20T18:25:41.771583] [INFO ] [MAIN ] EAF: Fetched primary data is valid. [2025-10-20T18:25:41.771623] [DEBUG] [LOCAL ] EAF: Saving primary data. [2025-10-20T18:25:41.772852] [DEBUG] [LOCAL ] EAF: Trimmed 62351 records to 41341 (keeping 200 days of data) [2025-10-20T18:25:41.853504] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-04. [2025-10-20T18:25:42.002671] [DEBUG] [LOCAL ] EAF: Primary data saved (41341 records). [2025-10-20T18:25:42.002806] [INFO ] [MAIN ] EAF: Full history saved. [2025-10-20T18:25:42.002873] [INFO ] [MAIN ] Update Progress: 674/2367 symbols completed. EAF took 29.6s. Estimated remaining: 3h 35m 5s. Rate: 7.9 symbols/min [2025-10-20T18:25:42.171355] [INFO ] [MAIN ] EB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:42.205549] [DEBUG] [LOCAL ] EB: Loaded 36946 existing records from primary storage. [2025-10-20T18:25:42.205878] [INFO ] [MAIN ] EB: Loaded 36946 existing records. [2025-10-20T18:25:42.386425] [WARN ] [MAIN ] EB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:42.386733] [INFO ] [MAIN ] EB: Wiping existing primary data before full refresh. [2025-10-20T18:25:42.386786] [INFO ] [LOCAL ] EB: Starting deletion of primary data. [2025-10-20T18:25:42.386966] [DEBUG] [LOCAL ] EB: Deleted primary data file. [2025-10-20T18:25:42.387003] [INFO ] [MAIN ] EB: Starting full primary refresh. [2025-10-20T18:25:42.656979] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-10. [2025-10-20T18:25:42.749169] [INFO ] [REMOTE ] EB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:42.772473] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T18:25:43.281432] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T18:25:43.282529] [INFO ] [REMOTE ] EAI: Full fetch complete (5878 records). [2025-10-20T18:25:43.282617] [DEBUG] [VALIDATOR ] EAI: Validating primary data. [2025-10-20T18:25:43.454364] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-01. [2025-10-20T18:25:43.705363] [DEBUG] [VALIDATOR ] EAI: All expected trading days are present. [2025-10-20T18:25:43.705586] [DEBUG] [VALIDATOR ] EAI: Primary data validation passed. [2025-10-20T18:25:43.705862] [INFO ] [MAIN ] EAI: Fetched primary data is valid. [2025-10-20T18:25:43.705947] [DEBUG] [LOCAL ] EAI: Saving primary data. [2025-10-20T18:25:43.706679] [DEBUG] [LOCAL ] EAI: Trimmed 5878 records to 3940 (keeping 200 days of data) [2025-10-20T18:25:43.762064] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T18:25:43.777143] [DEBUG] [LOCAL ] EAI: Primary data saved (3940 records). [2025-10-20T18:25:43.777263] [INFO ] [MAIN ] EAI: Full history saved. [2025-10-20T18:25:43.777335] [INFO ] [MAIN ] Update Progress: 675/2367 symbols completed. EAI took 21.9s. Estimated remaining: 3h 34m 43s. Rate: 7.9 symbols/min [2025-10-20T18:25:43.948388] [INFO ] [MAIN ] EBF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:43.972534] [DEBUG] [LOCAL ] EBF: Loaded 19324 existing records from primary storage. [2025-10-20T18:25:43.972898] [INFO ] [MAIN ] EBF: Loaded 19324 existing records. [2025-10-20T18:25:44.180731] [WARN ] [MAIN ] EBF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:44.180972] [INFO ] [MAIN ] EBF: Wiping existing primary data before full refresh. [2025-10-20T18:25:44.181018] [INFO ] [LOCAL ] EBF: Starting deletion of primary data. [2025-10-20T18:25:44.181205] [DEBUG] [LOCAL ] EBF: Deleted primary data file. [2025-10-20T18:25:44.181260] [INFO ] [MAIN ] EBF: Starting full primary refresh. [2025-10-20T18:25:44.256487] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-05. [2025-10-20T18:25:44.538453] [INFO ] [REMOTE ] EBF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:44.716416] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:25:45.051419] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-02. [2025-10-20T18:25:45.851337] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-04. [2025-10-20T18:25:45.934050] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:25:45.990575] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:25:46.651393] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-01. [2025-10-20T18:25:47.451366] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-06. [2025-10-20T18:25:47.503404] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:25:47.717004] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T18:25:48.251398] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-05. [2025-10-20T18:25:49.051356] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-02. [2025-10-20T18:25:49.117142] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:25:49.776767] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T18:25:49.851404] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-03. [2025-10-20T18:25:50.200212] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T18:25:50.651406] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-03. [2025-10-20T18:25:51.215426] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T18:25:51.356342] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T18:25:51.451366] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-04. [2025-10-20T18:25:52.251347] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-07. [2025-10-20T18:25:52.631288] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T18:25:53.051374] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-04. [2025-10-20T18:25:53.851366] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-06. [2025-10-20T18:25:53.957091] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T18:25:54.513118] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:25:54.603031] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T18:25:54.651350] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-05. [2025-10-20T18:25:55.451404] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-07. [2025-10-20T18:25:56.073334] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T18:25:56.251364] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-08. [2025-10-20T18:25:56.284262] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T18:25:57.051384] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-06. [2025-10-20T18:25:57.851359] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-08. [2025-10-20T18:25:57.929164] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T18:25:58.396447] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T18:25:58.651395] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-09. [2025-10-20T18:25:58.775597] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T18:25:59.451407] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-07. [2025-10-20T18:26:00.251413] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-09. [2025-10-20T18:26:00.359857] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T18:26:01.051412] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-05. [2025-10-20T18:26:01.489094] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:26:01.856592] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-08. [2025-10-20T18:26:01.860315] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:26:02.651381] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-10. [2025-10-20T18:26:03.053705] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T18:26:03.280527] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T18:26:03.451430] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-09. [2025-10-20T18:26:03.934226] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:26:03.938906] [INFO ] [REMOTE ] EARN: Full fetch complete (45360 records). [2025-10-20T18:26:03.939050] [DEBUG] [VALIDATOR ] EARN: Validating primary data. [2025-10-20T18:26:04.256499] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-10. [2025-10-20T18:26:04.335690] [DEBUG] [VALIDATOR ] EARN: All expected trading days are present. [2025-10-20T18:26:04.335796] [DEBUG] [VALIDATOR ] EARN: Primary data validation passed. [2025-10-20T18:26:04.336530] [INFO ] [MAIN ] EARN: Fetched primary data is valid. [2025-10-20T18:26:04.336598] [DEBUG] [LOCAL ] EARN: Saving primary data. [2025-10-20T18:26:04.338010] [DEBUG] [LOCAL ] EARN: Trimmed 45360 records to 28002 (keeping 200 days of data) [2025-10-20T18:26:04.500337] [DEBUG] [LOCAL ] EARN: Primary data saved (28002 records). [2025-10-20T18:26:04.500484] [INFO ] [MAIN ] EARN: Full history saved. [2025-10-20T18:26:04.500565] [INFO ] [MAIN ] Update Progress: 676/2367 symbols completed. EARN took 40.0s. Estimated remaining: 3h 35m 8s. Rate: 7.9 symbols/min [2025-10-20T18:26:04.676459] [INFO ] [MAIN ] EBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:04.696352] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T18:26:04.738524] [DEBUG] [LOCAL ] EBR: Loaded 43964 existing records from primary storage. [2025-10-20T18:26:04.738867] [INFO ] [MAIN ] EBR: Loaded 43964 existing records. [2025-10-20T18:26:04.928868] [WARN ] [MAIN ] EBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:04.929086] [INFO ] [MAIN ] EBR: Wiping existing primary data before full refresh. [2025-10-20T18:26:04.929135] [INFO ] [LOCAL ] EBR: Starting deletion of primary data. [2025-10-20T18:26:04.929605] [DEBUG] [LOCAL ] EBR: Deleted primary data file. [2025-10-20T18:26:04.929706] [INFO ] [MAIN ] EBR: Starting full primary refresh. [2025-10-20T18:26:05.056559] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-10. [2025-10-20T18:26:05.320833] [INFO ] [REMOTE ] EBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:05.851420] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-01. [2025-10-20T18:26:06.146417] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T18:26:06.154226] [INFO ] [REMOTE ] EAT: Full fetch complete (72389 records). [2025-10-20T18:26:06.154399] [DEBUG] [VALIDATOR ] EAT: Validating primary data. [2025-10-20T18:26:06.543478] [DEBUG] [VALIDATOR ] EAT: All expected trading days are present. [2025-10-20T18:26:06.543630] [DEBUG] [VALIDATOR ] EAT: Primary data validation passed. [2025-10-20T18:26:06.544555] [INFO ] [MAIN ] EAT: Fetched primary data is valid. [2025-10-20T18:26:06.544605] [DEBUG] [LOCAL ] EAT: Saving primary data. [2025-10-20T18:26:06.546194] [DEBUG] [LOCAL ] EAT: Trimmed 72389 records to 49806 (keeping 200 days of data) [2025-10-20T18:26:06.656805] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-06. [2025-10-20T18:26:06.781851] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:26:06.791844] [INFO ] [REMOTE ] EB: Full fetch complete (55115 records). [2025-10-20T18:26:06.793888] [DEBUG] [VALIDATOR ] EB: Validating primary data. [2025-10-20T18:26:07.239912] [DEBUG] [LOCAL ] EAT: Primary data saved (49806 records). [2025-10-20T18:26:07.254950] [INFO ] [MAIN ] EAT: Full history saved. [2025-10-20T18:26:07.255751] [INFO ] [MAIN ] Update Progress: 677/2367 symbols completed. EAT took 37.0s. Estimated remaining: 3h 34m 48s. Rate: 7.9 symbols/min [2025-10-20T18:26:07.377552] [DEBUG] [VALIDATOR ] EB: All expected trading days are present. [2025-10-20T18:26:07.377717] [DEBUG] [VALIDATOR ] EB: Primary data validation passed. [2025-10-20T18:26:07.378728] [INFO ] [MAIN ] EB: Fetched primary data is valid. [2025-10-20T18:26:07.378889] [DEBUG] [LOCAL ] EB: Saving primary data. [2025-10-20T18:26:07.380409] [DEBUG] [LOCAL ] EB: Trimmed 55115 records to 36273 (keeping 200 days of data) [2025-10-20T18:26:07.507329] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T18:26:07.518833] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-02. [2025-10-20T18:26:07.644410] [DEBUG] [LOCAL ] EB: Primary data saved (36273 records). [2025-10-20T18:26:07.645542] [INFO ] [MAIN ] EB: Full history saved. [2025-10-20T18:26:07.645721] [INFO ] [MAIN ] Update Progress: 678/2367 symbols completed. EB took 25.6s. Estimated remaining: 3h 34m 22s. Rate: 7.9 symbols/min [2025-10-20T18:26:07.646787] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T18:26:08.251505] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-07. [2025-10-20T18:26:08.953018] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T18:26:09.051430] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-03. [2025-10-20T18:26:09.234577] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T18:26:09.851380] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-08. [2025-10-20T18:26:10.445934] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T18:26:10.651482] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-04. [2025-10-20T18:26:10.727286] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T18:26:11.451429] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-09. [2025-10-20T18:26:12.363715] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T18:26:12.375732] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-05. [2025-10-20T18:26:12.466534] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T18:26:13.051445] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-10. [2025-10-20T18:26:13.955999] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:26:13.967007] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-06. [2025-10-20T18:26:14.256452] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T18:26:14.260586] [INFO ] [REMOTE ] EBF: Full fetch complete (28078 records). [2025-10-20T18:26:14.260725] [DEBUG] [VALIDATOR ] EBF: Validating primary data. [2025-10-20T18:26:14.634055] [DEBUG] [VALIDATOR ] EBF: All expected trading days are present. [2025-10-20T18:26:14.634140] [DEBUG] [VALIDATOR ] EBF: Primary data validation passed. [2025-10-20T18:26:14.634708] [INFO ] [MAIN ] EBF: Fetched primary data is valid. [2025-10-20T18:26:14.634771] [DEBUG] [LOCAL ] EBF: Saving primary data. [2025-10-20T18:26:14.635510] [DEBUG] [LOCAL ] EBF: Trimmed 28078 records to 19140 (keeping 200 days of data) [2025-10-20T18:26:14.750001] [DEBUG] [LOCAL ] EBF: Primary data saved (19140 records). [2025-10-20T18:26:14.750148] [INFO ] [MAIN ] EBF: Full history saved. [2025-10-20T18:26:14.750227] [INFO ] [MAIN ] Update Progress: 679/2367 symbols completed. EBF took 31.0s. Estimated remaining: 3h 34m 14s. Rate: 7.9 symbols/min [2025-10-20T18:26:15.473613] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T18:26:15.482200] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-07. [2025-10-20T18:26:16.885463] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T18:26:16.894973] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-08. [2025-10-20T18:26:18.156906] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T18:26:18.171356] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-09. [2025-10-20T18:26:19.728017] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T18:26:19.737813] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-10. [2025-10-20T18:26:21.532820] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T18:26:21.539822] [INFO ] [REMOTE ] EBR: Full fetch complete (62557 records). [2025-10-20T18:26:21.539971] [DEBUG] [VALIDATOR ] EBR: Validating primary data. [2025-10-20T18:26:21.945175] [DEBUG] [VALIDATOR ] EBR: All expected trading days are present. [2025-10-20T18:26:21.945271] [DEBUG] [VALIDATOR ] EBR: Primary data validation passed. [2025-10-20T18:26:21.946237] [INFO ] [MAIN ] EBR: Fetched primary data is valid. [2025-10-20T18:26:21.946304] [DEBUG] [LOCAL ] EBR: Saving primary data. [2025-10-20T18:26:21.947753] [DEBUG] [LOCAL ] EBR: Trimmed 62557 records to 43386 (keeping 200 days of data) [2025-10-20T18:26:22.201806] [DEBUG] [LOCAL ] EBR: Primary data saved (43386 records). [2025-10-20T18:26:22.201939] [INFO ] [MAIN ] EBR: Full history saved. [2025-10-20T18:26:22.202018] [INFO ] [MAIN ] Update Progress: 680/2367 symbols completed. EBR took 17.7s. Estimated remaining: 3h 34m 6s. Rate: 7.9 symbols/min [2025-10-20T18:26:22.835440] [INFO ] [MAIN ] ECC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:22.863472] [INFO ] [MAIN ] EBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:22.933237] [INFO ] [MAIN ] ECAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:22.934807] [INFO ] [MAIN ] EC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:22.997944] [DEBUG] [LOCAL ] ECAT: Loaded 29841 existing records from primary storage. [2025-10-20T18:26:22.998430] [INFO ] [MAIN ] ECAT: Loaded 29841 existing records. [2025-10-20T18:26:23.048827] [DEBUG] [LOCAL ] ECC: Loaded 43973 existing records from primary storage. [2025-10-20T18:26:23.093148] [INFO ] [MAIN ] ECC: Loaded 43973 existing records. [2025-10-20T18:26:23.151833] [DEBUG] [LOCAL ] EC: Loaded 49674 existing records from primary storage. [2025-10-20T18:26:23.145872] [DEBUG] [LOCAL ] EBS: Loaded 47428 existing records from primary storage. [2025-10-20T18:26:23.152591] [INFO ] [MAIN ] EBS: Loaded 47428 existing records. [2025-10-20T18:26:23.152701] [INFO ] [MAIN ] EC: Loaded 49674 existing records. [2025-10-20T18:26:23.460948] [WARN ] [MAIN ] ECAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:23.557859] [INFO ] [MAIN ] ECAT: Wiping existing primary data before full refresh. [2025-10-20T18:26:23.558157] [INFO ] [LOCAL ] ECAT: Starting deletion of primary data. [2025-10-20T18:26:23.585439] [DEBUG] [LOCAL ] ECAT: Deleted primary data file. [2025-10-20T18:26:23.692822] [INFO ] [MAIN ] ECAT: Starting full primary refresh. [2025-10-20T18:26:23.775951] [WARN ] [MAIN ] EC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:23.816333] [WARN ] [MAIN ] ECC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:23.830455] [WARN ] [MAIN ] EBS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:23.856768] [INFO ] [MAIN ] ECC: Wiping existing primary data before full refresh. [2025-10-20T18:26:23.866820] [INFO ] [LOCAL ] ECC: Starting deletion of primary data. [2025-10-20T18:26:23.856442] [INFO ] [MAIN ] EC: Wiping existing primary data before full refresh. [2025-10-20T18:26:23.856597] [INFO ] [MAIN ] EBS: Wiping existing primary data before full refresh. [2025-10-20T18:26:23.889532] [INFO ] [LOCAL ] EC: Starting deletion of primary data. [2025-10-20T18:26:23.889463] [DEBUG] [LOCAL ] ECC: Deleted primary data file. [2025-10-20T18:26:23.910262] [INFO ] [LOCAL ] EBS: Starting deletion of primary data. [2025-10-20T18:26:23.915592] [INFO ] [MAIN ] ECC: Starting full primary refresh. [2025-10-20T18:26:23.919680] [DEBUG] [LOCAL ] EC: Deleted primary data file. [2025-10-20T18:26:23.930795] [INFO ] [MAIN ] EC: Starting full primary refresh. [2025-10-20T18:26:23.930716] [DEBUG] [LOCAL ] EBS: Deleted primary data file. [2025-10-20T18:26:23.930964] [INFO ] [MAIN ] EBS: Starting full primary refresh. [2025-10-20T18:26:24.972988] [INFO ] [REMOTE ] ECAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:25.022844] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-01. [2025-10-20T18:26:25.181251] [INFO ] [REMOTE ] EBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:25.272344] [INFO ] [REMOTE ] EC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:25.298374] [INFO ] [REMOTE ] ECC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:25.298987] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-01. [2025-10-20T18:26:25.298636] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-01. [2025-10-20T18:26:25.300704] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-01. [2025-10-20T18:26:26.654167] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:26:26.660289] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-02. [2025-10-20T18:26:26.884892] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T18:26:26.892911] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-02. [2025-10-20T18:26:26.994029] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:26:27.003978] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-02. [2025-10-20T18:26:27.086908] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:26:27.096391] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-02. [2025-10-20T18:26:27.642525] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:26:27.648582] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-03. [2025-10-20T18:26:28.059087] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T18:26:28.067235] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-03. [2025-10-20T18:26:28.571612] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:26:28.582375] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-03. [2025-10-20T18:26:28.608087] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T18:26:28.615762] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-04. [2025-10-20T18:26:28.655846] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:26:28.667269] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-03. [2025-10-20T18:26:29.775633] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T18:26:29.793218] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-04. [2025-10-20T18:26:29.912913] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T18:26:29.919785] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-05. [2025-10-20T18:26:30.089097] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T18:26:30.099165] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-04. [2025-10-20T18:26:30.450999] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T18:26:30.651457] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-04. [2025-10-20T18:26:30.833466] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T18:26:31.451448] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-06. [2025-10-20T18:26:31.511812] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T18:26:31.602900] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T18:26:32.079638] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:26:32.251394] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-05. [2025-10-20T18:26:32.709305] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T18:26:33.051396] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-07. [2025-10-20T18:26:33.834763] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:26:33.851347] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-05. [2025-10-20T18:26:34.395929] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T18:26:34.651407] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-06. [2025-10-20T18:26:35.451366] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-08. [2025-10-20T18:26:35.541295] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T18:26:36.117829] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T18:26:36.251372] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-07. [2025-10-20T18:26:36.524800] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T18:26:37.051381] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-05. [2025-10-20T18:26:37.851349] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-06. [2025-10-20T18:26:38.119430] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:26:38.651376] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-09. [2025-10-20T18:26:38.915715] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T18:26:39.463500] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-06. [2025-10-20T18:26:39.465874] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:26:39.916692] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T18:26:40.251367] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-08. [2025-10-20T18:26:41.051429] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-07. [2025-10-20T18:26:41.111554] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:26:41.851354] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-07. [2025-10-20T18:26:41.916005] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T18:26:42.651411] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-09. [2025-10-20T18:26:42.842348] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T18:26:43.154126] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T18:26:43.451388] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-08. [2025-10-20T18:26:44.251423] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-08. [2025-10-20T18:26:44.416572] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T18:26:45.051432] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-10. [2025-10-20T18:26:45.143123] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T18:26:45.851393] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-09. [2025-10-20T18:26:45.903488] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T18:26:46.651442] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-10. [2025-10-20T18:26:46.807816] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T18:26:46.814908] [INFO ] [REMOTE ] EC: Full fetch complete (71887 records). [2025-10-20T18:26:46.815066] [DEBUG] [VALIDATOR ] EC: Validating primary data. [2025-10-20T18:26:47.210591] [DEBUG] [VALIDATOR ] EC: All expected trading days are present. [2025-10-20T18:26:47.210731] [DEBUG] [VALIDATOR ] EC: Primary data validation passed. [2025-10-20T18:26:47.211547] [INFO ] [MAIN ] EC: Fetched primary data is valid. [2025-10-20T18:26:47.211591] [DEBUG] [LOCAL ] EC: Saving primary data. [2025-10-20T18:26:47.212904] [DEBUG] [LOCAL ] EC: Trimmed 71887 records to 48929 (keeping 200 days of data) [2025-10-20T18:26:47.465706] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-09. [2025-10-20T18:26:47.497227] [DEBUG] [LOCAL ] EC: Primary data saved (48929 records). [2025-10-20T18:26:47.497766] [INFO ] [MAIN ] EC: Full history saved. [2025-10-20T18:26:47.497892] [INFO ] [MAIN ] Update Progress: 681/2367 symbols completed. EC took 25.3s. Estimated remaining: 3h 34m 42s. Rate: 7.9 symbols/min [2025-10-20T18:26:47.636488] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T18:26:47.713974] [INFO ] [MAIN ] ECCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:47.714188] [DEBUG] [LOCAL ] ECCC: No existing data file found in primary storage. [2025-10-20T18:26:47.714233] [WARN ] [MAIN ] ECCC: No existing data found, performing full refresh instead. [2025-10-20T18:26:47.714348] [INFO ] [MAIN ] ECCC: Wiping existing primary data before full refresh. [2025-10-20T18:26:47.714391] [INFO ] [LOCAL ] ECCC: Starting deletion of primary data. [2025-10-20T18:26:47.714433] [DEBUG] [LOCAL ] ECCC: No primary data file found to delete. [2025-10-20T18:26:47.714466] [INFO ] [MAIN ] ECCC: Starting full primary refresh. [2025-10-20T18:26:48.009922] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T18:26:48.049563] [INFO ] [REMOTE ] ECAT: Full fetch complete (42976 records). [2025-10-20T18:26:48.051985] [DEBUG] [VALIDATOR ] ECAT: Validating primary data. [2025-10-20T18:26:48.239231] [INFO ] [REMOTE ] ECCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:48.254746] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-10. [2025-10-20T18:26:48.507049] [DEBUG] [VALIDATOR ] ECAT: All expected trading days are present. [2025-10-20T18:26:48.507139] [DEBUG] [VALIDATOR ] ECAT: Primary data validation passed. [2025-10-20T18:26:48.507894] [INFO ] [MAIN ] ECAT: Fetched primary data is valid. [2025-10-20T18:26:48.507962] [DEBUG] [LOCAL ] ECAT: Saving primary data. [2025-10-20T18:26:48.509078] [DEBUG] [LOCAL ] ECAT: Trimmed 42976 records to 29346 (keeping 200 days of data) [2025-10-20T18:26:48.686471] [DEBUG] [LOCAL ] ECAT: Primary data saved (29346 records). [2025-10-20T18:26:48.686606] [INFO ] [MAIN ] ECAT: Full history saved. [2025-10-20T18:26:48.686712] [INFO ] [MAIN ] Update Progress: 682/2367 symbols completed. ECAT took 26.5s. Estimated remaining: 3h 34m 18s. Rate: 7.9 symbols/min [2025-10-20T18:26:48.864681] [INFO ] [MAIN ] ECCU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:48.896110] [DEBUG] [LOCAL ] ECCU: Loaded 2234 existing records from primary storage. [2025-10-20T18:26:48.893941] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T18:26:48.905527] [INFO ] [MAIN ] ECCU: Loaded 2234 existing records. [2025-10-20T18:26:49.056553] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-01. [2025-10-20T18:26:49.087556] [WARN ] [MAIN ] ECCU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:49.087841] [INFO ] [MAIN ] ECCU: Wiping existing primary data before full refresh. [2025-10-20T18:26:49.087911] [INFO ] [LOCAL ] ECCU: Starting deletion of primary data. [2025-10-20T18:26:49.088117] [DEBUG] [LOCAL ] ECCU: Deleted primary data file. [2025-10-20T18:26:49.088182] [INFO ] [MAIN ] ECCU: Starting full primary refresh. [2025-10-20T18:26:49.443080] [INFO ] [REMOTE ] ECCU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:49.565145] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T18:26:49.851387] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-02. [2025-10-20T18:26:50.148472] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T18:26:50.156076] [INFO ] [REMOTE ] ECC: Full fetch complete (61538 records). [2025-10-20T18:26:50.156215] [DEBUG] [VALIDATOR ] ECC: Validating primary data. [2025-10-20T18:26:50.246021] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T18:26:50.625154] [DEBUG] [VALIDATOR ] ECC: All expected trading days are present. [2025-10-20T18:26:50.625247] [DEBUG] [VALIDATOR ] ECC: Primary data validation passed. [2025-10-20T18:26:50.626399] [INFO ] [MAIN ] ECC: Fetched primary data is valid. [2025-10-20T18:26:50.626502] [DEBUG] [LOCAL ] ECC: Saving primary data. [2025-10-20T18:26:50.628278] [DEBUG] [LOCAL ] ECC: Trimmed 61538 records to 43475 (keeping 200 days of data) [2025-10-20T18:26:50.690053] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-03. [2025-10-20T18:26:50.912046] [DEBUG] [LOCAL ] ECC: Primary data saved (43475 records). [2025-10-20T18:26:50.912158] [INFO ] [MAIN ] ECC: Full history saved. [2025-10-20T18:26:50.912274] [INFO ] [MAIN ] Update Progress: 683/2367 symbols completed. ECC took 28.7s. Estimated remaining: 3h 33m 57s. Rate: 7.9 symbols/min [2025-10-20T18:26:51.086634] [INFO ] [MAIN ] ECCW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:51.088890] [DEBUG] [LOCAL ] ECCW: No existing data file found in primary storage. [2025-10-20T18:26:51.088993] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T18:26:51.090142] [WARN ] [MAIN ] ECCW: No existing data found, performing full refresh instead. [2025-10-20T18:26:51.090254] [INFO ] [MAIN ] ECCW: Wiping existing primary data before full refresh. [2025-10-20T18:26:51.090296] [INFO ] [LOCAL ] ECCW: Starting deletion of primary data. [2025-10-20T18:26:51.090337] [DEBUG] [LOCAL ] ECCW: No primary data file found to delete. [2025-10-20T18:26:51.090367] [INFO ] [MAIN ] ECCW: Starting full primary refresh. [2025-10-20T18:26:51.437926] [INFO ] [REMOTE ] ECCW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:51.451332] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-01. [2025-10-20T18:26:51.897069] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T18:26:52.251379] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-04. [2025-10-20T18:26:52.591950] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-10-20T18:26:53.051412] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-01. [2025-10-20T18:26:53.630895] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T18:26:53.851365] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-05. [2025-10-20T18:26:54.202183] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T18:26:54.651428] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-02. [2025-10-20T18:26:55.252862] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T18:26:55.451386] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-03. [2025-10-20T18:26:56.060208] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T18:26:56.251371] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-04. [2025-10-20T18:26:56.765373] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-10-20T18:26:57.051412] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-02. [2025-10-20T18:26:57.293319] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-10-20T18:26:57.851422] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-10. [2025-10-20T18:26:58.651373] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-03. [2025-10-20T18:26:58.988793] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-10-20T18:26:59.451433] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-04. [2025-10-20T18:26:59.615390] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T18:26:59.622252] [INFO ] [REMOTE ] EBS: Full fetch complete (67400 records). [2025-10-20T18:26:59.622403] [DEBUG] [VALIDATOR ] EBS: Validating primary data. [2025-10-20T18:26:59.728235] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T18:27:00.018195] [DEBUG] [VALIDATOR ] EBS: All expected trading days are present. [2025-10-20T18:27:00.018284] [DEBUG] [VALIDATOR ] EBS: Primary data validation passed. [2025-10-20T18:27:00.019086] [INFO ] [MAIN ] EBS: Fetched primary data is valid. [2025-10-20T18:27:00.019151] [DEBUG] [LOCAL ] EBS: Saving primary data. [2025-10-20T18:27:00.020513] [DEBUG] [LOCAL ] EBS: Trimmed 67400 records to 46671 (keeping 200 days of data) [2025-10-20T18:27:00.257189] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-06. [2025-10-20T18:27:00.282090] [DEBUG] [LOCAL ] EBS: Primary data saved (46671 records). [2025-10-20T18:27:00.282248] [INFO ] [MAIN ] EBS: Full history saved. [2025-10-20T18:27:00.282850] [INFO ] [MAIN ] Update Progress: 684/2367 symbols completed. EBS took 38.1s. Estimated remaining: 3h 33m 54s. Rate: 7.9 symbols/min [2025-10-20T18:27:00.455954] [INFO ] [MAIN ] ECCX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:00.456152] [DEBUG] [LOCAL ] ECCX: No existing data file found in primary storage. [2025-10-20T18:27:00.456189] [WARN ] [MAIN ] ECCX: No existing data found, performing full refresh instead. [2025-10-20T18:27:00.456298] [INFO ] [MAIN ] ECCX: Wiping existing primary data before full refresh. [2025-10-20T18:27:00.456339] [INFO ] [LOCAL ] ECCX: Starting deletion of primary data. [2025-10-20T18:27:00.456379] [DEBUG] [LOCAL ] ECCX: No primary data file found to delete. [2025-10-20T18:27:00.456409] [INFO ] [MAIN ] ECCX: Starting full primary refresh. [2025-10-20T18:27:00.807338] [INFO ] [REMOTE ] ECCX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:01.051376] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-01. [2025-10-20T18:27:01.093947] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T18:27:01.559932] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T18:27:01.851381] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-05. [2025-10-20T18:27:02.440683] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:27:02.651379] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-07. [2025-10-20T18:27:03.018999] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-10-20T18:27:03.451362] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-02. [2025-10-20T18:27:03.785253] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T18:27:04.251368] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-05. [2025-10-20T18:27:04.808328] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T18:27:05.051432] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-06. [2025-10-20T18:27:05.454308] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-10-20T18:27:05.851456] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-07. [2025-10-20T18:27:06.192344] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T18:27:06.651387] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-06. [2025-10-20T18:27:07.176125] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-10-20T18:27:07.451408] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-08. [2025-10-20T18:27:07.801868] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-10-20T18:27:08.251359] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-09. [2025-10-20T18:27:08.571047] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-10-20T18:27:09.051381] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-03. [2025-10-20T18:27:09.510173] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T18:27:09.851376] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-04. [2025-10-20T18:27:10.126549] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T18:27:10.651387] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-05. [2025-10-20T18:27:11.003731] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T18:27:11.451434] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-07. [2025-10-20T18:27:11.961561] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T18:27:12.251396] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-08. [2025-10-20T18:27:12.680423] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T18:27:13.051415] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-08. [2025-10-20T18:27:13.481357] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T18:27:13.851426] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-09. [2025-10-20T18:27:14.194562] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-09 (0.34 seconds). [2025-10-20T18:27:14.651378] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-06. [2025-10-20T18:27:15.124310] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T18:27:15.451378] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-09. [2025-10-20T18:27:15.862679] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T18:27:16.251379] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-10. [2025-10-20T18:27:17.007716] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T18:27:17.008703] [INFO ] [REMOTE ] ECCU: Full fetch complete (4310 records). [2025-10-20T18:27:17.008773] [DEBUG] [VALIDATOR ] ECCU: Validating primary data. [2025-10-20T18:27:17.056595] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-07. [2025-10-20T18:27:17.384733] [DEBUG] [VALIDATOR ] ECCU: All expected trading days are present. [2025-10-20T18:27:17.384819] [DEBUG] [VALIDATOR ] ECCU: Primary data validation passed. [2025-10-20T18:27:17.384963] [INFO ] [MAIN ] ECCU: Fetched primary data is valid. [2025-10-20T18:27:17.385002] [DEBUG] [LOCAL ] ECCU: Saving primary data. [2025-10-20T18:27:17.385420] [DEBUG] [LOCAL ] ECCU: Trimmed 4310 records to 2179 (keeping 200 days of data) [2025-10-20T18:27:17.400109] [DEBUG] [LOCAL ] ECCU: Primary data saved (2179 records). [2025-10-20T18:27:17.400243] [INFO ] [MAIN ] ECCU: Full history saved. [2025-10-20T18:27:17.400326] [INFO ] [MAIN ] Update Progress: 685/2367 symbols completed. ECCU took 28.7s. Estimated remaining: 3h 34m 9s. Rate: 7.9 symbols/min [2025-10-20T18:27:17.582520] [INFO ] [MAIN ] ECG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:17.585690] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T18:27:17.622741] [DEBUG] [LOCAL ] ECG: Loaded 33657 existing records from primary storage. [2025-10-20T18:27:17.623032] [INFO ] [MAIN ] ECG: Loaded 33657 existing records. [2025-10-20T18:27:17.804806] [WARN ] [MAIN ] ECG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:17.805036] [INFO ] [MAIN ] ECG: Wiping existing primary data before full refresh. [2025-10-20T18:27:17.805084] [INFO ] [LOCAL ] ECG: Starting deletion of primary data. [2025-10-20T18:27:17.805286] [DEBUG] [LOCAL ] ECG: Deleted primary data file. [2025-10-20T18:27:17.805325] [INFO ] [MAIN ] ECG: Starting full primary refresh. [2025-10-20T18:27:17.858599] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-10. [2025-10-20T18:27:18.167605] [INFO ] [REMOTE ] ECG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:18.319838] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T18:27:18.320693] [INFO ] [REMOTE ] ECCC: Full fetch complete (1299 records). [2025-10-20T18:27:18.320765] [DEBUG] [VALIDATOR ] ECCC: Validating primary data. [2025-10-20T18:27:18.321145] [FATAL] [VALIDATOR ] ECCC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:27:18.321228] [FATAL] [MAIN ] ECCC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:27:18.321271] [WARN ] [MAIN ] ECCC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:27:18.321338] [INFO ] [LOCAL ] ECCC: Starting deletion of primary data. [2025-10-20T18:27:18.321401] [DEBUG] [LOCAL ] ECCC: No primary data file found to delete. [2025-10-20T18:27:18.321438] [INFO ] [LOCAL ] ECCC: Starting deletion of derivative data. [2025-10-20T18:27:18.321478] [DEBUG] [LOCAL ] ECCC: No derivative data file found to delete. [2025-10-20T18:27:18.321531] [INFO ] [MAIN ] Update Progress: 686/2367 symbols completed. ECCC took 30.8s. Estimated remaining: 3h 33m 45s. Rate: 7.9 symbols/min [2025-10-20T18:27:18.529819] [INFO ] [MAIN ] ECL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:18.575791] [DEBUG] [LOCAL ] ECL: Loaded 47419 existing records from primary storage. [2025-10-20T18:27:18.576156] [INFO ] [MAIN ] ECL: Loaded 47419 existing records. [2025-10-20T18:27:18.656630] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-01. [2025-10-20T18:27:18.777553] [WARN ] [MAIN ] ECL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:18.777792] [INFO ] [MAIN ] ECL: Wiping existing primary data before full refresh. [2025-10-20T18:27:18.777843] [INFO ] [LOCAL ] ECL: Starting deletion of primary data. [2025-10-20T18:27:18.778466] [DEBUG] [LOCAL ] ECL: Deleted primary data file. [2025-10-20T18:27:18.778523] [INFO ] [MAIN ] ECL: Starting full primary refresh. [2025-10-20T18:27:19.122061] [INFO ] [REMOTE ] ECL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:19.451380] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-10. [2025-10-20T18:27:20.149338] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T18:27:20.151464] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T18:27:20.150675] [INFO ] [REMOTE ] ECCW: Full fetch complete (1424 records). [2025-10-20T18:27:20.162958] [DEBUG] [VALIDATOR ] ECCW: Validating primary data. [2025-10-20T18:27:20.256549] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-08. [2025-10-20T18:27:20.376316] [WARN ] [VALIDATOR ] ECCW: Validation failed - insufficient historical data. Have 191 unique days, need 200 days [2025-10-20T18:27:20.376530] [FATAL] [MAIN ] ECCW: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:27:20.376599] [WARN ] [MAIN ] ECCW: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:27:20.376698] [INFO ] [LOCAL ] ECCW: Starting deletion of primary data. [2025-10-20T18:27:20.376766] [DEBUG] [LOCAL ] ECCW: No primary data file found to delete. [2025-10-20T18:27:20.376803] [INFO ] [LOCAL ] ECCW: Starting deletion of derivative data. [2025-10-20T18:27:20.376845] [DEBUG] [LOCAL ] ECCW: No derivative data file found to delete. [2025-10-20T18:27:20.376900] [INFO ] [MAIN ] Update Progress: 687/2367 symbols completed. ECCW took 29.5s. Estimated remaining: 3h 33m 24s. Rate: 7.9 symbols/min [2025-10-20T18:27:20.547714] [INFO ] [MAIN ] ECO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:20.565783] [DEBUG] [LOCAL ] ECO: Loaded 18402 existing records from primary storage. [2025-10-20T18:27:20.566037] [INFO ] [MAIN ] ECO: Loaded 18402 existing records. [2025-10-20T18:27:20.668406] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-10-20T18:27:20.746801] [WARN ] [MAIN ] ECO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:20.747031] [INFO ] [MAIN ] ECO: Wiping existing primary data before full refresh. [2025-10-20T18:27:20.747079] [INFO ] [LOCAL ] ECO: Starting deletion of primary data. [2025-10-20T18:27:20.747275] [DEBUG] [LOCAL ] ECO: Deleted primary data file. [2025-10-20T18:27:20.747338] [INFO ] [MAIN ] ECO: Starting full primary refresh. [2025-10-20T18:27:21.062119] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-09. [2025-10-20T18:27:21.143185] [INFO ] [REMOTE ] ECO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:21.590577] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T18:27:21.851363] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-01. [2025-10-20T18:27:22.651402] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-01. [2025-10-20T18:27:23.175468] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T18:27:23.451371] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-10. [2025-10-20T18:27:23.917092] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T18:27:23.917848] [INFO ] [REMOTE ] ECCX: Full fetch complete (1450 records). [2025-10-20T18:27:23.917917] [DEBUG] [VALIDATOR ] ECCX: Validating primary data. [2025-10-20T18:27:24.096601] [WARN ] [VALIDATOR ] ECCX: Validation failed - insufficient historical data. Have 195 unique days, need 200 days [2025-10-20T18:27:24.096837] [FATAL] [MAIN ] ECCX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:27:24.096922] [WARN ] [MAIN ] ECCX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:27:24.097015] [INFO ] [LOCAL ] ECCX: Starting deletion of primary data. [2025-10-20T18:27:24.097097] [DEBUG] [LOCAL ] ECCX: No primary data file found to delete. [2025-10-20T18:27:24.097134] [INFO ] [LOCAL ] ECCX: Starting deletion of derivative data. [2025-10-20T18:27:24.097172] [DEBUG] [LOCAL ] ECCX: No derivative data file found to delete. [2025-10-20T18:27:24.097226] [INFO ] [MAIN ] Update Progress: 688/2367 symbols completed. ECCX took 23.8s. Estimated remaining: 3h 33m 7s. Rate: 7.9 symbols/min [2025-10-20T18:27:24.256413] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-02. [2025-10-20T18:27:24.285565] [INFO ] [MAIN ] ECVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:24.330963] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:27:24.377105] [DEBUG] [LOCAL ] ECVT: Loaded 43948 existing records from primary storage. [2025-10-20T18:27:24.377421] [INFO ] [MAIN ] ECVT: Loaded 43948 existing records. [2025-10-20T18:27:24.564355] [WARN ] [MAIN ] ECVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:24.564522] [INFO ] [MAIN ] ECVT: Wiping existing primary data before full refresh. [2025-10-20T18:27:24.564567] [INFO ] [LOCAL ] ECVT: Starting deletion of primary data. [2025-10-20T18:27:24.564925] [DEBUG] [LOCAL ] ECVT: Deleted primary data file. [2025-10-20T18:27:24.564980] [INFO ] [MAIN ] ECVT: Starting full primary refresh. [2025-10-20T18:27:24.909318] [INFO ] [REMOTE ] ECVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:25.051332] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-01. [2025-10-20T18:27:25.722799] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T18:27:25.851360] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-03. [2025-10-20T18:27:26.452674] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T18:27:26.651392] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-02. [2025-10-20T18:27:27.277761] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T18:27:27.451420] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-04. [2025-10-20T18:27:27.805893] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T18:27:28.251393] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-02. [2025-10-20T18:27:28.612413] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T18:27:29.051417] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-02. [2025-10-20T18:27:29.851361] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-03. [2025-10-20T18:27:29.916448] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:27:30.428970] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:27:30.651364] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-03. [2025-10-20T18:27:31.037746] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T18:27:31.451378] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-03. [2025-10-20T18:27:32.003294] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T18:27:32.251381] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-04. [2025-10-20T18:27:33.050981] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T18:27:33.060509] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-05. [2025-10-20T18:27:33.686189] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:27:33.851373] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-04. [2025-10-20T18:27:34.577602] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T18:27:34.651374] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-05. [2025-10-20T18:27:34.760347] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T18:27:35.451369] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-04. [2025-10-20T18:27:35.818190] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:27:36.251379] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-05. [2025-10-20T18:27:37.059780] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-06. [2025-10-20T18:27:37.085008] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:27:37.292181] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:27:37.851369] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-05. [2025-10-20T18:27:38.651380] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-06. [2025-10-20T18:27:38.755593] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T18:27:39.388184] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:27:39.451371] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-07. [2025-10-20T18:27:39.527607] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T18:27:40.251400] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-06. [2025-10-20T18:27:40.652499] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T18:27:41.051427] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-07. [2025-10-20T18:27:41.213164] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T18:27:41.851450] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-06. [2025-10-20T18:27:42.116440] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:27:42.651341] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-08. [2025-10-20T18:27:43.232058] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:27:43.451445] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-08. [2025-10-20T18:27:43.672433] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T18:27:44.251416] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-09. [2025-10-20T18:27:44.947271] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T18:27:45.051402] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-07. [2025-10-20T18:27:45.660392] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T18:27:45.851391] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-07. [2025-10-20T18:27:46.651423] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-10. [2025-10-20T18:27:46.854408] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:27:46.905965] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T18:27:47.451397] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-08. [2025-10-20T18:27:47.666327] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T18:27:47.671270] [INFO ] [REMOTE ] ECG: Full fetch complete (50864 records). [2025-10-20T18:27:47.671357] [DEBUG] [VALIDATOR ] ECG: Validating primary data. [2025-10-20T18:27:48.043947] [DEBUG] [VALIDATOR ] ECG: All expected trading days are present. [2025-10-20T18:27:48.044054] [DEBUG] [VALIDATOR ] ECG: Primary data validation passed. [2025-10-20T18:27:48.044751] [INFO ] [MAIN ] ECG: Fetched primary data is valid. [2025-10-20T18:27:48.044813] [DEBUG] [LOCAL ] ECG: Saving primary data. [2025-10-20T18:27:48.045958] [DEBUG] [LOCAL ] ECG: Trimmed 50864 records to 32902 (keeping 200 days of data) [2025-10-20T18:27:48.232232] [DEBUG] [LOCAL ] ECG: Primary data saved (32902 records). [2025-10-20T18:27:48.232363] [INFO ] [MAIN ] ECG: Full history saved. [2025-10-20T18:27:48.232440] [INFO ] [MAIN ] Update Progress: 689/2367 symbols completed. ECG took 30.8s. Estimated remaining: 3h 33m 40s. Rate: 7.9 symbols/min [2025-10-20T18:27:48.263114] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-09. [2025-10-20T18:27:48.402713] [INFO ] [MAIN ] ED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:48.452315] [DEBUG] [LOCAL ] ED: Loaded 53411 existing records from primary storage. [2025-10-20T18:27:48.452686] [INFO ] [MAIN ] ED: Loaded 53411 existing records. [2025-10-20T18:27:48.628661] [WARN ] [MAIN ] ED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:48.628945] [INFO ] [MAIN ] ED: Wiping existing primary data before full refresh. [2025-10-20T18:27:48.629081] [INFO ] [LOCAL ] ED: Starting deletion of primary data. [2025-10-20T18:27:48.629759] [DEBUG] [LOCAL ] ED: Deleted primary data file. [2025-10-20T18:27:48.629828] [INFO ] [MAIN ] ED: Starting full primary refresh. [2025-10-20T18:27:48.891041] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T18:27:49.036981] [INFO ] [REMOTE ] ED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:49.051334] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-01. [2025-10-20T18:27:49.851344] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-08. [2025-10-20T18:27:49.980225] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:27:50.423850] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T18:27:50.651354] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-09. [2025-10-20T18:27:50.946740] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:27:51.451384] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-02. [2025-10-20T18:27:51.941167] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T18:27:52.251428] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-10. [2025-10-20T18:27:53.051407] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-10. [2025-10-20T18:27:53.096445] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:27:53.763468] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T18:27:53.770215] [INFO ] [REMOTE ] ECVT: Full fetch complete (60056 records). [2025-10-20T18:27:53.770321] [DEBUG] [VALIDATOR ] ECVT: Validating primary data. [2025-10-20T18:27:53.771107] [FATAL] [VALIDATOR ] ECVT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:27:53.771194] [FATAL] [MAIN ] ECVT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:27:53.771237] [WARN ] [MAIN ] ECVT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:27:53.771275] [INFO ] [LOCAL ] ECVT: Starting deletion of primary data. [2025-10-20T18:27:53.771333] [DEBUG] [LOCAL ] ECVT: No primary data file found to delete. [2025-10-20T18:27:53.771368] [INFO ] [LOCAL ] ECVT: Starting deletion of derivative data. [2025-10-20T18:27:53.771552] [DEBUG] [LOCAL ] ECVT: Deleted derivative data file. [2025-10-20T18:27:53.772274] [INFO ] [MAIN ] Update Progress: 690/2367 symbols completed. ECVT took 29.7s. Estimated remaining: 3h 33m 27s. Rate: 7.9 symbols/min [2025-10-20T18:27:53.860983] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-03. [2025-10-20T18:27:53.940583] [INFO ] [MAIN ] EDD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:53.940791] [DEBUG] [LOCAL ] EDD: No existing data file found in primary storage. [2025-10-20T18:27:53.940835] [WARN ] [MAIN ] EDD: No existing data found, performing full refresh instead. [2025-10-20T18:27:53.940949] [INFO ] [MAIN ] EDD: Wiping existing primary data before full refresh. [2025-10-20T18:27:53.940991] [INFO ] [LOCAL ] EDD: Starting deletion of primary data. [2025-10-20T18:27:53.941046] [DEBUG] [LOCAL ] EDD: No primary data file found to delete. [2025-10-20T18:27:53.941077] [INFO ] [MAIN ] EDD: Starting full primary refresh. [2025-10-20T18:27:54.294395] [INFO ] [REMOTE ] EDD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:54.651351] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-01. [2025-10-20T18:27:54.739688] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T18:27:54.747625] [INFO ] [REMOTE ] ECL: Full fetch complete (68357 records). [2025-10-20T18:27:54.747792] [DEBUG] [VALIDATOR ] ECL: Validating primary data. [2025-10-20T18:27:55.139262] [DEBUG] [VALIDATOR ] ECL: All expected trading days are present. [2025-10-20T18:27:55.139333] [DEBUG] [VALIDATOR ] ECL: Primary data validation passed. [2025-10-20T18:27:55.140175] [INFO ] [MAIN ] ECL: Fetched primary data is valid. [2025-10-20T18:27:55.140236] [DEBUG] [LOCAL ] ECL: Saving primary data. [2025-10-20T18:27:55.141419] [DEBUG] [LOCAL ] ECL: Trimmed 68357 records to 46730 (keeping 200 days of data) [2025-10-20T18:27:55.427880] [DEBUG] [LOCAL ] ECL: Primary data saved (46730 records). [2025-10-20T18:27:55.428057] [INFO ] [MAIN ] ECL: Full history saved. [2025-10-20T18:27:55.428951] [INFO ] [MAIN ] Update Progress: 691/2367 symbols completed. ECL took 37.1s. Estimated remaining: 3h 33m 5s. Rate: 7.9 symbols/min [2025-10-20T18:27:55.459742] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-09. [2025-10-20T18:27:55.602725] [INFO ] [MAIN ] EDF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:55.617335] [DEBUG] [LOCAL ] EDF: Loaded 13279 existing records from primary storage. [2025-10-20T18:27:55.617552] [INFO ] [MAIN ] EDF: Loaded 13279 existing records. [2025-10-20T18:27:55.800151] [WARN ] [MAIN ] EDF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:55.800482] [INFO ] [MAIN ] EDF: Wiping existing primary data before full refresh. [2025-10-20T18:27:55.800716] [INFO ] [LOCAL ] EDF: Starting deletion of primary data. [2025-10-20T18:27:55.814802] [DEBUG] [LOCAL ] EDF: Deleted primary data file. [2025-10-20T18:27:55.815690] [INFO ] [MAIN ] EDF: Starting full primary refresh. [2025-10-20T18:27:55.816407] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:27:55.960314] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T18:27:56.220490] [INFO ] [REMOTE ] EDF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:56.251327] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-01. [2025-10-20T18:27:56.588403] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T18:27:57.016015] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T18:27:57.051328] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-02. [2025-10-20T18:27:57.851425] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-10. [2025-10-20T18:27:58.178891] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:27:58.651421] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-04. [2025-10-20T18:27:58.925411] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T18:27:58.928528] [INFO ] [REMOTE ] ECO: Full fetch complete (28402 records). [2025-10-20T18:27:58.928627] [DEBUG] [VALIDATOR ] ECO: Validating primary data. [2025-10-20T18:27:59.310518] [DEBUG] [VALIDATOR ] ECO: All expected trading days are present. [2025-10-20T18:27:59.310598] [DEBUG] [VALIDATOR ] ECO: Primary data validation passed. [2025-10-20T18:27:59.311179] [INFO ] [MAIN ] ECO: Fetched primary data is valid. [2025-10-20T18:27:59.311243] [DEBUG] [LOCAL ] ECO: Saving primary data. [2025-10-20T18:27:59.312005] [DEBUG] [LOCAL ] ECO: Trimmed 28402 records to 18204 (keeping 200 days of data) [2025-10-20T18:27:59.419079] [DEBUG] [LOCAL ] ECO: Primary data saved (18204 records). [2025-10-20T18:27:59.419248] [INFO ] [MAIN ] ECO: Full history saved. [2025-10-20T18:27:59.419341] [INFO ] [MAIN ] Update Progress: 692/2367 symbols completed. ECO took 39.0s. Estimated remaining: 3h 32m 48s. Rate: 7.9 symbols/min [2025-10-20T18:27:59.456473] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-03. [2025-10-20T18:27:59.594093] [INFO ] [MAIN ] EDN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:59.613054] [DEBUG] [LOCAL ] EDN: Loaded 19338 existing records from primary storage. [2025-10-20T18:27:59.613280] [INFO ] [MAIN ] EDN: Loaded 19338 existing records. [2025-10-20T18:27:59.803289] [WARN ] [MAIN ] EDN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:59.803548] [INFO ] [MAIN ] EDN: Wiping existing primary data before full refresh. [2025-10-20T18:27:59.803597] [INFO ] [LOCAL ] EDN: Starting deletion of primary data. [2025-10-20T18:27:59.803801] [DEBUG] [LOCAL ] EDN: Deleted primary data file. [2025-10-20T18:27:59.803854] [INFO ] [MAIN ] EDN: Starting full primary refresh. [2025-10-20T18:28:00.195110] [INFO ] [REMOTE ] EDN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:00.251360] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-02. [2025-10-20T18:28:00.531554] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T18:28:00.561953] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T18:28:01.051396] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-01. [2025-10-20T18:28:01.579522] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:28:01.851409] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-04. [2025-10-20T18:28:02.264157] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T18:28:02.651352] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-03. [2025-10-20T18:28:02.801278] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T18:28:03.451431] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-05. [2025-10-20T18:28:03.886317] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T18:28:04.251376] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-02. [2025-10-20T18:28:05.051358] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-04. [2025-10-20T18:28:05.357241] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T18:28:05.386547] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:28:05.851409] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-03. [2025-10-20T18:28:06.031286] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T18:28:06.651369] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-05. [2025-10-20T18:28:06.817120] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T18:28:07.451443] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-04. [2025-10-20T18:28:07.890697] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T18:28:08.251405] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-06. [2025-10-20T18:28:08.472259] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T18:28:09.051363] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-06. [2025-10-20T18:28:09.851343] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-05. [2025-10-20T18:28:09.897269] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:28:10.124504] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T18:28:10.651418] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-07. [2025-10-20T18:28:10.778411] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:28:11.451390] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-07. [2025-10-20T18:28:12.251398] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-05. [2025-10-20T18:28:12.586494] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T18:28:12.838349] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:28:12.969500] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T18:28:13.051342] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-08. [2025-10-20T18:28:13.851415] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-06. [2025-10-20T18:28:14.651393] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-06. [2025-10-20T18:28:14.782884] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T18:28:14.864329] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:28:15.451361] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T18:28:15.451433] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-09. [2025-10-20T18:28:16.251372] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-07. [2025-10-20T18:28:16.912019] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T18:28:16.948410] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:28:17.051391] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-08. [2025-10-20T18:28:17.851419] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-10. [2025-10-20T18:28:18.125558] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T18:28:18.651392] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-07. [2025-10-20T18:28:19.451364] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-08. [2025-10-20T18:28:19.641780] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T18:28:19.678554] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:28:19.686439] [INFO ] [REMOTE ] ED: Full fetch complete (76405 records). [2025-10-20T18:28:19.686606] [DEBUG] [VALIDATOR ] ED: Validating primary data. [2025-10-20T18:28:20.086457] [DEBUG] [VALIDATOR ] ED: All expected trading days are present. [2025-10-20T18:28:20.086570] [DEBUG] [VALIDATOR ] ED: Primary data validation passed. [2025-10-20T18:28:20.087721] [INFO ] [MAIN ] ED: Fetched primary data is valid. [2025-10-20T18:28:20.087824] [DEBUG] [LOCAL ] ED: Saving primary data. [2025-10-20T18:28:20.097004] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T18:28:20.101372] [DEBUG] [LOCAL ] ED: Trimmed 76405 records to 52649 (keeping 200 days of data) [2025-10-20T18:28:20.295303] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-09. [2025-10-20T18:28:20.454201] [DEBUG] [LOCAL ] ED: Primary data saved (52649 records). [2025-10-20T18:28:20.454356] [INFO ] [MAIN ] ED: Full history saved. [2025-10-20T18:28:20.454436] [INFO ] [MAIN ] Update Progress: 693/2367 symbols completed. ED took 32.2s. Estimated remaining: 3h 33m 13s. Rate: 7.9 symbols/min [2025-10-20T18:28:20.672235] [INFO ] [MAIN ] EDU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:20.719245] [DEBUG] [LOCAL ] EDU: Loaded 45507 existing records from primary storage. [2025-10-20T18:28:20.719621] [INFO ] [MAIN ] EDU: Loaded 45507 existing records. [2025-10-20T18:28:20.913333] [WARN ] [MAIN ] EDU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:20.913579] [INFO ] [MAIN ] EDU: Wiping existing primary data before full refresh. [2025-10-20T18:28:20.913630] [INFO ] [LOCAL ] EDU: Starting deletion of primary data. [2025-10-20T18:28:20.914107] [DEBUG] [LOCAL ] EDU: Deleted primary data file. [2025-10-20T18:28:20.914151] [INFO ] [MAIN ] EDU: Starting full primary refresh. [2025-10-20T18:28:21.061815] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-08. [2025-10-20T18:28:21.286576] [INFO ] [REMOTE ] EDU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:21.488589] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T18:28:21.851355] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-01. [2025-10-20T18:28:22.077194] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T18:28:22.651386] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-09. [2025-10-20T18:28:23.174282] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T18:28:23.451398] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-09. [2025-10-20T18:28:23.906233] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:28:24.103459] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T18:28:24.251373] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-10. [2025-10-20T18:28:25.051408] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-10. [2025-10-20T18:28:25.525744] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T18:28:25.530024] [INFO ] [REMOTE ] EDD: Full fetch complete (38510 records). [2025-10-20T18:28:25.530119] [DEBUG] [VALIDATOR ] EDD: Validating primary data. [2025-10-20T18:28:25.530685] [FATAL] [VALIDATOR ] EDD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:28:25.530767] [FATAL] [MAIN ] EDD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:28:25.530808] [WARN ] [MAIN ] EDD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:28:25.530845] [INFO ] [LOCAL ] EDD: Starting deletion of primary data. [2025-10-20T18:28:25.530914] [DEBUG] [LOCAL ] EDD: No primary data file found to delete. [2025-10-20T18:28:25.530969] [INFO ] [LOCAL ] EDD: Starting deletion of derivative data. [2025-10-20T18:28:25.531012] [DEBUG] [LOCAL ] EDD: No derivative data file found to delete. [2025-10-20T18:28:25.531064] [INFO ] [MAIN ] Update Progress: 694/2367 symbols completed. EDD took 31.8s. Estimated remaining: 3h 32m 59s. Rate: 7.9 symbols/min [2025-10-20T18:28:25.699202] [INFO ] [MAIN ] EE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:25.732456] [DEBUG] [LOCAL ] EE: Loaded 33450 existing records from primary storage. [2025-10-20T18:28:25.732764] [INFO ] [MAIN ] EE: Loaded 33450 existing records. [2025-10-20T18:28:25.856578] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-10. [2025-10-20T18:28:25.918231] [WARN ] [MAIN ] EE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:25.918430] [INFO ] [MAIN ] EE: Wiping existing primary data before full refresh. [2025-10-20T18:28:25.918478] [INFO ] [LOCAL ] EE: Starting deletion of primary data. [2025-10-20T18:28:25.918727] [DEBUG] [LOCAL ] EE: Deleted primary data file. [2025-10-20T18:28:25.918784] [INFO ] [MAIN ] EE: Starting full primary refresh. [2025-10-20T18:28:26.269224] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T18:28:26.296851] [INFO ] [REMOTE ] EE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:26.297900] [INFO ] [REMOTE ] EDN: Full fetch complete (30037 records). [2025-10-20T18:28:26.298036] [DEBUG] [VALIDATOR ] EDN: Validating primary data. [2025-10-20T18:28:26.656515] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-01. [2025-10-20T18:28:26.702008] [DEBUG] [VALIDATOR ] EDN: All expected trading days are present. [2025-10-20T18:28:26.702085] [DEBUG] [VALIDATOR ] EDN: Primary data validation passed. [2025-10-20T18:28:26.702777] [INFO ] [MAIN ] EDN: Fetched primary data is valid. [2025-10-20T18:28:26.702993] [DEBUG] [LOCAL ] EDN: Saving primary data. [2025-10-20T18:28:26.703797] [DEBUG] [LOCAL ] EDN: Trimmed 30037 records to 19210 (keeping 200 days of data) [2025-10-20T18:28:26.814149] [DEBUG] [LOCAL ] EDN: Primary data saved (19210 records). [2025-10-20T18:28:26.814284] [INFO ] [MAIN ] EDN: Full history saved. [2025-10-20T18:28:26.814361] [INFO ] [MAIN ] Update Progress: 695/2367 symbols completed. EDN took 27.4s. Estimated remaining: 3h 32m 36s. Rate: 7.9 symbols/min [2025-10-20T18:28:26.981266] [INFO ] [MAIN ] EEA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:26.981541] [DEBUG] [LOCAL ] EEA: No existing data file found in primary storage. [2025-10-20T18:28:26.981592] [WARN ] [MAIN ] EEA: No existing data found, performing full refresh instead. [2025-10-20T18:28:26.981730] [INFO ] [MAIN ] EEA: Wiping existing primary data before full refresh. [2025-10-20T18:28:26.981775] [INFO ] [LOCAL ] EEA: Starting deletion of primary data. [2025-10-20T18:28:26.981817] [DEBUG] [LOCAL ] EEA: No primary data file found to delete. [2025-10-20T18:28:26.981848] [INFO ] [MAIN ] EEA: Starting full primary refresh. [2025-10-20T18:28:27.077214] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T18:28:27.098787] [INFO ] [REMOTE ] EDF: Full fetch complete (20306 records). [2025-10-20T18:28:27.109195] [DEBUG] [VALIDATOR ] EDF: Validating primary data. [2025-10-20T18:28:27.132443] [FATAL] [VALIDATOR ] EDF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:28:27.151668] [FATAL] [MAIN ] EDF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:28:27.168214] [WARN ] [MAIN ] EDF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:28:27.214775] [INFO ] [LOCAL ] EDF: Starting deletion of primary data. [2025-10-20T18:28:27.252297] [DEBUG] [LOCAL ] EDF: No primary data file found to delete. [2025-10-20T18:28:27.270781] [INFO ] [LOCAL ] EDF: Starting deletion of derivative data. [2025-10-20T18:28:27.273403] [DEBUG] [LOCAL ] EDF: Deleted derivative data file. [2025-10-20T18:28:27.278946] [INFO ] [MAIN ] Update Progress: 696/2367 symbols completed. EDF took 31.8s. Estimated remaining: 3h 32m 11s. Rate: 7.9 symbols/min [2025-10-20T18:28:27.413074] [INFO ] [REMOTE ] EEA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:27.458727] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-02. [2025-10-20T18:28:27.564613] [INFO ] [MAIN ] EEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:27.575699] [DEBUG] [LOCAL ] EEX: Loaded 10348 existing records from primary storage. [2025-10-20T18:28:27.575940] [INFO ] [MAIN ] EEX: Loaded 10348 existing records. [2025-10-20T18:28:27.768932] [WARN ] [MAIN ] EEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:27.769225] [INFO ] [MAIN ] EEX: Wiping existing primary data before full refresh. [2025-10-20T18:28:27.769302] [INFO ] [LOCAL ] EEX: Starting deletion of primary data. [2025-10-20T18:28:27.784050] [DEBUG] [LOCAL ] EEX: Deleted primary data file. [2025-10-20T18:28:27.784193] [INFO ] [MAIN ] EEX: Starting full primary refresh. [2025-10-20T18:28:27.817592] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:28:28.170127] [INFO ] [REMOTE ] EEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:28.251429] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-02. [2025-10-20T18:28:29.051382] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-01. [2025-10-20T18:28:29.227345] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T18:28:29.372768] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T18:28:29.483867] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-01 (0.43 seconds). [2025-10-20T18:28:29.851374] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-03. [2025-10-20T18:28:30.651424] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-02. [2025-10-20T18:28:31.196078] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T18:28:31.444638] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:28:31.456675] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-03. [2025-10-20T18:28:32.075232] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:28:32.251384] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-04. [2025-10-20T18:28:33.051405] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-04. [2025-10-20T18:28:33.621176] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T18:28:33.851347] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-03. [2025-10-20T18:28:33.901968] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:28:34.651385] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-05. [2025-10-20T18:28:34.918449] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T18:28:35.451367] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-05. [2025-10-20T18:28:35.886216] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T18:28:35.943476] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T18:28:36.251407] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-06. [2025-10-20T18:28:36.553014] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-06 (0.30 seconds). [2025-10-20T18:28:37.051344] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-01. [2025-10-20T18:28:37.851436] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-07. [2025-10-20T18:28:38.011583] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T18:28:38.181248] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-10-20T18:28:38.651377] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-08. [2025-10-20T18:28:39.133463] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T18:28:39.451381] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-04. [2025-10-20T18:28:40.251397] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-06. [2025-10-20T18:28:40.959810] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:28:41.051382] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-05. [2025-10-20T18:28:41.845091] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T18:28:41.853592] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-09. [2025-10-20T18:28:42.228857] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-10-20T18:28:42.321509] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:28:42.651375] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-07. [2025-10-20T18:28:43.451432] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-02. [2025-10-20T18:28:44.243702] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T18:28:44.251318] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-03. [2025-10-20T18:28:44.464940] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:28:45.051409] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-10. [2025-10-20T18:28:45.316711] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T18:28:45.624229] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T18:28:45.625013] [INFO ] [REMOTE ] EEA: Full fetch complete (3748 records). [2025-10-20T18:28:45.625079] [DEBUG] [VALIDATOR ] EEA: Validating primary data. [2025-10-20T18:28:45.806907] [WARN ] [VALIDATOR ] EEA: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T18:28:45.807149] [FATAL] [MAIN ] EEA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:28:45.807247] [WARN ] [MAIN ] EEA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:28:45.807319] [INFO ] [LOCAL ] EEA: Starting deletion of primary data. [2025-10-20T18:28:45.807385] [DEBUG] [LOCAL ] EEA: No primary data file found to delete. [2025-10-20T18:28:45.807422] [INFO ] [LOCAL ] EEA: Starting deletion of derivative data. [2025-10-20T18:28:45.807463] [DEBUG] [LOCAL ] EEA: No derivative data file found to delete. [2025-10-20T18:28:45.807521] [INFO ] [MAIN ] Update Progress: 697/2367 symbols completed. EEA took 19.0s. Estimated remaining: 3h 32m 30s. Rate: 7.9 symbols/min [2025-10-20T18:28:45.860996] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-08. [2025-10-20T18:28:45.990780] [INFO ] [MAIN ] EFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:46.035013] [DEBUG] [LOCAL ] EFC: Loaded 47042 existing records from primary storage. [2025-10-20T18:28:46.035332] [INFO ] [MAIN ] EFC: Loaded 47042 existing records. [2025-10-20T18:28:46.219137] [WARN ] [MAIN ] EFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:46.219365] [INFO ] [MAIN ] EFC: Wiping existing primary data before full refresh. [2025-10-20T18:28:46.219412] [INFO ] [LOCAL ] EFC: Starting deletion of primary data. [2025-10-20T18:28:46.219785] [DEBUG] [LOCAL ] EFC: Deleted primary data file. [2025-10-20T18:28:46.219843] [INFO ] [MAIN ] EFC: Starting full primary refresh. [2025-10-20T18:28:46.574111] [INFO ] [REMOTE ] EFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:46.651384] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-01. [2025-10-20T18:28:47.397428] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:28:47.451336] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-09. [2025-10-20T18:28:48.260520] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-04. [2025-10-20T18:28:48.263423] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T18:28:48.670231] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T18:28:48.975101] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T18:28:49.051348] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-05. [2025-10-20T18:28:49.855399] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-10. [2025-10-20T18:28:49.858284] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T18:28:50.651437] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-06. [2025-10-20T18:28:51.440724] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T18:28:51.451329] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-07. [2025-10-20T18:28:51.494275] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T18:28:51.501276] [INFO ] [REMOTE ] EDU: Full fetch complete (67148 records). [2025-10-20T18:28:51.501391] [DEBUG] [VALIDATOR ] EDU: Validating primary data. [2025-10-20T18:28:51.882229] [DEBUG] [VALIDATOR ] EDU: All expected trading days are present. [2025-10-20T18:28:51.882312] [DEBUG] [VALIDATOR ] EDU: Primary data validation passed. [2025-10-20T18:28:51.883112] [INFO ] [MAIN ] EDU: Fetched primary data is valid. [2025-10-20T18:28:51.883174] [DEBUG] [LOCAL ] EDU: Saving primary data. [2025-10-20T18:28:51.884476] [DEBUG] [LOCAL ] EDU: Trimmed 67148 records to 44811 (keeping 200 days of data) [2025-10-20T18:28:52.088857] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T18:28:52.149723] [DEBUG] [LOCAL ] EDU: Primary data saved (44811 records). [2025-10-20T18:28:52.149853] [INFO ] [MAIN ] EDU: Full history saved. [2025-10-20T18:28:52.149952] [INFO ] [MAIN ] Update Progress: 698/2367 symbols completed. EDU took 31.7s. Estimated remaining: 3h 32m 19s. Rate: 7.9 symbols/min [2025-10-20T18:28:52.256484] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-06. [2025-10-20T18:28:52.320269] [INFO ] [MAIN ] EFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:52.336332] [DEBUG] [LOCAL ] EFR: Loaded 15309 existing records from primary storage. [2025-10-20T18:28:52.336543] [INFO ] [MAIN ] EFR: Loaded 15309 existing records. [2025-10-20T18:28:52.509564] [WARN ] [MAIN ] EFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:52.509799] [INFO ] [MAIN ] EFR: Wiping existing primary data before full refresh. [2025-10-20T18:28:52.509846] [INFO ] [LOCAL ] EFR: Starting deletion of primary data. [2025-10-20T18:28:52.510128] [DEBUG] [LOCAL ] EFR: Deleted primary data file. [2025-10-20T18:28:52.510183] [INFO ] [MAIN ] EFR: Starting full primary refresh. [2025-10-20T18:28:52.864480] [INFO ] [REMOTE ] EFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:53.051361] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-01. [2025-10-20T18:28:53.683580] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T18:28:53.698491] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T18:28:53.851405] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-07. [2025-10-20T18:28:54.651435] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-08. [2025-10-20T18:28:55.244104] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:28:55.244795] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T18:28:55.451397] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-02. [2025-10-20T18:28:56.251427] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-08. [2025-10-20T18:28:57.053033] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-02. [2025-10-20T18:28:57.055404] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T18:28:57.238071] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:28:57.804920] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:28:57.851370] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-03. [2025-10-20T18:28:58.651376] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-09. [2025-10-20T18:28:59.451338] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-03. [2025-10-20T18:28:59.826588] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T18:28:59.852895] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:29:00.251362] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-09. [2025-10-20T18:29:00.277324] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:29:01.051354] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-04. [2025-10-20T18:29:01.129535] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T18:29:01.851354] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-04. [2025-10-20T18:29:02.651364] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-10. [2025-10-20T18:29:02.718697] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:29:02.849492] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T18:29:03.451394] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-10. [2025-10-20T18:29:03.945565] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T18:29:03.951741] [INFO ] [REMOTE ] EE: Full fetch complete (44238 records). [2025-10-20T18:29:03.951877] [DEBUG] [VALIDATOR ] EE: Validating primary data. [2025-10-20T18:29:04.253967] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-05. [2025-10-20T18:29:04.253900] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T18:29:04.268571] [INFO ] [REMOTE ] EEX: Full fetch complete (18458 records). [2025-10-20T18:29:04.269499] [DEBUG] [VALIDATOR ] EEX: Validating primary data. [2025-10-20T18:29:04.491009] [DEBUG] [VALIDATOR ] EE: All expected trading days are present. [2025-10-20T18:29:04.491440] [DEBUG] [VALIDATOR ] EE: Primary data validation passed. [2025-10-20T18:29:04.502689] [INFO ] [MAIN ] EE: Fetched primary data is valid. [2025-10-20T18:29:04.507908] [DEBUG] [LOCAL ] EE: Saving primary data. [2025-10-20T18:29:04.574583] [DEBUG] [LOCAL ] EE: Trimmed 44238 records to 32694 (keeping 200 days of data) [2025-10-20T18:29:04.929747] [DEBUG] [VALIDATOR ] EEX: All expected trading days are present. [2025-10-20T18:29:04.930171] [DEBUG] [VALIDATOR ] EEX: Primary data validation passed. [2025-10-20T18:29:04.931685] [INFO ] [MAIN ] EEX: Fetched primary data is valid. [2025-10-20T18:29:04.931765] [DEBUG] [LOCAL ] EEX: Saving primary data. [2025-10-20T18:29:04.938689] [DEBUG] [LOCAL ] EEX: Trimmed 18458 records to 10029 (keeping 200 days of data) [2025-10-20T18:29:05.053802] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-05. [2025-10-20T18:29:05.074202] [DEBUG] [LOCAL ] EEX: Primary data saved (10029 records). [2025-10-20T18:29:05.077271] [INFO ] [MAIN ] EEX: Full history saved. [2025-10-20T18:29:05.077697] [INFO ] [MAIN ] Update Progress: 699/2367 symbols completed. EEX took 37.8s. Estimated remaining: 3h 32m 24s. Rate: 7.9 symbols/min [2025-10-20T18:29:05.116794] [DEBUG] [LOCAL ] EE: Primary data saved (32694 records). [2025-10-20T18:29:05.122119] [INFO ] [MAIN ] EE: Full history saved. [2025-10-20T18:29:05.136492] [INFO ] [MAIN ] Update Progress: 700/2367 symbols completed. EE took 39.6s. Estimated remaining: 3h 31m 59s. Rate: 7.9 symbols/min [2025-10-20T18:29:05.322302] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T18:29:05.466262] [INFO ] [MAIN ] EFT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:05.486343] [INFO ] [MAIN ] EFX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:05.502457] [DEBUG] [LOCAL ] EFT: Loaded 13215 existing records from primary storage. [2025-10-20T18:29:05.502795] [INFO ] [MAIN ] EFT: Loaded 13215 existing records. [2025-10-20T18:29:05.710103] [WARN ] [MAIN ] EFT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:05.723804] [INFO ] [MAIN ] EFT: Wiping existing primary data before full refresh. [2025-10-20T18:29:05.723891] [INFO ] [LOCAL ] EFT: Starting deletion of primary data. [2025-10-20T18:29:05.724539] [DEBUG] [LOCAL ] EFX: Loaded 45309 existing records from primary storage. [2025-10-20T18:29:05.724864] [DEBUG] [LOCAL ] EFT: Deleted primary data file. [2025-10-20T18:29:05.724841] [INFO ] [MAIN ] EFX: Loaded 45309 existing records. [2025-10-20T18:29:05.730071] [INFO ] [MAIN ] EFT: Starting full primary refresh. [2025-10-20T18:29:05.856495] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-06. [2025-10-20T18:29:06.108268] [WARN ] [MAIN ] EFX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:06.192619] [INFO ] [MAIN ] EFX: Wiping existing primary data before full refresh. [2025-10-20T18:29:06.195094] [INFO ] [LOCAL ] EFX: Starting deletion of primary data. [2025-10-20T18:29:06.195445] [DEBUG] [LOCAL ] EFX: Deleted primary data file. [2025-10-20T18:29:06.206000] [INFO ] [MAIN ] EFX: Starting full primary refresh. [2025-10-20T18:29:06.363004] [INFO ] [REMOTE ] EFT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:06.642632] [INFO ] [REMOTE ] EFX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:06.651330] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-01. [2025-10-20T18:29:06.763837] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T18:29:06.903396] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:29:07.451438] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-06. [2025-10-20T18:29:08.251403] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-01. [2025-10-20T18:29:08.323332] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T18:29:08.728142] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T18:29:09.031480] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T18:29:09.051369] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-02. [2025-10-20T18:29:09.800524] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:29:09.851383] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-07. [2025-10-20T18:29:10.651386] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-02. [2025-10-20T18:29:11.451402] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-03. [2025-10-20T18:29:11.662029] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:29:12.088820] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:29:12.201395] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T18:29:12.251341] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-03. [2025-10-20T18:29:13.051432] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-04. [2025-10-20T18:29:13.852755] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-08. [2025-10-20T18:29:13.858092] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:29:14.083594] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T18:29:14.651413] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-07. [2025-10-20T18:29:15.451377] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-04. [2025-10-20T18:29:15.616279] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:29:15.663462] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T18:29:16.251374] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-09. [2025-10-20T18:29:17.051371] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-05. [2025-10-20T18:29:17.613355] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T18:29:17.642161] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:29:17.851343] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-08. [2025-10-20T18:29:18.208897] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T18:29:18.651421] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-10. [2025-10-20T18:29:18.744180] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T18:29:19.451359] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-06. [2025-10-20T18:29:20.251395] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-09. [2025-10-20T18:29:20.424858] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T18:29:20.520412] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:29:20.527453] [INFO ] [REMOTE ] EFC: Full fetch complete (66710 records). [2025-10-20T18:29:20.527611] [DEBUG] [VALIDATOR ] EFC: Validating primary data. [2025-10-20T18:29:20.917578] [DEBUG] [VALIDATOR ] EFC: All expected trading days are present. [2025-10-20T18:29:20.917689] [DEBUG] [VALIDATOR ] EFC: Primary data validation passed. [2025-10-20T18:29:20.918471] [INFO ] [MAIN ] EFC: Fetched primary data is valid. [2025-10-20T18:29:20.918513] [DEBUG] [LOCAL ] EFC: Saving primary data. [2025-10-20T18:29:20.919890] [DEBUG] [LOCAL ] EFC: Trimmed 66710 records to 46426 (keeping 200 days of data) [2025-10-20T18:29:21.097409] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-07. [2025-10-20T18:29:21.207061] [DEBUG] [LOCAL ] EFC: Primary data saved (46426 records). [2025-10-20T18:29:21.207198] [INFO ] [MAIN ] EFC: Full history saved. [2025-10-20T18:29:21.207274] [INFO ] [MAIN ] Update Progress: 701/2367 symbols completed. EFC took 35.4s. Estimated remaining: 3h 32m 11s. Rate: 7.9 symbols/min [2025-10-20T18:29:21.319772] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:29:21.395567] [INFO ] [MAIN ] EFXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:21.425498] [DEBUG] [LOCAL ] EFXT: Loaded 30200 existing records from primary storage. [2025-10-20T18:29:21.425836] [INFO ] [MAIN ] EFXT: Loaded 30200 existing records. [2025-10-20T18:29:21.603043] [WARN ] [MAIN ] EFXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:21.603246] [INFO ] [MAIN ] EFXT: Wiping existing primary data before full refresh. [2025-10-20T18:29:21.603292] [INFO ] [LOCAL ] EFXT: Starting deletion of primary data. [2025-10-20T18:29:21.603497] [DEBUG] [LOCAL ] EFXT: Deleted primary data file. [2025-10-20T18:29:21.603552] [INFO ] [MAIN ] EFXT: Starting full primary refresh. [2025-10-20T18:29:21.852376] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-10. [2025-10-20T18:29:21.947122] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T18:29:22.022793] [INFO ] [REMOTE ] EFXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:22.651402] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-05. [2025-10-20T18:29:22.828888] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T18:29:22.833277] [INFO ] [REMOTE ] EFR: Full fetch complete (19397 records). [2025-10-20T18:29:22.833444] [DEBUG] [VALIDATOR ] EFR: Validating primary data. [2025-10-20T18:29:22.834099] [FATAL] [VALIDATOR ] EFR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:29:22.834201] [FATAL] [MAIN ] EFR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:29:22.834256] [WARN ] [MAIN ] EFR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:29:22.834307] [INFO ] [LOCAL ] EFR: Starting deletion of primary data. [2025-10-20T18:29:22.834379] [DEBUG] [LOCAL ] EFR: No primary data file found to delete. [2025-10-20T18:29:22.834424] [INFO ] [LOCAL ] EFR: Starting deletion of derivative data. [2025-10-20T18:29:22.834684] [DEBUG] [LOCAL ] EFR: Deleted derivative data file. [2025-10-20T18:29:22.834793] [INFO ] [MAIN ] Update Progress: 702/2367 symbols completed. EFR took 30.7s. Estimated remaining: 3h 31m 49s. Rate: 7.9 symbols/min [2025-10-20T18:29:23.013121] [INFO ] [MAIN ] EG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:23.042154] [DEBUG] [LOCAL ] EG: Loaded 28182 existing records from primary storage. [2025-10-20T18:29:23.042460] [INFO ] [MAIN ] EG: Loaded 28182 existing records. [2025-10-20T18:29:23.222695] [WARN ] [MAIN ] EG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:23.222982] [INFO ] [MAIN ] EG: Wiping existing primary data before full refresh. [2025-10-20T18:29:23.223036] [INFO ] [LOCAL ] EG: Starting deletion of primary data. [2025-10-20T18:29:23.223218] [DEBUG] [LOCAL ] EG: Deleted primary data file. [2025-10-20T18:29:23.223273] [INFO ] [MAIN ] EG: Starting full primary refresh. [2025-10-20T18:29:23.460303] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-01. [2025-10-20T18:29:23.579984] [INFO ] [REMOTE ] EG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:24.251412] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-01. [2025-10-20T18:29:24.321515] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:29:25.051377] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-08. [2025-10-20T18:29:25.075523] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T18:29:25.371464] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T18:29:25.835676] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T18:29:25.851347] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-09. [2025-10-20T18:29:26.651375] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-02. [2025-10-20T18:29:26.684282] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:29:27.451380] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-02. [2025-10-20T18:29:27.850730] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:29:28.251377] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-10. [2025-10-20T18:29:28.786614] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T18:29:29.051401] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-06. [2025-10-20T18:29:29.296785] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T18:29:29.299884] [INFO ] [REMOTE ] EFT: Full fetch complete (17740 records). [2025-10-20T18:29:29.300014] [DEBUG] [VALIDATOR ] EFT: Validating primary data. [2025-10-20T18:29:29.764907] [DEBUG] [VALIDATOR ] EFT: All expected trading days are present. [2025-10-20T18:29:29.764994] [DEBUG] [VALIDATOR ] EFT: Primary data validation passed. [2025-10-20T18:29:29.765432] [INFO ] [MAIN ] EFT: Fetched primary data is valid. [2025-10-20T18:29:29.765496] [DEBUG] [LOCAL ] EFT: Saving primary data. [2025-10-20T18:29:29.766212] [DEBUG] [LOCAL ] EFT: Trimmed 17740 records to 12980 (keeping 200 days of data) [2025-10-20T18:29:29.851438] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-03. [2025-10-20T18:29:29.868000] [DEBUG] [LOCAL ] EFT: Primary data saved (12980 records). [2025-10-20T18:29:29.868154] [INFO ] [MAIN ] EFT: Full history saved. [2025-10-20T18:29:29.868238] [INFO ] [MAIN ] Update Progress: 703/2367 symbols completed. EFT took 24.8s. Estimated remaining: 3h 31m 40s. Rate: 7.9 symbols/min [2025-10-20T18:29:30.038147] [INFO ] [MAIN ] EGO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:30.089866] [DEBUG] [LOCAL ] EGO: Loaded 52505 existing records from primary storage. [2025-10-20T18:29:30.090206] [INFO ] [MAIN ] EGO: Loaded 52505 existing records. [2025-10-20T18:29:30.274868] [WARN ] [MAIN ] EGO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:30.275161] [INFO ] [MAIN ] EGO: Wiping existing primary data before full refresh. [2025-10-20T18:29:30.275221] [INFO ] [LOCAL ] EGO: Starting deletion of primary data. [2025-10-20T18:29:30.275420] [DEBUG] [LOCAL ] EGO: Deleted primary data file. [2025-10-20T18:29:30.275475] [INFO ] [MAIN ] EGO: Starting full primary refresh. [2025-10-20T18:29:30.658992] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-03. [2025-10-20T18:29:30.663752] [INFO ] [REMOTE ] EGO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:30.690984] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:29:31.315063] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T18:29:31.451346] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-01. [2025-10-20T18:29:31.901841] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T18:29:32.251368] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-04. [2025-10-20T18:29:33.051381] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-07. [2025-10-20T18:29:33.250076] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T18:29:33.723501] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T18:29:33.851380] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-05. [2025-10-20T18:29:34.651354] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-04. [2025-10-20T18:29:34.835325] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T18:29:35.451401] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-08. [2025-10-20T18:29:35.556298] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:29:36.161104] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:29:36.251439] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-02. [2025-10-20T18:29:37.051363] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-06. [2025-10-20T18:29:37.851336] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-05. [2025-10-20T18:29:37.943855] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T18:29:38.021155] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-08 (2.57 seconds). [2025-10-20T18:29:38.091401] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:29:38.651352] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-03. [2025-10-20T18:29:39.018872] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:29:39.451423] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-06. [2025-10-20T18:29:39.772866] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:29:40.251363] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-04. [2025-10-20T18:29:40.597263] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T18:29:41.051385] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-07. [2025-10-20T18:29:41.851353] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-09. [2025-10-20T18:29:42.014360] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:29:42.393508] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T18:29:42.651358] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-05. [2025-10-20T18:29:43.451350] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-08. [2025-10-20T18:29:43.518332] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T18:29:44.251367] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-07. [2025-10-20T18:29:44.510287] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T18:29:44.535236] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:29:45.051382] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-10. [2025-10-20T18:29:45.324765] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:29:45.851377] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-09. [2025-10-20T18:29:46.651418] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-06. [2025-10-20T18:29:46.918223] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:29:46.925596] [INFO ] [REMOTE ] EFX: Full fetch complete (64974 records). [2025-10-20T18:29:46.925747] [DEBUG] [VALIDATOR ] EFX: Validating primary data. [2025-10-20T18:29:47.131852] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:29:47.357213] [DEBUG] [VALIDATOR ] EFX: All expected trading days are present. [2025-10-20T18:29:47.357292] [DEBUG] [VALIDATOR ] EFX: Primary data validation passed. [2025-10-20T18:29:47.358076] [INFO ] [MAIN ] EFX: Fetched primary data is valid. [2025-10-20T18:29:47.358138] [DEBUG] [LOCAL ] EFX: Saving primary data. [2025-10-20T18:29:47.359393] [DEBUG] [LOCAL ] EFX: Trimmed 64974 records to 44781 (keeping 200 days of data) [2025-10-20T18:29:47.451737] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-08. [2025-10-20T18:29:47.638496] [DEBUG] [LOCAL ] EFX: Primary data saved (44781 records). [2025-10-20T18:29:47.638631] [INFO ] [MAIN ] EFX: Full history saved. [2025-10-20T18:29:47.639244] [INFO ] [MAIN ] Update Progress: 704/2367 symbols completed. EFX took 42.5s. Estimated remaining: 3h 31m 56s. Rate: 7.8 symbols/min [2025-10-20T18:29:47.822437] [INFO ] [MAIN ] EGP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:47.862355] [DEBUG] [LOCAL ] EGP: Loaded 30899 existing records from primary storage. [2025-10-20T18:29:47.862686] [INFO ] [MAIN ] EGP: Loaded 30899 existing records. [2025-10-20T18:29:48.049677] [WARN ] [MAIN ] EGP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:48.049921] [INFO ] [MAIN ] EGP: Wiping existing primary data before full refresh. [2025-10-20T18:29:48.049988] [INFO ] [LOCAL ] EGP: Starting deletion of primary data. [2025-10-20T18:29:48.050326] [DEBUG] [LOCAL ] EGP: Deleted primary data file. [2025-10-20T18:29:48.050401] [INFO ] [MAIN ] EGP: Starting full primary refresh. [2025-10-20T18:29:48.262690] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-10. [2025-10-20T18:29:48.415398] [INFO ] [REMOTE ] EGP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:48.442966] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T18:29:48.879764] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T18:29:49.051372] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-07. [2025-10-20T18:29:49.693026] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T18:29:49.697031] [INFO ] [REMOTE ] EG: Full fetch complete (43121 records). [2025-10-20T18:29:49.697149] [DEBUG] [VALIDATOR ] EG: Validating primary data. [2025-10-20T18:29:49.871245] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-09. [2025-10-20T18:29:50.069447] [DEBUG] [VALIDATOR ] EG: All expected trading days are present. [2025-10-20T18:29:50.069520] [DEBUG] [VALIDATOR ] EG: Primary data validation passed. [2025-10-20T18:29:50.070138] [INFO ] [MAIN ] EG: Fetched primary data is valid. [2025-10-20T18:29:50.070198] [DEBUG] [LOCAL ] EG: Saving primary data. [2025-10-20T18:29:50.071036] [DEBUG] [LOCAL ] EG: Trimmed 43121 records to 27781 (keeping 200 days of data) [2025-10-20T18:29:50.238719] [DEBUG] [LOCAL ] EG: Primary data saved (27781 records). [2025-10-20T18:29:50.238836] [INFO ] [MAIN ] EG: Full history saved. [2025-10-20T18:29:50.238913] [INFO ] [MAIN ] Update Progress: 705/2367 symbols completed. EG took 27.4s. Estimated remaining: 3h 31m 37s. Rate: 7.9 symbols/min [2025-10-20T18:29:50.410186] [INFO ] [MAIN ] EGY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:50.442010] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:29:50.484295] [DEBUG] [LOCAL ] EGY: Loaded 42160 existing records from primary storage. [2025-10-20T18:29:50.484566] [INFO ] [MAIN ] EGY: Loaded 42160 existing records. [2025-10-20T18:29:50.656788] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-08. [2025-10-20T18:29:50.661570] [WARN ] [MAIN ] EGY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:50.662669] [INFO ] [MAIN ] EGY: Wiping existing primary data before full refresh. [2025-10-20T18:29:50.662732] [INFO ] [LOCAL ] EGY: Starting deletion of primary data. [2025-10-20T18:29:50.662902] [DEBUG] [LOCAL ] EGY: Deleted primary data file. [2025-10-20T18:29:50.662943] [INFO ] [MAIN ] EGY: Starting full primary refresh. [2025-10-20T18:29:51.037486] [INFO ] [REMOTE ] EGY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:51.054484] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:29:51.451384] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-01. [2025-10-20T18:29:52.251349] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-01. [2025-10-20T18:29:52.445963] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T18:29:53.051403] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-10. [2025-10-20T18:29:53.106904] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T18:29:53.221903] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T18:29:53.851372] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-02. [2025-10-20T18:29:54.299795] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T18:29:54.307697] [INFO ] [REMOTE ] EFXT: Full fetch complete (44320 records). [2025-10-20T18:29:54.307846] [DEBUG] [VALIDATOR ] EFXT: Validating primary data. [2025-10-20T18:29:54.661860] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-02. [2025-10-20T18:29:54.702086] [DEBUG] [VALIDATOR ] EFXT: All expected trading days are present. [2025-10-20T18:29:54.702159] [DEBUG] [VALIDATOR ] EFXT: Primary data validation passed. [2025-10-20T18:29:54.702794] [INFO ] [MAIN ] EFXT: Fetched primary data is valid. [2025-10-20T18:29:54.702854] [DEBUG] [LOCAL ] EFXT: Saving primary data. [2025-10-20T18:29:54.703771] [DEBUG] [LOCAL ] EFXT: Trimmed 44320 records to 29637 (keeping 200 days of data) [2025-10-20T18:29:54.876461] [DEBUG] [LOCAL ] EFXT: Primary data saved (29637 records). [2025-10-20T18:29:54.876600] [INFO ] [MAIN ] EFXT: Full history saved. [2025-10-20T18:29:54.876710] [INFO ] [MAIN ] Update Progress: 706/2367 symbols completed. EFXT took 33.7s. Estimated remaining: 3h 31m 22s. Rate: 7.9 symbols/min [2025-10-20T18:29:55.056245] [INFO ] [MAIN ] EHAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:55.084261] [DEBUG] [LOCAL ] EHAB: Loaded 29656 existing records from primary storage. [2025-10-20T18:29:55.084529] [INFO ] [MAIN ] EHAB: Loaded 29656 existing records. [2025-10-20T18:29:55.235708] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:29:55.302466] [WARN ] [MAIN ] EHAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:55.302948] [INFO ] [MAIN ] EHAB: Wiping existing primary data before full refresh. [2025-10-20T18:29:55.303002] [INFO ] [LOCAL ] EHAB: Starting deletion of primary data. [2025-10-20T18:29:55.303183] [DEBUG] [LOCAL ] EHAB: Deleted primary data file. [2025-10-20T18:29:55.303239] [INFO ] [MAIN ] EHAB: Starting full primary refresh. [2025-10-20T18:29:55.455062] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-09. [2025-10-20T18:29:55.658804] [INFO ] [REMOTE ] EHAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:56.070901] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T18:29:56.251382] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-03. [2025-10-20T18:29:57.051388] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-03. [2025-10-20T18:29:57.274565] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:29:57.815749] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T18:29:57.851368] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-10. [2025-10-20T18:29:58.173695] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:29:58.651362] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-04. [2025-10-20T18:29:59.451408] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-04. [2025-10-20T18:29:59.731810] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T18:29:59.739541] [INFO ] [REMOTE ] EGO: Full fetch complete (74449 records). [2025-10-20T18:29:59.739695] [DEBUG] [VALIDATOR ] EGO: Validating primary data. [2025-10-20T18:30:00.130843] [DEBUG] [VALIDATOR ] EGO: All expected trading days are present. [2025-10-20T18:30:00.131022] [DEBUG] [VALIDATOR ] EGO: Primary data validation passed. [2025-10-20T18:30:00.131934] [INFO ] [MAIN ] EGO: Fetched primary data is valid. [2025-10-20T18:30:00.132014] [DEBUG] [LOCAL ] EGO: Saving primary data. [2025-10-20T18:30:00.133509] [DEBUG] [LOCAL ] EGO: Trimmed 74449 records to 51745 (keeping 200 days of data) [2025-10-20T18:30:00.252837] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-01. [2025-10-20T18:30:00.288261] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T18:30:00.494672] [DEBUG] [LOCAL ] EGO: Primary data saved (51745 records). [2025-10-20T18:30:00.494869] [INFO ] [MAIN ] EGO: Full history saved. [2025-10-20T18:30:00.495018] [INFO ] [MAIN ] Update Progress: 707/2367 symbols completed. EGO took 30.6s. Estimated remaining: 3h 31m 10s. Rate: 7.9 symbols/min [2025-10-20T18:30:00.745359] [INFO ] [MAIN ] EHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:00.793547] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:30:00.844672] [DEBUG] [LOCAL ] EHC: Loaded 41587 existing records from primary storage. [2025-10-20T18:30:00.845147] [INFO ] [MAIN ] EHC: Loaded 41587 existing records. [2025-10-20T18:30:01.065693] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-05. [2025-10-20T18:30:01.093596] [WARN ] [MAIN ] EHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:30:01.094957] [INFO ] [MAIN ] EHC: Wiping existing primary data before full refresh. [2025-10-20T18:30:01.095030] [INFO ] [LOCAL ] EHC: Starting deletion of primary data. [2025-10-20T18:30:01.095739] [DEBUG] [LOCAL ] EHC: Deleted primary data file. [2025-10-20T18:30:01.095843] [INFO ] [MAIN ] EHC: Starting full primary refresh. [2025-10-20T18:30:01.592115] [INFO ] [REMOTE ] EHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:01.615879] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T18:30:01.851434] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-05. [2025-10-20T18:30:02.373574] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T18:30:02.651443] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-01. [2025-10-20T18:30:03.423308] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T18:30:03.451343] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-06. [2025-10-20T18:30:04.251422] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-06. [2025-10-20T18:30:04.311334] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T18:30:05.040545] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T18:30:05.051333] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-07. [2025-10-20T18:30:05.401221] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T18:30:05.851394] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-02. [2025-10-20T18:30:06.615839] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T18:30:06.651350] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-08. [2025-10-20T18:30:07.459325] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-02. [2025-10-20T18:30:07.465950] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:30:08.214175] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T18:30:08.251364] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-09. [2025-10-20T18:30:08.294766] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T18:30:09.051432] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-07. [2025-10-20T18:30:09.837774] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:30:09.851340] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-10. [2025-10-20T18:30:10.510502] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:30:10.651417] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-08. [2025-10-20T18:30:11.451615] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-03. [2025-10-20T18:30:11.481501] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T18:30:11.489068] [INFO ] [REMOTE ] EGY: Full fetch complete (60896 records). [2025-10-20T18:30:11.489329] [DEBUG] [VALIDATOR ] EGY: Validating primary data. [2025-10-20T18:30:11.930903] [DEBUG] [VALIDATOR ] EGY: All expected trading days are present. [2025-10-20T18:30:11.931467] [DEBUG] [VALIDATOR ] EGY: Primary data validation passed. [2025-10-20T18:30:11.933120] [INFO ] [MAIN ] EGY: Fetched primary data is valid. [2025-10-20T18:30:11.935959] [DEBUG] [LOCAL ] EGY: Saving primary data. [2025-10-20T18:30:11.935842] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T18:30:11.944777] [DEBUG] [LOCAL ] EGY: Trimmed 60896 records to 41464 (keeping 200 days of data) [2025-10-20T18:30:12.179344] [DEBUG] [LOCAL ] EGY: Primary data saved (41464 records). [2025-10-20T18:30:12.179480] [INFO ] [MAIN ] EGY: Full history saved. [2025-10-20T18:30:12.179557] [INFO ] [MAIN ] Update Progress: 708/2367 symbols completed. EGY took 21.9s. Estimated remaining: 3h 31m 12s. Rate: 7.9 symbols/min [2025-10-20T18:30:12.256543] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-03. [2025-10-20T18:30:12.352937] [INFO ] [MAIN ] EHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:12.364564] [DEBUG] [LOCAL ] EHI: Loaded 10782 existing records from primary storage. [2025-10-20T18:30:12.364835] [INFO ] [MAIN ] EHI: Loaded 10782 existing records. [2025-10-20T18:30:12.551343] [WARN ] [MAIN ] EHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:30:12.551581] [INFO ] [MAIN ] EHI: Wiping existing primary data before full refresh. [2025-10-20T18:30:12.551629] [INFO ] [LOCAL ] EHI: Starting deletion of primary data. [2025-10-20T18:30:12.551888] [DEBUG] [LOCAL ] EHI: Deleted primary data file. [2025-10-20T18:30:12.551960] [INFO ] [MAIN ] EHI: Starting full primary refresh. [2025-10-20T18:30:12.929371] [INFO ] [REMOTE ] EHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:12.959720] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T18:30:13.051389] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-09. [2025-10-20T18:30:13.653615] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:30:13.851413] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-04. [2025-10-20T18:30:14.182088] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T18:30:14.651445] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-10. [2025-10-20T18:30:14.837769] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T18:30:15.451416] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-04. [2025-10-20T18:30:15.853296] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:30:15.857813] [INFO ] [REMOTE ] EGP: Full fetch complete (45389 records). [2025-10-20T18:30:15.857967] [DEBUG] [VALIDATOR ] EGP: Validating primary data. [2025-10-20T18:30:15.858667] [FATAL] [VALIDATOR ] EGP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:30:15.858774] [FATAL] [MAIN ] EGP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:30:15.858845] [WARN ] [MAIN ] EGP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:30:15.858942] [INFO ] [LOCAL ] EGP: Starting deletion of primary data. [2025-10-20T18:30:15.859024] [DEBUG] [LOCAL ] EGP: No primary data file found to delete. [2025-10-20T18:30:15.859062] [INFO ] [LOCAL ] EGP: Starting deletion of derivative data. [2025-10-20T18:30:15.859291] [DEBUG] [LOCAL ] EGP: Deleted derivative data file. [2025-10-20T18:30:15.859894] [INFO ] [MAIN ] Update Progress: 709/2367 symbols completed. EGP took 28.2s. Estimated remaining: 3h 30m 55s. Rate: 7.9 symbols/min [2025-10-20T18:30:16.036188] [INFO ] [MAIN ] EIC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:16.063691] [DEBUG] [LOCAL ] EIC: Loaded 23126 existing records from primary storage. [2025-10-20T18:30:16.064006] [INFO ] [MAIN ] EIC: Loaded 23126 existing records. [2025-10-20T18:30:16.258412] [WARN ] [MAIN ] EIC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:30:16.253896] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-05. [2025-10-20T18:30:16.258961] [INFO ] [MAIN ] EIC: Wiping existing primary data before full refresh. [2025-10-20T18:30:16.259056] [INFO ] [LOCAL ] EIC: Starting deletion of primary data. [2025-10-20T18:30:16.260366] [DEBUG] [LOCAL ] EIC: Deleted primary data file. [2025-10-20T18:30:16.260447] [INFO ] [MAIN ] EIC: Starting full primary refresh. [2025-10-20T18:30:16.637149] [INFO ] [REMOTE ] EIC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:17.051423] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-01. [2025-10-20T18:30:17.144865] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:30:17.579275] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:30:17.851461] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-01. [2025-10-20T18:30:18.651370] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-06. [2025-10-20T18:30:18.738120] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T18:30:18.902492] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T18:30:19.451354] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-02. [2025-10-20T18:30:19.557737] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T18:30:20.251383] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-07. [2025-10-20T18:30:20.320347] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T18:30:21.051414] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-05. [2025-10-20T18:30:21.744605] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:30:21.851386] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-02. [2025-10-20T18:30:22.225054] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:30:22.651404] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-08. [2025-10-20T18:30:22.999365] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T18:30:23.451388] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-03. [2025-10-20T18:30:24.025761] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T18:30:24.251394] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-09. [2025-10-20T18:30:24.363411] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T18:30:25.051361] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-06. [2025-10-20T18:30:25.343908] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T18:30:25.851343] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-04. [2025-10-20T18:30:26.589440] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T18:30:26.626171] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T18:30:26.651334] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-07. [2025-10-20T18:30:27.451367] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-10. [2025-10-20T18:30:28.257288] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-03. [2025-10-20T18:30:28.269762] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T18:30:28.680254] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T18:30:28.683951] [INFO ] [REMOTE ] EHAB: Full fetch complete (40774 records). [2025-10-20T18:30:28.684070] [DEBUG] [VALIDATOR ] EHAB: Validating primary data. [2025-10-20T18:30:28.684581] [FATAL] [VALIDATOR ] EHAB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:30:28.684705] [FATAL] [MAIN ] EHAB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:30:28.684756] [WARN ] [MAIN ] EHAB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:30:28.684820] [INFO ] [LOCAL ] EHAB: Starting deletion of primary data. [2025-10-20T18:30:28.684883] [DEBUG] [LOCAL ] EHAB: No primary data file found to delete. [2025-10-20T18:30:28.684917] [INFO ] [LOCAL ] EHAB: Starting deletion of derivative data. [2025-10-20T18:30:28.685043] [DEBUG] [LOCAL ] EHAB: Deleted derivative data file. [2025-10-20T18:30:28.685110] [INFO ] [MAIN ] Update Progress: 710/2367 symbols completed. EHAB took 33.8s. Estimated remaining: 3h 30m 59s. Rate: 7.9 symbols/min [2025-10-20T18:30:28.851773] [INFO ] [MAIN ] EICA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:28.851888] [DEBUG] [LOCAL ] EICA: No existing data file found in primary storage. [2025-10-20T18:30:28.851924] [WARN ] [MAIN ] EICA: No existing data found, performing full refresh instead. [2025-10-20T18:30:28.852030] [INFO ] [MAIN ] EICA: Wiping existing primary data before full refresh. [2025-10-20T18:30:28.852070] [INFO ] [LOCAL ] EICA: Starting deletion of primary data. [2025-10-20T18:30:28.852122] [DEBUG] [LOCAL ] EICA: No primary data file found to delete. [2025-10-20T18:30:28.852155] [INFO ] [MAIN ] EICA: Starting full primary refresh. [2025-10-20T18:30:29.056550] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-08. [2025-10-20T18:30:29.203877] [INFO ] [REMOTE ] EICA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:29.415337] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T18:30:29.851432] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-01. [2025-10-20T18:30:30.651365] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-05. [2025-10-20T18:30:31.451374] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-04. [2025-10-20T18:30:31.535995] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:30:31.578384] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:30:31.604930] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-08 (2.55 seconds). [2025-10-20T18:30:32.251371] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-02. [2025-10-20T18:30:32.518303] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:30:32.539395] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-10-20T18:30:33.051378] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-03. [2025-10-20T18:30:33.404608] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T18:30:33.851385] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-06. [2025-10-20T18:30:34.584078] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:30:34.651376] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-09. [2025-10-20T18:30:35.451456] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-05. [2025-10-20T18:30:36.048494] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T18:30:36.251410] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-10. [2025-10-20T18:30:36.535075] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T18:30:37.051448] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-04. [2025-10-20T18:30:37.404592] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T18:30:37.661986] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T18:30:37.668164] [INFO ] [REMOTE ] EHC: Full fetch complete (58779 records). [2025-10-20T18:30:37.668305] [DEBUG] [VALIDATOR ] EHC: Validating primary data. [2025-10-20T18:30:37.861455] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-05. [2025-10-20T18:30:38.056482] [DEBUG] [VALIDATOR ] EHC: All expected trading days are present. [2025-10-20T18:30:38.056568] [DEBUG] [VALIDATOR ] EHC: Primary data validation passed. [2025-10-20T18:30:38.057313] [INFO ] [MAIN ] EHC: Fetched primary data is valid. [2025-10-20T18:30:38.057378] [DEBUG] [LOCAL ] EHC: Saving primary data. [2025-10-20T18:30:38.058675] [DEBUG] [LOCAL ] EHC: Trimmed 58779 records to 41026 (keeping 200 days of data) [2025-10-20T18:30:38.317317] [DEBUG] [LOCAL ] EHC: Primary data saved (41026 records). [2025-10-20T18:30:38.317524] [INFO ] [MAIN ] EHC: Full history saved. [2025-10-20T18:30:38.318601] [INFO ] [MAIN ] Update Progress: 711/2367 symbols completed. EHC took 37.8s. Estimated remaining: 3h 30m 56s. Rate: 7.9 symbols/min [2025-10-20T18:30:38.396140] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T18:30:38.492782] [INFO ] [MAIN ] EICB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:38.492972] [DEBUG] [LOCAL ] EICB: No existing data file found in primary storage. [2025-10-20T18:30:38.493054] [WARN ] [MAIN ] EICB: No existing data found, performing full refresh instead. [2025-10-20T18:30:38.493194] [INFO ] [MAIN ] EICB: Wiping existing primary data before full refresh. [2025-10-20T18:30:38.493234] [INFO ] [LOCAL ] EICB: Starting deletion of primary data. [2025-10-20T18:30:38.493273] [DEBUG] [LOCAL ] EICB: No primary data file found to delete. [2025-10-20T18:30:38.493305] [INFO ] [MAIN ] EICB: Starting full primary refresh. [2025-10-20T18:30:38.657821] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-06. [2025-10-20T18:30:38.878896] [INFO ] [REMOTE ] EICB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:39.142865] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T18:30:39.451380] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-01. [2025-10-20T18:30:39.886717] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-10-20T18:30:40.251442] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-07. [2025-10-20T18:30:41.051420] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-07. [2025-10-20T18:30:41.069765] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T18:30:41.661269] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:30:41.851400] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-02. [2025-10-20T18:30:42.261903] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T18:30:42.651371] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-08. [2025-10-20T18:30:43.007407] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T18:30:43.451408] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-08. [2025-10-20T18:30:44.251369] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-06. [2025-10-20T18:30:44.335663] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T18:30:45.051404] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-09. [2025-10-20T18:30:45.403288] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-10-20T18:30:45.601695] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T18:30:45.851378] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-07. [2025-10-20T18:30:46.651419] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-03. [2025-10-20T18:30:47.083977] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-10-20T18:30:47.234458] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T18:30:47.451382] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-10. [2025-10-20T18:30:47.909616] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T18:30:47.910548] [INFO ] [REMOTE ] EICA: Full fetch complete (1256 records). [2025-10-20T18:30:47.910632] [DEBUG] [VALIDATOR ] EICA: Validating primary data. [2025-10-20T18:30:48.104139] [WARN ] [VALIDATOR ] EICA: Validation failed - insufficient historical data. Have 193 unique days, need 200 days [2025-10-20T18:30:48.104321] [FATAL] [MAIN ] EICA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:30:48.104390] [WARN ] [MAIN ] EICA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:30:48.104461] [INFO ] [LOCAL ] EICA: Starting deletion of primary data. [2025-10-20T18:30:48.104527] [DEBUG] [LOCAL ] EICA: No primary data file found to delete. [2025-10-20T18:30:48.104563] [INFO ] [LOCAL ] EICA: Starting deletion of derivative data. [2025-10-20T18:30:48.104602] [DEBUG] [LOCAL ] EICA: No derivative data file found to delete. [2025-10-20T18:30:48.104684] [INFO ] [MAIN ] Update Progress: 712/2367 symbols completed. EICA took 19.4s. Estimated remaining: 3h 30m 54s. Rate: 7.8 symbols/min [2025-10-20T18:30:48.255592] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-08. [2025-10-20T18:30:48.277130] [INFO ] [MAIN ] EICC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:48.278021] [DEBUG] [LOCAL ] EICC: No existing data file found in primary storage. [2025-10-20T18:30:48.278079] [WARN ] [MAIN ] EICC: No existing data found, performing full refresh instead. [2025-10-20T18:30:48.278207] [INFO ] [MAIN ] EICC: Wiping existing primary data before full refresh. [2025-10-20T18:30:48.278261] [INFO ] [LOCAL ] EICC: Starting deletion of primary data. [2025-10-20T18:30:48.278314] [DEBUG] [LOCAL ] EICC: No primary data file found to delete. [2025-10-20T18:30:48.278358] [INFO ] [MAIN ] EICC: Starting full primary refresh. [2025-10-20T18:30:48.653585] [INFO ] [REMOTE ] EICC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:49.051418] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-04. [2025-10-20T18:30:49.156246] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T18:30:49.412469] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-10-20T18:30:49.851361] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-01. [2025-10-20T18:30:50.453962] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T18:30:50.651376] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-09. [2025-10-20T18:30:51.444535] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:30:51.451324] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-10. [2025-10-20T18:30:52.251395] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-09. [2025-10-20T18:30:52.261142] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T18:30:52.263440] [INFO ] [REMOTE ] EHI: Full fetch complete (15828 records). [2025-10-20T18:30:52.263541] [DEBUG] [VALIDATOR ] EHI: Validating primary data. [2025-10-20T18:30:52.632048] [DEBUG] [VALIDATOR ] EHI: All expected trading days are present. [2025-10-20T18:30:52.632353] [DEBUG] [VALIDATOR ] EHI: Primary data validation passed. [2025-10-20T18:30:52.632755] [INFO ] [MAIN ] EHI: Fetched primary data is valid. [2025-10-20T18:30:52.632900] [DEBUG] [LOCAL ] EHI: Saving primary data. [2025-10-20T18:30:52.633700] [DEBUG] [LOCAL ] EHI: Trimmed 15828 records to 10543 (keeping 200 days of data) [2025-10-20T18:30:52.695593] [DEBUG] [LOCAL ] EHI: Primary data saved (10543 records). [2025-10-20T18:30:52.695744] [INFO ] [MAIN ] EHI: Full history saved. [2025-10-20T18:30:52.695833] [INFO ] [MAIN ] Update Progress: 713/2367 symbols completed. EHI took 40.5s. Estimated remaining: 3h 30m 39s. Rate: 7.9 symbols/min [2025-10-20T18:30:52.870605] [INFO ] [MAIN ] EIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:52.891390] [DEBUG] [LOCAL ] EIG: Loaded 20485 existing records from primary storage. [2025-10-20T18:30:52.891692] [INFO ] [MAIN ] EIG: Loaded 20485 existing records. [2025-10-20T18:30:53.053532] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-05. [2025-10-20T18:30:53.102158] [WARN ] [MAIN ] EIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:30:53.102438] [INFO ] [MAIN ] EIG: Wiping existing primary data before full refresh. [2025-10-20T18:30:53.102490] [INFO ] [LOCAL ] EIG: Starting deletion of primary data. [2025-10-20T18:30:53.102926] [DEBUG] [LOCAL ] EIG: Deleted primary data file. [2025-10-20T18:30:53.102983] [INFO ] [MAIN ] EIG: Starting full primary refresh. [2025-10-20T18:30:53.321457] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:30:53.585781] [INFO ] [REMOTE ] EIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:53.586071] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T18:30:53.851452] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-01. [2025-10-20T18:30:54.651597] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-10. [2025-10-20T18:30:54.947753] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T18:30:55.451392] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-02. [2025-10-20T18:30:55.797667] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T18:30:55.800817] [INFO ] [REMOTE ] EIC: Full fetch complete (34300 records). [2025-10-20T18:30:55.800895] [DEBUG] [VALIDATOR ] EIC: Validating primary data. [2025-10-20T18:30:55.985368] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:30:56.195012] [DEBUG] [VALIDATOR ] EIC: All expected trading days are present. [2025-10-20T18:30:56.195093] [DEBUG] [VALIDATOR ] EIC: Primary data validation passed. [2025-10-20T18:30:56.195706] [INFO ] [MAIN ] EIC: Fetched primary data is valid. [2025-10-20T18:30:56.195765] [DEBUG] [LOCAL ] EIC: Saving primary data. [2025-10-20T18:30:56.196491] [DEBUG] [LOCAL ] EIC: Trimmed 34300 records to 22699 (keeping 200 days of data) [2025-10-20T18:30:56.256328] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-03. [2025-10-20T18:30:56.336848] [DEBUG] [LOCAL ] EIC: Primary data saved (22699 records). [2025-10-20T18:30:56.337028] [INFO ] [MAIN ] EIC: Full history saved. [2025-10-20T18:30:56.337114] [INFO ] [MAIN ] Update Progress: 714/2367 symbols completed. EIC took 40.5s. Estimated remaining: 3h 30m 22s. Rate: 7.9 symbols/min [2025-10-20T18:30:56.502784] [INFO ] [MAIN ] EIX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:56.502948] [DEBUG] [LOCAL ] EIX: No existing data file found in primary storage. [2025-10-20T18:30:56.502983] [WARN ] [MAIN ] EIX: No existing data found, performing full refresh instead. [2025-10-20T18:30:56.503095] [INFO ] [MAIN ] EIX: Wiping existing primary data before full refresh. [2025-10-20T18:30:56.503134] [INFO ] [LOCAL ] EIX: Starting deletion of primary data. [2025-10-20T18:30:56.503173] [DEBUG] [LOCAL ] EIX: No primary data file found to delete. [2025-10-20T18:30:56.503203] [INFO ] [MAIN ] EIX: Starting full primary refresh. [2025-10-20T18:30:56.853899] [INFO ] [REMOTE ] EIX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:56.876215] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:30:57.051448] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-01. [2025-10-20T18:30:57.851379] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-04. [2025-10-20T18:30:58.519214] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T18:30:58.651451] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-05. [2025-10-20T18:30:58.991808] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-01 (1.94 seconds). [2025-10-20T18:30:59.302964] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T18:30:59.451403] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-06. [2025-10-20T18:31:00.061524] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T18:31:00.251436] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-02. [2025-10-20T18:31:01.051395] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-07. [2025-10-20T18:31:01.739598] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:31:01.849868] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:31:01.851993] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-06. [2025-10-20T18:31:02.651350] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-02. [2025-10-20T18:31:02.680965] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T18:31:03.451384] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-03. [2025-10-20T18:31:04.251358] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-08. [2025-10-20T18:31:04.398977] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:31:04.486046] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T18:31:04.644550] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-10-20T18:31:05.051431] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-09. [2025-10-20T18:31:05.803738] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T18:31:05.851384] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-03. [2025-10-20T18:31:06.651388] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-10. [2025-10-20T18:31:07.335182] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T18:31:07.336805] [INFO ] [REMOTE ] EICC: Full fetch complete (8439 records). [2025-10-20T18:31:07.336890] [DEBUG] [VALIDATOR ] EICC: Validating primary data. [2025-10-20T18:31:07.337297] [FATAL] [VALIDATOR ] EICC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:31:07.337379] [FATAL] [MAIN ] EICC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:31:07.337422] [WARN ] [MAIN ] EICC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:31:07.337459] [INFO ] [LOCAL ] EICC: Starting deletion of primary data. [2025-10-20T18:31:07.337513] [DEBUG] [LOCAL ] EICC: No primary data file found to delete. [2025-10-20T18:31:07.337546] [INFO ] [LOCAL ] EICC: Starting deletion of derivative data. [2025-10-20T18:31:07.337583] [DEBUG] [LOCAL ] EICC: No derivative data file found to delete. [2025-10-20T18:31:07.337634] [INFO ] [MAIN ] Update Progress: 715/2367 symbols completed. EICC took 19.2s. Estimated remaining: 3h 30m 22s. Rate: 7.9 symbols/min [2025-10-20T18:31:07.456612] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-04. [2025-10-20T18:31:07.522326] [INFO ] [MAIN ] EL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:07.586733] [DEBUG] [LOCAL ] EL: Loaded 54155 existing records from primary storage. [2025-10-20T18:31:07.587105] [INFO ] [MAIN ] EL: Loaded 54155 existing records. [2025-10-20T18:31:07.772214] [WARN ] [MAIN ] EL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:07.772566] [INFO ] [MAIN ] EL: Wiping existing primary data before full refresh. [2025-10-20T18:31:07.772728] [INFO ] [LOCAL ] EL: Starting deletion of primary data. [2025-10-20T18:31:07.772953] [DEBUG] [LOCAL ] EL: Deleted primary data file. [2025-10-20T18:31:07.773009] [INFO ] [MAIN ] EL: Starting full primary refresh. [2025-10-20T18:31:08.192572] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-03 (2.34 seconds). [2025-10-20T18:31:08.190943] [INFO ] [REMOTE ] EL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:08.251371] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-01. [2025-10-20T18:31:08.286243] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T18:31:09.051426] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-04. [2025-10-20T18:31:09.851371] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-07. [2025-10-20T18:31:10.014213] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T18:31:10.396089] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-10-20T18:31:10.651383] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-05. [2025-10-20T18:31:10.882780] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:31:11.451395] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-02. [2025-10-20T18:31:11.781476] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T18:31:12.251422] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-08. [2025-10-20T18:31:12.848432] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T18:31:13.051413] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-05. [2025-10-20T18:31:13.509313] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T18:31:13.851353] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-03. [2025-10-20T18:31:14.651346] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-06. [2025-10-20T18:31:14.733162] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:31:15.451349] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-06. [2025-10-20T18:31:15.712788] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:31:15.853236] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T18:31:16.251429] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-07. [2025-10-20T18:31:17.051439] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-09. [2025-10-20T18:31:17.332798] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T18:31:17.380526] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T18:31:17.490332] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T18:31:17.851367] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-07. [2025-10-20T18:31:18.651421] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-08. [2025-10-20T18:31:19.451407] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-10. [2025-10-20T18:31:19.893320] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:31:19.926056] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:31:19.977298] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T18:31:19.978229] [INFO ] [REMOTE ] EICB: Full fetch complete (2692 records). [2025-10-20T18:31:19.978297] [DEBUG] [VALIDATOR ] EICB: Validating primary data. [2025-10-20T18:31:19.978637] [FATAL] [VALIDATOR ] EICB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:31:19.978740] [FATAL] [MAIN ] EICB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:31:19.978783] [WARN ] [MAIN ] EICB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:31:19.978825] [INFO ] [LOCAL ] EICB: Starting deletion of primary data. [2025-10-20T18:31:19.978881] [DEBUG] [LOCAL ] EICB: No primary data file found to delete. [2025-10-20T18:31:19.978915] [INFO ] [LOCAL ] EICB: Starting deletion of derivative data. [2025-10-20T18:31:19.978954] [DEBUG] [LOCAL ] EICB: No derivative data file found to delete. [2025-10-20T18:31:19.979004] [INFO ] [MAIN ] Update Progress: 716/2367 symbols completed. EICB took 41.7s. Estimated remaining: 3h 30m 26s. Rate: 7.8 symbols/min [2025-10-20T18:31:20.147960] [INFO ] [MAIN ] ELAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:20.200697] [DEBUG] [LOCAL ] ELAN: Loaded 54080 existing records from primary storage. [2025-10-20T18:31:20.201033] [INFO ] [MAIN ] ELAN: Loaded 54080 existing records. [2025-10-20T18:31:20.259421] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-08. [2025-10-20T18:31:20.386512] [WARN ] [MAIN ] ELAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:20.386743] [INFO ] [MAIN ] ELAN: Wiping existing primary data before full refresh. [2025-10-20T18:31:20.386807] [INFO ] [LOCAL ] ELAN: Starting deletion of primary data. [2025-10-20T18:31:20.387010] [DEBUG] [LOCAL ] ELAN: Deleted primary data file. [2025-10-20T18:31:20.387047] [INFO ] [MAIN ] ELAN: Starting full primary refresh. [2025-10-20T18:31:20.734914] [INFO ] [REMOTE ] ELAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:21.051373] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-09. [2025-10-20T18:31:21.851371] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-01. [2025-10-20T18:31:22.079608] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T18:31:22.355937] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-08 (2.10 seconds). [2025-10-20T18:31:22.651374] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-10. [2025-10-20T18:31:23.121875] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T18:31:23.451387] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-04. [2025-10-20T18:31:23.716386] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T18:31:23.718964] [INFO ] [REMOTE ] EIG: Full fetch complete (27136 records). [2025-10-20T18:31:23.719034] [DEBUG] [VALIDATOR ] EIG: Validating primary data. [2025-10-20T18:31:23.719609] [FATAL] [VALIDATOR ] EIG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:31:23.719711] [FATAL] [MAIN ] EIG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:31:23.719754] [WARN ] [MAIN ] EIG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:31:23.719790] [INFO ] [LOCAL ] EIG: Starting deletion of primary data. [2025-10-20T18:31:23.719843] [DEBUG] [LOCAL ] EIG: No primary data file found to delete. [2025-10-20T18:31:23.719876] [INFO ] [LOCAL ] EIG: Starting deletion of derivative data. [2025-10-20T18:31:23.720029] [DEBUG] [LOCAL ] EIG: Deleted derivative data file. [2025-10-20T18:31:23.720111] [INFO ] [MAIN ] Update Progress: 717/2367 symbols completed. EIG took 31.0s. Estimated remaining: 3h 30m 9s. Rate: 7.9 symbols/min [2025-10-20T18:31:24.251396] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-09. [2025-10-20T18:31:25.051445] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-02. [2025-10-20T18:31:25.539937] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-04 (2.09 seconds). [2025-10-20T18:31:25.757236] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:31:25.851353] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-10. [2025-10-20T18:31:26.651397] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-05. [2025-10-20T18:31:26.678364] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T18:31:26.685752] [INFO ] [REMOTE ] EIX: Full fetch complete (77063 records). [2025-10-20T18:31:26.685879] [DEBUG] [VALIDATOR ] EIX: Validating primary data. [2025-10-20T18:31:26.686572] [FATAL] [VALIDATOR ] EIX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:31:26.686682] [FATAL] [MAIN ] EIX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:31:26.686734] [WARN ] [MAIN ] EIX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:31:26.686793] [INFO ] [LOCAL ] EIX: Starting deletion of primary data. [2025-10-20T18:31:26.686849] [DEBUG] [LOCAL ] EIX: No primary data file found to delete. [2025-10-20T18:31:26.686883] [INFO ] [LOCAL ] EIX: Starting deletion of derivative data. [2025-10-20T18:31:26.686920] [DEBUG] [LOCAL ] EIX: No derivative data file found to delete. [2025-10-20T18:31:26.686974] [INFO ] [MAIN ] Update Progress: 718/2367 symbols completed. EIX took 30.3s. Estimated remaining: 3h 29m 51s. Rate: 7.9 symbols/min [2025-10-20T18:31:26.906486] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T18:31:27.451371] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-03. [2025-10-20T18:31:28.325766] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:31:28.337183] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-06. [2025-10-20T18:31:28.795150] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:31:29.051381] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-04. [2025-10-20T18:31:30.465207] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-10-20T18:31:30.477190] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-07. [2025-10-20T18:31:30.822801] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:31:30.836329] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-05. [2025-10-20T18:31:32.968976] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T18:31:32.981220] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-06. [2025-10-20T18:31:33.499688] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-07 (3.02 seconds). [2025-10-20T18:31:33.512347] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-08. [2025-10-20T18:31:35.201158] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-06 (2.22 seconds). [2025-10-20T18:31:35.211961] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-07. [2025-10-20T18:31:36.183424] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-08 (2.67 seconds). [2025-10-20T18:31:36.195469] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-09. [2025-10-20T18:31:36.700941] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:31:36.712622] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-08. [2025-10-20T18:31:37.711161] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T18:31:37.722215] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-10. [2025-10-20T18:31:38.266464] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T18:31:38.278498] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-09. [2025-10-20T18:31:38.398324] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T18:31:38.406363] [INFO ] [REMOTE ] EL: Full fetch complete (77440 records). [2025-10-20T18:31:38.406512] [DEBUG] [VALIDATOR ] EL: Validating primary data. [2025-10-20T18:31:38.864473] [DEBUG] [VALIDATOR ] EL: All expected trading days are present. [2025-10-20T18:31:38.864550] [DEBUG] [VALIDATOR ] EL: Primary data validation passed. [2025-10-20T18:31:38.865421] [INFO ] [MAIN ] EL: Fetched primary data is valid. [2025-10-20T18:31:38.865484] [DEBUG] [LOCAL ] EL: Saving primary data. [2025-10-20T18:31:38.866906] [DEBUG] [LOCAL ] EL: Trimmed 77440 records to 53387 (keeping 200 days of data) [2025-10-20T18:31:39.191699] [DEBUG] [LOCAL ] EL: Primary data saved (53387 records). [2025-10-20T18:31:39.191849] [INFO ] [MAIN ] EL: Full history saved. [2025-10-20T18:31:39.191930] [INFO ] [MAIN ] Update Progress: 719/2367 symbols completed. EL took 31.9s. Estimated remaining: 3h 29m 54s. Rate: 7.9 symbols/min [2025-10-20T18:31:39.905363] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:31:39.918183] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-10. [2025-10-20T18:31:41.490422] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T18:31:41.498373] [INFO ] [REMOTE ] ELAN: Full fetch complete (77093 records). [2025-10-20T18:31:41.498529] [DEBUG] [VALIDATOR ] ELAN: Validating primary data. [2025-10-20T18:31:41.883912] [DEBUG] [VALIDATOR ] ELAN: All expected trading days are present. [2025-10-20T18:31:41.883998] [DEBUG] [VALIDATOR ] ELAN: Primary data validation passed. [2025-10-20T18:31:41.884874] [INFO ] [MAIN ] ELAN: Fetched primary data is valid. [2025-10-20T18:31:41.884937] [DEBUG] [LOCAL ] ELAN: Saving primary data. [2025-10-20T18:31:41.886349] [DEBUG] [LOCAL ] ELAN: Trimmed 77093 records to 53315 (keeping 200 days of data) [2025-10-20T18:31:42.189693] [DEBUG] [LOCAL ] ELAN: Primary data saved (53315 records). [2025-10-20T18:31:42.189828] [INFO ] [MAIN ] ELAN: Full history saved. [2025-10-20T18:31:42.189903] [INFO ] [MAIN ] Update Progress: 720/2367 symbols completed. ELAN took 22.2s. Estimated remaining: 3h 29m 36s. Rate: 7.9 symbols/min [2025-10-20T18:31:42.791239] [INFO ] [MAIN ] ELME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:42.916572] [INFO ] [MAIN ] ELP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:42.953427] [INFO ] [MAIN ] ELF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:42.956488] [INFO ] [MAIN ] ELC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:42.965264] [DEBUG] [LOCAL ] ELC: Loaded 3542 existing records from primary storage. [2025-10-20T18:31:42.965728] [INFO ] [MAIN ] ELC: Loaded 3542 existing records. [2025-10-20T18:31:43.124974] [DEBUG] [LOCAL ] ELP: Loaded 26040 existing records from primary storage. [2025-10-20T18:31:43.169951] [DEBUG] [LOCAL ] ELME: Loaded 39953 existing records from primary storage. [2025-10-20T18:31:43.166891] [INFO ] [MAIN ] ELP: Loaded 26040 existing records. [2025-10-20T18:31:43.198489] [WARN ] [MAIN ] ELC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:43.219964] [INFO ] [MAIN ] ELME: Loaded 39953 existing records. [2025-10-20T18:31:43.220422] [INFO ] [MAIN ] ELC: Wiping existing primary data before full refresh. [2025-10-20T18:31:43.273796] [INFO ] [LOCAL ] ELC: Starting deletion of primary data. [2025-10-20T18:31:43.347152] [DEBUG] [LOCAL ] ELC: Deleted primary data file. [2025-10-20T18:31:43.404553] [INFO ] [MAIN ] ELC: Starting full primary refresh. [2025-10-20T18:31:43.438371] [DEBUG] [LOCAL ] ELF: Loaded 52711 existing records from primary storage. [2025-10-20T18:31:43.583187] [INFO ] [MAIN ] ELF: Loaded 52711 existing records. [2025-10-20T18:31:43.619976] [WARN ] [MAIN ] ELP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:43.696846] [INFO ] [MAIN ] ELP: Wiping existing primary data before full refresh. [2025-10-20T18:31:43.742131] [INFO ] [LOCAL ] ELP: Starting deletion of primary data. [2025-10-20T18:31:43.728565] [WARN ] [MAIN ] ELME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:43.778538] [DEBUG] [LOCAL ] ELP: Deleted primary data file. [2025-10-20T18:31:43.799598] [INFO ] [MAIN ] ELME: Wiping existing primary data before full refresh. [2025-10-20T18:31:43.810024] [INFO ] [MAIN ] ELP: Starting full primary refresh. [2025-10-20T18:31:43.815350] [INFO ] [LOCAL ] ELME: Starting deletion of primary data. [2025-10-20T18:31:43.996733] [DEBUG] [LOCAL ] ELME: Deleted primary data file. [2025-10-20T18:31:44.021227] [INFO ] [MAIN ] ELME: Starting full primary refresh. [2025-10-20T18:31:44.265913] [WARN ] [MAIN ] ELF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:44.376808] [INFO ] [MAIN ] ELF: Wiping existing primary data before full refresh. [2025-10-20T18:31:44.402782] [INFO ] [LOCAL ] ELF: Starting deletion of primary data. [2025-10-20T18:31:44.493615] [DEBUG] [LOCAL ] ELF: Deleted primary data file. [2025-10-20T18:31:44.579897] [INFO ] [MAIN ] ELF: Starting full primary refresh. [2025-10-20T18:31:44.609284] [INFO ] [REMOTE ] ELC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:44.622344] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-01. [2025-10-20T18:31:44.982530] [INFO ] [REMOTE ] ELP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:45.048332] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-01. [2025-10-20T18:31:45.267347] [INFO ] [REMOTE ] ELME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:45.283605] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-01. [2025-10-20T18:31:45.349301] [INFO ] [REMOTE ] ELF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:45.349606] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-01. [2025-10-20T18:31:45.487153] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T18:31:45.488826] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-02. [2025-10-20T18:31:45.812161] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T18:31:45.813467] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-03. [2025-10-20T18:31:46.365872] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T18:31:46.372806] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-02. [2025-10-20T18:31:46.378372] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T18:31:46.380124] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-04. [2025-10-20T18:31:46.738023] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:31:46.748923] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-02. [2025-10-20T18:31:47.034710] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T18:31:47.036819] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-05. [2025-10-20T18:31:47.121923] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T18:31:47.134345] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-02. [2025-10-20T18:31:47.310286] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:31:47.315627] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-03. [2025-10-20T18:31:47.622076] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:31:47.624005] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-06. [2025-10-20T18:31:47.691217] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:31:48.004023] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-10-20T18:31:48.194382] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T18:31:48.251366] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-07. [2025-10-20T18:31:48.583179] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-10-20T18:31:49.051338] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-04. [2025-10-20T18:31:49.260576] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-10-20T18:31:49.851373] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-03. [2025-10-20T18:31:50.651429] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-03. [2025-10-20T18:31:51.097821] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T18:31:51.451379] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-05. [2025-10-20T18:31:52.251383] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-08. [2025-10-20T18:31:52.601880] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T18:31:52.781096] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-03 (2.13 seconds). [2025-10-20T18:31:52.791607] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T18:31:53.051410] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-06. [2025-10-20T18:31:53.197477] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-03 (3.35 seconds). [2025-10-20T18:31:53.851490] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-04. [2025-10-20T18:31:54.082452] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T18:31:54.651488] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-04. [2025-10-20T18:31:55.451410] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-09. [2025-10-20T18:31:55.847122] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T18:31:56.114564] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-10-20T18:31:56.191447] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T18:31:56.251399] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-05. [2025-10-20T18:31:57.051419] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-10. [2025-10-20T18:31:57.636610] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T18:31:57.638310] [INFO ] [REMOTE ] ELC: Full fetch complete (4927 records). [2025-10-20T18:31:57.638386] [DEBUG] [VALIDATOR ] ELC: Validating primary data. [2025-10-20T18:31:57.859786] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-05. [2025-10-20T18:31:58.040517] [DEBUG] [VALIDATOR ] ELC: All expected trading days are present. [2025-10-20T18:31:58.040628] [DEBUG] [VALIDATOR ] ELC: Primary data validation passed. [2025-10-20T18:31:58.040876] [INFO ] [MAIN ] ELC: Fetched primary data is valid. [2025-10-20T18:31:58.041048] [DEBUG] [LOCAL ] ELC: Saving primary data. [2025-10-20T18:31:58.041636] [DEBUG] [LOCAL ] ELC: Trimmed 4927 records to 3476 (keeping 200 days of data) [2025-10-20T18:31:58.066354] [DEBUG] [LOCAL ] ELC: Primary data saved (3476 records). [2025-10-20T18:31:58.066555] [INFO ] [MAIN ] ELC: Full history saved. [2025-10-20T18:31:58.066683] [INFO ] [MAIN ] Update Progress: 721/2367 symbols completed. ELC took 15.9s. Estimated remaining: 3h 29m 47s. Rate: 7.8 symbols/min [2025-10-20T18:31:58.253407] [INFO ] [MAIN ] ELPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:58.253778] [DEBUG] [LOCAL ] ELPC: No existing data file found in primary storage. [2025-10-20T18:31:58.253847] [WARN ] [MAIN ] ELPC: No existing data found, performing full refresh instead. [2025-10-20T18:31:58.253956] [INFO ] [MAIN ] ELPC: Wiping existing primary data before full refresh. [2025-10-20T18:31:58.254095] [INFO ] [LOCAL ] ELPC: Starting deletion of primary data. [2025-10-20T18:31:58.254201] [DEBUG] [LOCAL ] ELPC: No primary data file found to delete. [2025-10-20T18:31:58.254247] [INFO ] [MAIN ] ELPC: Starting full primary refresh. [2025-10-20T18:31:58.465431] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-05 (2.21 seconds). [2025-10-20T18:31:58.656703] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-07. [2025-10-20T18:31:58.708148] [INFO ] [REMOTE ] ELPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:59.178079] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T18:31:59.451431] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-01. [2025-10-20T18:31:59.852634] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T18:31:59.982395] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-10-20T18:32:00.251417] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-06. [2025-10-20T18:32:01.051431] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-08. [2025-10-20T18:32:01.851403] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-02. [2025-10-20T18:32:02.186301] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T18:32:02.280929] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T18:32:02.397833] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T18:32:02.651450] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-06. [2025-10-20T18:32:03.451455] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-09. [2025-10-20T18:32:04.251368] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-07. [2025-10-20T18:32:04.310596] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T18:32:04.647424] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:32:05.051414] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-10. [2025-10-20T18:32:05.851401] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-07. [2025-10-20T18:32:06.225992] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T18:32:06.249448] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:32:06.253378] [INFO ] [REMOTE ] ELP: Full fetch complete (36762 records). [2025-10-20T18:32:06.253472] [DEBUG] [VALIDATOR ] ELP: Validating primary data. [2025-10-20T18:32:06.254235] [FATAL] [VALIDATOR ] ELP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:32:06.254325] [FATAL] [MAIN ] ELP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:32:06.254370] [WARN ] [MAIN ] ELP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:32:06.254411] [INFO ] [LOCAL ] ELP: Starting deletion of primary data. [2025-10-20T18:32:06.254476] [DEBUG] [LOCAL ] ELP: No primary data file found to delete. [2025-10-20T18:32:06.254511] [INFO ] [LOCAL ] ELP: Starting deletion of derivative data. [2025-10-20T18:32:06.254698] [DEBUG] [LOCAL ] ELP: Deleted derivative data file. [2025-10-20T18:32:06.254798] [INFO ] [MAIN ] Update Progress: 722/2367 symbols completed. ELP took 24.1s. Estimated remaining: 3h 29m 41s. Rate: 7.8 symbols/min [2025-10-20T18:32:06.429748] [INFO ] [MAIN ] ELS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:06.487612] [DEBUG] [LOCAL ] ELS: Loaded 50599 existing records from primary storage. [2025-10-20T18:32:06.488291] [INFO ] [MAIN ] ELS: Loaded 50599 existing records. [2025-10-20T18:32:06.656542] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-08. [2025-10-20T18:32:06.692350] [WARN ] [MAIN ] ELS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:06.692971] [INFO ] [MAIN ] ELS: Wiping existing primary data before full refresh. [2025-10-20T18:32:06.693041] [INFO ] [LOCAL ] ELS: Starting deletion of primary data. [2025-10-20T18:32:06.693863] [DEBUG] [LOCAL ] ELS: Deleted primary data file. [2025-10-20T18:32:06.693935] [INFO ] [MAIN ] ELS: Starting full primary refresh. [2025-10-20T18:32:07.098543] [INFO ] [REMOTE ] ELS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:07.398009] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T18:32:07.451395] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-08. [2025-10-20T18:32:08.251402] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-03. [2025-10-20T18:32:08.545371] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-10-20T18:32:08.593704] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T18:32:08.653873] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T18:32:09.051411] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-09. [2025-10-20T18:32:09.851416] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-01. [2025-10-20T18:32:10.651444] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-09. [2025-10-20T18:32:10.734403] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:32:11.413096] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T18:32:11.451382] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-04. [2025-10-20T18:32:11.996478] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T18:32:12.251348] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-10. [2025-10-20T18:32:12.504683] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T18:32:13.051364] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-10. [2025-10-20T18:32:13.772331] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T18:32:13.778264] [INFO ] [REMOTE ] ELME: Full fetch complete (55971 records). [2025-10-20T18:32:13.778361] [DEBUG] [VALIDATOR ] ELME: Validating primary data. [2025-10-20T18:32:13.856699] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-05. [2025-10-20T18:32:14.156710] [DEBUG] [VALIDATOR ] ELME: All expected trading days are present. [2025-10-20T18:32:14.156816] [DEBUG] [VALIDATOR ] ELME: Primary data validation passed. [2025-10-20T18:32:14.157525] [INFO ] [MAIN ] ELME: Fetched primary data is valid. [2025-10-20T18:32:14.157572] [DEBUG] [LOCAL ] ELME: Saving primary data. [2025-10-20T18:32:14.158790] [DEBUG] [LOCAL ] ELME: Trimmed 55971 records to 39361 (keeping 200 days of data) [2025-10-20T18:32:14.254843] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-10-20T18:32:14.402715] [DEBUG] [LOCAL ] ELME: Primary data saved (39361 records). [2025-10-20T18:32:14.402860] [INFO ] [MAIN ] ELME: Full history saved. [2025-10-20T18:32:14.402940] [INFO ] [MAIN ] Update Progress: 723/2367 symbols completed. ELME took 32.2s. Estimated remaining: 3h 29m 34s. Rate: 7.8 symbols/min [2025-10-20T18:32:14.576636] [INFO ] [MAIN ] ELV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:14.626745] [DEBUG] [LOCAL ] ELV: Loaded 51393 existing records from primary storage. [2025-10-20T18:32:14.627221] [INFO ] [MAIN ] ELV: Loaded 51393 existing records. [2025-10-20T18:32:14.657825] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-06. [2025-10-20T18:32:14.714440] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T18:32:14.766185] [INFO ] [REMOTE ] ELF: Full fetch complete (75113 records). [2025-10-20T18:32:14.776669] [DEBUG] [VALIDATOR ] ELF: Validating primary data. [2025-10-20T18:32:14.800856] [FATAL] [VALIDATOR ] ELF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:32:14.836441] [FATAL] [MAIN ] ELF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:32:14.841194] [WARN ] [MAIN ] ELV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:14.841327] [WARN ] [MAIN ] ELF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:32:14.841590] [INFO ] [MAIN ] ELV: Wiping existing primary data before full refresh. [2025-10-20T18:32:14.841724] [INFO ] [LOCAL ] ELF: Starting deletion of primary data. [2025-10-20T18:32:14.841831] [DEBUG] [LOCAL ] ELF: No primary data file found to delete. [2025-10-20T18:32:14.841927] [INFO ] [LOCAL ] ELV: Starting deletion of primary data. [2025-10-20T18:32:14.842028] [INFO ] [LOCAL ] ELF: Starting deletion of derivative data. [2025-10-20T18:32:14.842341] [DEBUG] [LOCAL ] ELV: Deleted primary data file. [2025-10-20T18:32:14.842318] [DEBUG] [LOCAL ] ELF: Deleted derivative data file. [2025-10-20T18:32:14.842895] [INFO ] [MAIN ] Update Progress: 724/2367 symbols completed. ELF took 32.6s. Estimated remaining: 3h 29m 10s. Rate: 7.9 symbols/min [2025-10-20T18:32:14.843001] [INFO ] [MAIN ] ELV: Starting full primary refresh. [2025-10-20T18:32:15.064512] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-10-20T18:32:15.204582] [INFO ] [MAIN ] EMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:15.240184] [DEBUG] [LOCAL ] EMA: No existing data file found in primary storage. [2025-10-20T18:32:15.272372] [WARN ] [MAIN ] EMA: No existing data found, performing full refresh instead. [2025-10-20T18:32:15.284057] [INFO ] [MAIN ] EMA: Wiping existing primary data before full refresh. [2025-10-20T18:32:15.289265] [INFO ] [LOCAL ] EMA: Starting deletion of primary data. [2025-10-20T18:32:15.300046] [DEBUG] [LOCAL ] EMA: No primary data file found to delete. [2025-10-20T18:32:15.310692] [INFO ] [MAIN ] EMA: Starting full primary refresh. [2025-10-20T18:32:15.456822] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-07. [2025-10-20T18:32:15.536850] [INFO ] [REMOTE ] ELV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:15.754722] [INFO ] [REMOTE ] EMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:15.966285] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T18:32:16.251371] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-01. [2025-10-20T18:32:16.461264] [DEBUG] [REMOTE ] EMA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:32:16.461601] [WARN ] [REMOTE ] EMA: No data returned for month 2025-01 [2025-10-20T18:32:16.461711] [FATAL] [REMOTE ] EMA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:32:16.461762] [FATAL] [MAIN ] EMA: Failed to fetch primary data. [2025-10-20T18:32:16.461823] [WARN ] [MAIN ] EMA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:32:16.461895] [INFO ] [LOCAL ] EMA: Starting deletion of primary data. [2025-10-20T18:32:16.461958] [DEBUG] [LOCAL ] EMA: No primary data file found to delete. [2025-10-20T18:32:16.461993] [INFO ] [LOCAL ] EMA: Starting deletion of derivative data. [2025-10-20T18:32:16.462032] [DEBUG] [LOCAL ] EMA: No derivative data file found to delete. [2025-10-20T18:32:16.462075] [INFO ] [MAIN ] Update Progress: 725/2367 symbols completed. EMA took 1.6s. Estimated remaining: 3h 28m 49s. Rate: 7.9 symbols/min [2025-10-20T18:32:16.633189] [INFO ] [MAIN ] EMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:16.654716] [DEBUG] [LOCAL ] EMD: Loaded 21209 existing records from primary storage. [2025-10-20T18:32:16.654967] [INFO ] [MAIN ] EMD: Loaded 21209 existing records. [2025-10-20T18:32:16.830199] [WARN ] [MAIN ] EMD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:16.830481] [INFO ] [MAIN ] EMD: Wiping existing primary data before full refresh. [2025-10-20T18:32:16.830528] [INFO ] [LOCAL ] EMD: Starting deletion of primary data. [2025-10-20T18:32:16.830783] [DEBUG] [LOCAL ] EMD: Deleted primary data file. [2025-10-20T18:32:16.830839] [INFO ] [MAIN ] EMD: Starting full primary refresh. [2025-10-20T18:32:17.051455] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-01. [2025-10-20T18:32:17.238613] [INFO ] [REMOTE ] EMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:17.851384] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-01. [2025-10-20T18:32:18.651422] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-08. [2025-10-20T18:32:18.777190] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T18:32:18.986612] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:32:19.035477] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T18:32:19.451377] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-02. [2025-10-20T18:32:20.251404] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-02. [2025-10-20T18:32:20.391890] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:32:21.051425] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-03. [2025-10-20T18:32:21.851353] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-02. [2025-10-20T18:32:21.884725] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:32:22.192868] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:32:22.651366] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-04. [2025-10-20T18:32:23.451358] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-09. [2025-10-20T18:32:23.530625] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T18:32:24.082584] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T18:32:24.111925] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:32:24.251383] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-05. [2025-10-20T18:32:25.051379] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-03. [2025-10-20T18:32:25.410846] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T18:32:25.851410] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-06. [2025-10-20T18:32:26.651373] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-10. [2025-10-20T18:32:26.801600] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:32:26.865823] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T18:32:27.010101] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T18:32:27.011280] [INFO ] [REMOTE ] ELPC: Full fetch complete (1395 records). [2025-10-20T18:32:27.011390] [DEBUG] [VALIDATOR ] ELPC: Validating primary data. [2025-10-20T18:32:27.011968] [FATAL] [VALIDATOR ] ELPC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:32:27.012076] [FATAL] [MAIN ] ELPC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:32:27.012139] [WARN ] [MAIN ] ELPC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:32:27.012208] [INFO ] [LOCAL ] ELPC: Starting deletion of primary data. [2025-10-20T18:32:27.012337] [DEBUG] [LOCAL ] ELPC: No primary data file found to delete. [2025-10-20T18:32:27.012409] [INFO ] [LOCAL ] ELPC: Starting deletion of derivative data. [2025-10-20T18:32:27.012504] [DEBUG] [LOCAL ] ELPC: No derivative data file found to delete. [2025-10-20T18:32:27.012770] [INFO ] [MAIN ] Update Progress: 726/2367 symbols completed. ELPC took 28.9s. Estimated remaining: 3h 28m 48s. Rate: 7.9 symbols/min [2025-10-20T18:32:27.209614] [INFO ] [MAIN ] EME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:27.209882] [DEBUG] [LOCAL ] EME: No existing data file found in primary storage. [2025-10-20T18:32:27.209926] [WARN ] [MAIN ] EME: No existing data found, performing full refresh instead. [2025-10-20T18:32:27.210058] [INFO ] [MAIN ] EME: Wiping existing primary data before full refresh. [2025-10-20T18:32:27.210101] [INFO ] [LOCAL ] EME: Starting deletion of primary data. [2025-10-20T18:32:27.210144] [DEBUG] [LOCAL ] EME: No primary data file found to delete. [2025-10-20T18:32:27.210207] [INFO ] [MAIN ] EME: Starting full primary refresh. [2025-10-20T18:32:27.456635] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-07. [2025-10-20T18:32:27.652986] [INFO ] [REMOTE ] EME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:28.251363] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-01. [2025-10-20T18:32:28.610225] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T18:32:29.051435] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-04. [2025-10-20T18:32:29.448566] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T18:32:29.851423] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-03. [2025-10-20T18:32:30.559996] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:32:30.651420] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-02. [2025-10-20T18:32:31.347371] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:32:31.451336] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-04. [2025-10-20T18:32:32.159799] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T18:32:32.251418] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-08. [2025-10-20T18:32:33.051370] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-05. [2025-10-20T18:32:33.217858] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T18:32:33.268621] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T18:32:33.851417] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-03. [2025-10-20T18:32:34.655307] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-09. [2025-10-20T18:32:34.671441] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:32:35.460632] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-06. [2025-10-20T18:32:35.465488] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:32:35.821901] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:32:36.251427] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-05. [2025-10-20T18:32:36.980371] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T18:32:37.051359] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-07. [2025-10-20T18:32:37.851481] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-10. [2025-10-20T18:32:38.238685] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T18:32:38.651449] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-04. [2025-10-20T18:32:38.751195] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T18:32:39.055473] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:32:39.059697] [INFO ] [REMOTE ] EMD: Full fetch complete (31975 records). [2025-10-20T18:32:39.059815] [DEBUG] [VALIDATOR ] EMD: Validating primary data. [2025-10-20T18:32:39.451798] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-06. [2025-10-20T18:32:39.452338] [DEBUG] [VALIDATOR ] EMD: All expected trading days are present. [2025-10-20T18:32:39.453474] [DEBUG] [VALIDATOR ] EMD: Primary data validation passed. [2025-10-20T18:32:39.454541] [INFO ] [MAIN ] EMD: Fetched primary data is valid. [2025-10-20T18:32:39.454753] [DEBUG] [LOCAL ] EMD: Saving primary data. [2025-10-20T18:32:39.455944] [DEBUG] [LOCAL ] EMD: Trimmed 31975 records to 20926 (keeping 200 days of data) [2025-10-20T18:32:39.589335] [DEBUG] [LOCAL ] EMD: Primary data saved (20926 records). [2025-10-20T18:32:39.589528] [INFO ] [MAIN ] EMD: Full history saved. [2025-10-20T18:32:39.589682] [INFO ] [MAIN ] Update Progress: 727/2367 symbols completed. EMD took 23.1s. Estimated remaining: 3h 28m 52s. Rate: 7.9 symbols/min [2025-10-20T18:32:39.781557] [INFO ] [MAIN ] EMF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:39.788866] [DEBUG] [LOCAL ] EMF: Loaded 3246 existing records from primary storage. [2025-10-20T18:32:39.789181] [INFO ] [MAIN ] EMF: Loaded 3246 existing records. [2025-10-20T18:32:39.982008] [WARN ] [MAIN ] EMF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:39.982381] [INFO ] [MAIN ] EMF: Wiping existing primary data before full refresh. [2025-10-20T18:32:39.982439] [INFO ] [LOCAL ] EMF: Starting deletion of primary data. [2025-10-20T18:32:39.982601] [DEBUG] [LOCAL ] EMF: Deleted primary data file. [2025-10-20T18:32:39.982693] [INFO ] [MAIN ] EMF: Starting full primary refresh. [2025-10-20T18:32:40.256499] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-08. [2025-10-20T18:32:40.400233] [INFO ] [REMOTE ] EMF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:40.423174] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:32:40.996515] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T18:32:41.051408] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-07. [2025-10-20T18:32:41.851432] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-05. [2025-10-20T18:32:42.019034] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:32:42.651364] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-09. [2025-10-20T18:32:42.980808] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T18:32:43.288900] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:32:43.451399] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-08. [2025-10-20T18:32:44.204991] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T18:32:44.251345] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-10. [2025-10-20T18:32:45.051359] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-06. [2025-10-20T18:32:45.391382] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T18:32:45.851388] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-01. [2025-10-20T18:32:46.161237] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:32:46.168858] [INFO ] [REMOTE ] ELS: Full fetch complete (71263 records). [2025-10-20T18:32:46.169005] [DEBUG] [VALIDATOR ] ELS: Validating primary data. [2025-10-20T18:32:46.483544] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T18:32:46.574762] [DEBUG] [VALIDATOR ] ELS: All expected trading days are present. [2025-10-20T18:32:46.574978] [DEBUG] [VALIDATOR ] ELS: Primary data validation passed. [2025-10-20T18:32:46.576020] [INFO ] [MAIN ] ELS: Fetched primary data is valid. [2025-10-20T18:32:46.576179] [DEBUG] [LOCAL ] ELS: Saving primary data. [2025-10-20T18:32:46.577712] [DEBUG] [LOCAL ] ELS: Trimmed 71263 records to 49903 (keeping 200 days of data) [2025-10-20T18:32:46.667288] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-02. [2025-10-20T18:32:46.698281] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:32:46.914246] [DEBUG] [LOCAL ] ELS: Primary data saved (49903 records). [2025-10-20T18:32:46.914383] [INFO ] [MAIN ] ELS: Full history saved. [2025-10-20T18:32:46.914462] [INFO ] [MAIN ] Update Progress: 728/2367 symbols completed. ELS took 40.7s. Estimated remaining: 3h 28m 43s. Rate: 7.9 symbols/min [2025-10-20T18:32:47.084519] [INFO ] [MAIN ] EMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:47.133047] [DEBUG] [LOCAL ] EMN: Loaded 51488 existing records from primary storage. [2025-10-20T18:32:47.133355] [INFO ] [MAIN ] EMN: Loaded 51488 existing records. [2025-10-20T18:32:47.309406] [WARN ] [MAIN ] EMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:47.309679] [INFO ] [MAIN ] EMN: Wiping existing primary data before full refresh. [2025-10-20T18:32:47.309738] [INFO ] [LOCAL ] EMN: Starting deletion of primary data. [2025-10-20T18:32:47.309920] [DEBUG] [LOCAL ] EMN: Deleted primary data file. [2025-10-20T18:32:47.309958] [INFO ] [MAIN ] EMN: Starting full primary refresh. [2025-10-20T18:32:47.455481] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-09. [2025-10-20T18:32:47.672281] [INFO ] [REMOTE ] EMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:47.868134] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:32:48.251361] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-01. [2025-10-20T18:32:49.051430] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-07. [2025-10-20T18:32:49.851431] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-03. [2025-10-20T18:32:50.056554] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T18:32:50.472460] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:32:50.537291] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-09 (3.08 seconds). [2025-10-20T18:32:50.651376] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-04. [2025-10-20T18:32:51.210629] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T18:32:51.342772] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T18:32:51.451390] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-08. [2025-10-20T18:32:52.251371] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-05. [2025-10-20T18:32:52.746441] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-10-20T18:32:53.051360] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-02. [2025-10-20T18:32:53.211472] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:32:53.851414] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-09. [2025-10-20T18:32:54.566393] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:32:54.651393] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-10. [2025-10-20T18:32:55.451392] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-03. [2025-10-20T18:32:55.659816] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T18:32:56.251459] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-06. [2025-10-20T18:32:56.487512] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:32:56.495489] [INFO ] [REMOTE ] ELV: Full fetch complete (73297 records). [2025-10-20T18:32:56.495741] [DEBUG] [VALIDATOR ] ELV: Validating primary data. [2025-10-20T18:32:56.916473] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T18:32:57.020947] [DEBUG] [VALIDATOR ] ELV: All expected trading days are present. [2025-10-20T18:32:57.021022] [DEBUG] [VALIDATOR ] ELV: Primary data validation passed. [2025-10-20T18:32:57.021882] [INFO ] [MAIN ] ELV: Fetched primary data is valid. [2025-10-20T18:32:57.021944] [DEBUG] [LOCAL ] ELV: Saving primary data. [2025-10-20T18:32:57.023252] [DEBUG] [LOCAL ] ELV: Trimmed 73297 records to 50756 (keeping 200 days of data) [2025-10-20T18:32:57.078115] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-07. [2025-10-20T18:32:57.359634] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T18:32:57.381722] [DEBUG] [LOCAL ] ELV: Primary data saved (50756 records). [2025-10-20T18:32:57.381858] [INFO ] [MAIN ] ELV: Full history saved. [2025-10-20T18:32:57.381938] [INFO ] [MAIN ] Update Progress: 729/2367 symbols completed. ELV took 43.0s. Estimated remaining: 3h 28m 42s. Rate: 7.8 symbols/min [2025-10-20T18:32:57.491928] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-10-20T18:32:57.557532] [INFO ] [MAIN ] EMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:57.571887] [DEBUG] [LOCAL ] EMO: Loaded 13445 existing records from primary storage. [2025-10-20T18:32:57.572118] [INFO ] [MAIN ] EMO: Loaded 13445 existing records. [2025-10-20T18:32:57.749298] [WARN ] [MAIN ] EMO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:57.749583] [INFO ] [MAIN ] EMO: Wiping existing primary data before full refresh. [2025-10-20T18:32:57.749638] [INFO ] [LOCAL ] EMO: Starting deletion of primary data. [2025-10-20T18:32:57.749880] [DEBUG] [LOCAL ] EMO: Deleted primary data file. [2025-10-20T18:32:57.749939] [INFO ] [MAIN ] EMO: Starting full primary refresh. [2025-10-20T18:32:57.856689] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-04. [2025-10-20T18:32:58.099787] [INFO ] [REMOTE ] EMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:58.651388] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-08. [2025-10-20T18:32:59.130018] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T18:32:59.451394] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-09. [2025-10-20T18:32:59.610841] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T18:33:00.081818] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T18:33:00.251411] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-01. [2025-10-20T18:33:01.051392] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-10. [2025-10-20T18:33:01.331909] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T18:33:01.851461] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-05. [2025-10-20T18:33:02.651362] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-02. [2025-10-20T18:33:02.790782] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T18:33:02.797127] [INFO ] [REMOTE ] EME: Full fetch complete (47081 records). [2025-10-20T18:33:02.797372] [DEBUG] [VALIDATOR ] EME: Validating primary data. [2025-10-20T18:33:03.227962] [DEBUG] [VALIDATOR ] EME: All expected trading days are present. [2025-10-20T18:33:03.228047] [DEBUG] [VALIDATOR ] EME: Primary data validation passed. [2025-10-20T18:33:03.228755] [INFO ] [MAIN ] EME: Fetched primary data is valid. [2025-10-20T18:33:03.228876] [DEBUG] [LOCAL ] EME: Saving primary data. [2025-10-20T18:33:03.229996] [DEBUG] [LOCAL ] EME: Trimmed 47081 records to 30809 (keeping 200 days of data) [2025-10-20T18:33:03.460027] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-10. [2025-10-20T18:33:03.488844] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T18:33:03.492772] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T18:33:03.491013] [DEBUG] [LOCAL ] EME: Primary data saved (30809 records). [2025-10-20T18:33:03.505129] [INFO ] [MAIN ] EME: Full history saved. [2025-10-20T18:33:03.506389] [INFO ] [MAIN ] Update Progress: 730/2367 symbols completed. EME took 36.5s. Estimated remaining: 3h 28m 31s. Rate: 7.9 symbols/min [2025-10-20T18:33:03.676164] [INFO ] [MAIN ] EMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:03.681544] [DEBUG] [LOCAL ] EMP: Loaded 3057 existing records from primary storage. [2025-10-20T18:33:03.681713] [INFO ] [MAIN ] EMP: Loaded 3057 existing records. [2025-10-20T18:33:03.857231] [WARN ] [MAIN ] EMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:03.857439] [INFO ] [MAIN ] EMP: Wiping existing primary data before full refresh. [2025-10-20T18:33:03.857495] [INFO ] [LOCAL ] EMP: Starting deletion of primary data. [2025-10-20T18:33:03.857806] [DEBUG] [LOCAL ] EMP: Deleted primary data file. [2025-10-20T18:33:03.857863] [INFO ] [MAIN ] EMP: Starting full primary refresh. [2025-10-20T18:33:04.203881] [INFO ] [REMOTE ] EMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:04.251351] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-01. [2025-10-20T18:33:04.256754] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T18:33:04.258153] [INFO ] [REMOTE ] EMF: Full fetch complete (6225 records). [2025-10-20T18:33:04.258225] [DEBUG] [VALIDATOR ] EMF: Validating primary data. [2025-10-20T18:33:04.634723] [DEBUG] [VALIDATOR ] EMF: All expected trading days are present. [2025-10-20T18:33:04.634845] [DEBUG] [VALIDATOR ] EMF: Primary data validation passed. [2025-10-20T18:33:04.635069] [INFO ] [MAIN ] EMF: Fetched primary data is valid. [2025-10-20T18:33:04.635109] [DEBUG] [LOCAL ] EMF: Saving primary data. [2025-10-20T18:33:04.635493] [DEBUG] [LOCAL ] EMF: Trimmed 6225 records to 3172 (keeping 200 days of data) [2025-10-20T18:33:04.654380] [DEBUG] [LOCAL ] EMF: Primary data saved (3172 records). [2025-10-20T18:33:04.654441] [INFO ] [MAIN ] EMF: Full history saved. [2025-10-20T18:33:04.654509] [INFO ] [MAIN ] Update Progress: 731/2367 symbols completed. EMF took 25.1s. Estimated remaining: 3h 28m 9s. Rate: 7.9 symbols/min [2025-10-20T18:33:04.821571] [INFO ] [MAIN ] EMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:04.873878] [DEBUG] [LOCAL ] EMR: Loaded 53972 existing records from primary storage. [2025-10-20T18:33:04.874186] [INFO ] [MAIN ] EMR: Loaded 53972 existing records. [2025-10-20T18:33:04.966423] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T18:33:05.051658] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-06. [2025-10-20T18:33:05.051564] [WARN ] [MAIN ] EMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:05.051881] [INFO ] [MAIN ] EMR: Wiping existing primary data before full refresh. [2025-10-20T18:33:05.052628] [INFO ] [LOCAL ] EMR: Starting deletion of primary data. [2025-10-20T18:33:05.052892] [DEBUG] [LOCAL ] EMR: Deleted primary data file. [2025-10-20T18:33:05.052948] [INFO ] [MAIN ] EMR: Starting full primary refresh. [2025-10-20T18:33:05.399874] [INFO ] [REMOTE ] EMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:05.851383] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-02. [2025-10-20T18:33:06.387107] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T18:33:06.656454] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-01. [2025-10-20T18:33:06.675828] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:33:07.451430] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-03. [2025-10-20T18:33:08.240257] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T18:33:08.251326] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-04. [2025-10-20T18:33:08.496272] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:33:09.051345] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-02. [2025-10-20T18:33:09.222070] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T18:33:09.851386] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-07. [2025-10-20T18:33:10.651411] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-03. [2025-10-20T18:33:10.942530] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T18:33:10.974834] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-10-20T18:33:11.422562] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T18:33:11.451337] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-08. [2025-10-20T18:33:12.251379] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-04. [2025-10-20T18:33:12.645718] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-10-20T18:33:13.051367] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-05. [2025-10-20T18:33:13.219251] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:33:13.520025] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T18:33:13.851409] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-06. [2025-10-20T18:33:14.324193] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T18:33:14.651379] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-05. [2025-10-20T18:33:15.451380] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-07. [2025-10-20T18:33:15.571753] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T18:33:15.915710] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T18:33:16.251382] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-03. [2025-10-20T18:33:17.051376] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-09. [2025-10-20T18:33:17.851367] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-06. [2025-10-20T18:33:18.042539] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T18:33:18.651364] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-08. [2025-10-20T18:33:18.711167] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T18:33:18.781578] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T18:33:19.167012] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T18:33:19.451369] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-07. [2025-10-20T18:33:20.251370] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-04. [2025-10-20T18:33:20.437955] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T18:33:21.051377] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-10. [2025-10-20T18:33:21.851368] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-09. [2025-10-20T18:33:22.212266] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-09 (0.36 seconds). [2025-10-20T18:33:22.212114] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T18:33:22.651394] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-08. [2025-10-20T18:33:23.076084] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-10 (2.02 seconds). [2025-10-20T18:33:23.083905] [INFO ] [REMOTE ] EMN: Full fetch complete (72393 records). [2025-10-20T18:33:23.084037] [DEBUG] [VALIDATOR ] EMN: Validating primary data. [2025-10-20T18:33:23.428512] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T18:33:23.463899] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-05. [2025-10-20T18:33:23.525327] [DEBUG] [VALIDATOR ] EMN: All expected trading days are present. [2025-10-20T18:33:23.525417] [DEBUG] [VALIDATOR ] EMN: Primary data validation passed. [2025-10-20T18:33:23.526489] [INFO ] [MAIN ] EMN: Fetched primary data is valid. [2025-10-20T18:33:23.526557] [DEBUG] [LOCAL ] EMN: Saving primary data. [2025-10-20T18:33:23.528284] [DEBUG] [LOCAL ] EMN: Trimmed 72393 records to 50838 (keeping 200 days of data) [2025-10-20T18:33:23.841845] [DEBUG] [LOCAL ] EMN: Primary data saved (50838 records). [2025-10-20T18:33:23.841989] [INFO ] [MAIN ] EMN: Full history saved. [2025-10-20T18:33:23.842070] [INFO ] [MAIN ] Update Progress: 732/2367 symbols completed. EMN took 36.9s. Estimated remaining: 3h 28m 27s. Rate: 7.8 symbols/min [2025-10-20T18:33:24.019617] [INFO ] [MAIN ] ENB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:24.075525] [DEBUG] [LOCAL ] ENB: Loaded 54292 existing records from primary storage. [2025-10-20T18:33:24.075917] [INFO ] [MAIN ] ENB: Loaded 54292 existing records. [2025-10-20T18:33:24.253702] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-09. [2025-10-20T18:33:24.253542] [WARN ] [MAIN ] ENB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:24.254215] [INFO ] [MAIN ] ENB: Wiping existing primary data before full refresh. [2025-10-20T18:33:24.255269] [INFO ] [LOCAL ] ENB: Starting deletion of primary data. [2025-10-20T18:33:24.255476] [DEBUG] [LOCAL ] ENB: Deleted primary data file. [2025-10-20T18:33:24.255517] [INFO ] [MAIN ] ENB: Starting full primary refresh. [2025-10-20T18:33:24.602768] [INFO ] [REMOTE ] ENB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:25.051372] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-10. [2025-10-20T18:33:25.292797] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T18:33:25.582376] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T18:33:25.585576] [INFO ] [REMOTE ] EMP: Full fetch complete (4089 records). [2025-10-20T18:33:25.584423] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T18:33:25.585794] [DEBUG] [VALIDATOR ] EMP: Validating primary data. [2025-10-20T18:33:25.856597] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-06. [2025-10-20T18:33:25.957903] [DEBUG] [VALIDATOR ] EMP: All expected trading days are present. [2025-10-20T18:33:25.957983] [DEBUG] [VALIDATOR ] EMP: Primary data validation passed. [2025-10-20T18:33:25.958124] [INFO ] [MAIN ] EMP: Fetched primary data is valid. [2025-10-20T18:33:25.958162] [DEBUG] [LOCAL ] EMP: Saving primary data. [2025-10-20T18:33:25.958534] [DEBUG] [LOCAL ] EMP: Trimmed 4089 records to 2945 (keeping 200 days of data) [2025-10-20T18:33:25.977617] [DEBUG] [LOCAL ] EMP: Primary data saved (2945 records). [2025-10-20T18:33:25.977773] [INFO ] [MAIN ] EMP: Full history saved. [2025-10-20T18:33:25.977856] [INFO ] [MAIN ] Update Progress: 733/2367 symbols completed. EMP took 22.5s. Estimated remaining: 3h 28m 7s. Rate: 7.9 symbols/min [2025-10-20T18:33:26.148207] [INFO ] [MAIN ] ENIC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:26.184818] [DEBUG] [LOCAL ] ENIC: Loaded 37209 existing records from primary storage. [2025-10-20T18:33:26.185175] [INFO ] [MAIN ] ENIC: Loaded 37209 existing records. [2025-10-20T18:33:26.384389] [WARN ] [MAIN ] ENIC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:26.384619] [INFO ] [MAIN ] ENIC: Wiping existing primary data before full refresh. [2025-10-20T18:33:26.384701] [INFO ] [LOCAL ] ENIC: Starting deletion of primary data. [2025-10-20T18:33:26.384932] [DEBUG] [LOCAL ] ENIC: Deleted primary data file. [2025-10-20T18:33:26.384988] [INFO ] [MAIN ] ENIC: Starting full primary refresh. [2025-10-20T18:33:26.657098] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-01. [2025-10-20T18:33:26.784142] [INFO ] [REMOTE ] ENIC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:27.456742] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-10. [2025-10-20T18:33:27.472416] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:33:28.251409] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-01. [2025-10-20T18:33:28.476716] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T18:33:28.508922] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T18:33:28.511846] [INFO ] [REMOTE ] EMO: Full fetch complete (17669 records). [2025-10-20T18:33:28.511935] [DEBUG] [VALIDATOR ] EMO: Validating primary data. [2025-10-20T18:33:28.882431] [DEBUG] [VALIDATOR ] EMO: All expected trading days are present. [2025-10-20T18:33:28.882539] [DEBUG] [VALIDATOR ] EMO: Primary data validation passed. [2025-10-20T18:33:28.882947] [INFO ] [MAIN ] EMO: Fetched primary data is valid. [2025-10-20T18:33:28.883007] [DEBUG] [LOCAL ] EMO: Saving primary data. [2025-10-20T18:33:28.883621] [DEBUG] [LOCAL ] EMO: Trimmed 17669 records to 13342 (keeping 200 days of data) [2025-10-20T18:33:28.964849] [DEBUG] [LOCAL ] EMO: Primary data saved (13342 records). [2025-10-20T18:33:28.964998] [INFO ] [MAIN ] EMO: Full history saved. [2025-10-20T18:33:28.965079] [INFO ] [MAIN ] Update Progress: 734/2367 symbols completed. EMO took 31.6s. Estimated remaining: 3h 27m 49s. Rate: 7.9 symbols/min [2025-10-20T18:33:29.061275] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-07. [2025-10-20T18:33:29.135030] [INFO ] [MAIN ] ENO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:29.139191] [DEBUG] [LOCAL ] ENO: Loaded 1872 existing records from primary storage. [2025-10-20T18:33:29.139346] [INFO ] [MAIN ] ENO: Loaded 1872 existing records. [2025-10-20T18:33:29.313596] [WARN ] [MAIN ] ENO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:29.313817] [INFO ] [MAIN ] ENO: Wiping existing primary data before full refresh. [2025-10-20T18:33:29.313863] [INFO ] [LOCAL ] ENO: Starting deletion of primary data. [2025-10-20T18:33:29.314046] [DEBUG] [LOCAL ] ENO: Deleted primary data file. [2025-10-20T18:33:29.314089] [INFO ] [MAIN ] ENO: Starting full primary refresh. [2025-10-20T18:33:29.703091] [INFO ] [REMOTE ] ENO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:29.726508] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:33:29.851359] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-01. [2025-10-20T18:33:30.367871] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T18:33:30.651413] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-02. [2025-10-20T18:33:31.016523] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T18:33:31.451409] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-02. [2025-10-20T18:33:31.920333] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T18:33:31.932614] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T18:33:32.251417] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-03. [2025-10-20T18:33:32.657323] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-10-20T18:33:33.051373] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-02. [2025-10-20T18:33:33.851388] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-03. [2025-10-20T18:33:34.651407] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-04. [2025-10-20T18:33:34.772802] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T18:33:35.187715] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T18:33:35.294955] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T18:33:35.451346] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-04. [2025-10-20T18:33:36.251357] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-08. [2025-10-20T18:33:36.882824] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:33:37.051590] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-05. [2025-10-20T18:33:37.488633] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T18:33:37.851378] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-05. [2025-10-20T18:33:38.152508] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T18:33:38.651352] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-06. [2025-10-20T18:33:39.079427] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T18:33:39.314146] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T18:33:39.451374] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-07. [2025-10-20T18:33:39.883057] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T18:33:40.251378] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-03. [2025-10-20T18:33:41.051410] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-08. [2025-10-20T18:33:41.512345] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T18:33:41.851376] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-09. [2025-10-20T18:33:42.328519] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T18:33:42.651448] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-06. [2025-10-20T18:33:43.451442] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-09. [2025-10-20T18:33:43.733215] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T18:33:43.935699] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T18:33:44.202040] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T18:33:44.251369] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-07. [2025-10-20T18:33:45.051397] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-04. [2025-10-20T18:33:45.851398] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-10. [2025-10-20T18:33:45.892154] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:33:46.520353] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T18:33:46.521177] [INFO ] [REMOTE ] ENO: Full fetch complete (2989 records). [2025-10-20T18:33:46.521287] [DEBUG] [VALIDATOR ] ENO: Validating primary data. [2025-10-20T18:33:46.521621] [FATAL] [VALIDATOR ] ENO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:33:46.521723] [FATAL] [MAIN ] ENO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:33:46.521790] [WARN ] [MAIN ] ENO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:33:46.521862] [INFO ] [LOCAL ] ENO: Starting deletion of primary data. [2025-10-20T18:33:46.521926] [DEBUG] [LOCAL ] ENO: No primary data file found to delete. [2025-10-20T18:33:46.521962] [INFO ] [LOCAL ] ENO: Starting deletion of derivative data. [2025-10-20T18:33:46.522090] [DEBUG] [LOCAL ] ENO: Deleted derivative data file. [2025-10-20T18:33:46.522157] [INFO ] [MAIN ] Update Progress: 735/2367 symbols completed. ENO took 17.6s. Estimated remaining: 3h 28m 3s. Rate: 7.8 symbols/min [2025-10-20T18:33:46.661438] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-08. [2025-10-20T18:33:46.696797] [INFO ] [MAIN ] ENOV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:46.737998] [DEBUG] [LOCAL ] ENOV: Loaded 45385 existing records from primary storage. [2025-10-20T18:33:46.738337] [INFO ] [MAIN ] ENOV: Loaded 45385 existing records. [2025-10-20T18:33:46.919777] [WARN ] [MAIN ] ENOV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:46.919984] [INFO ] [MAIN ] ENOV: Wiping existing primary data before full refresh. [2025-10-20T18:33:46.920029] [INFO ] [LOCAL ] ENOV: Starting deletion of primary data. [2025-10-20T18:33:46.920240] [DEBUG] [LOCAL ] ENOV: Deleted primary data file. [2025-10-20T18:33:46.920296] [INFO ] [MAIN ] ENOV: Starting full primary refresh. [2025-10-20T18:33:47.349247] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-04 (2.30 seconds). [2025-10-20T18:33:47.355153] [INFO ] [REMOTE ] ENOV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:47.451421] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-05. [2025-10-20T18:33:48.108083] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:33:48.251390] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-09. [2025-10-20T18:33:49.051513] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-10. [2025-10-20T18:33:49.227500] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T18:33:49.750249] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:33:49.851348] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-10. [2025-10-20T18:33:50.651424] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-01. [2025-10-20T18:33:51.327020] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T18:33:51.332163] [INFO ] [REMOTE ] ENIC: Full fetch complete (51383 records). [2025-10-20T18:33:51.332291] [DEBUG] [VALIDATOR ] ENIC: Validating primary data. [2025-10-20T18:33:51.455842] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-06. [2025-10-20T18:33:51.741204] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-10 (2.69 seconds). [2025-10-20T18:33:51.766903] [DEBUG] [VALIDATOR ] ENIC: All expected trading days are present. [2025-10-20T18:33:51.767179] [DEBUG] [VALIDATOR ] ENIC: Primary data validation passed. [2025-10-20T18:33:51.768029] [INFO ] [MAIN ] ENIC: Fetched primary data is valid. [2025-10-20T18:33:51.768116] [DEBUG] [LOCAL ] ENIC: Saving primary data. [2025-10-20T18:33:51.768441] [INFO ] [REMOTE ] EMR: Full fetch complete (77223 records). [2025-10-20T18:33:51.769190] [DEBUG] [VALIDATOR ] EMR: Validating primary data. [2025-10-20T18:33:51.769987] [DEBUG] [LOCAL ] ENIC: Trimmed 51383 records to 36571 (keeping 200 days of data) [2025-10-20T18:33:52.367366] [DEBUG] [LOCAL ] ENIC: Primary data saved (36571 records). [2025-10-20T18:33:52.390419] [INFO ] [MAIN ] ENIC: Full history saved. [2025-10-20T18:33:52.430687] [DEBUG] [VALIDATOR ] EMR: All expected trading days are present. [2025-10-20T18:33:52.430919] [INFO ] [MAIN ] Update Progress: 736/2367 symbols completed. ENIC took 26.5s. Estimated remaining: 3h 27m 52s. Rate: 7.8 symbols/min [2025-10-20T18:33:52.431149] [DEBUG] [VALIDATOR ] EMR: Primary data validation passed. [2025-10-20T18:33:52.437571] [INFO ] [MAIN ] EMR: Fetched primary data is valid. [2025-10-20T18:33:52.463944] [DEBUG] [LOCAL ] EMR: Saving primary data. [2025-10-20T18:33:52.545732] [DEBUG] [LOCAL ] EMR: Trimmed 77223 records to 53176 (keeping 200 days of data) [2025-10-20T18:33:52.637692] [INFO ] [MAIN ] ENR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:52.640556] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T18:33:52.707061] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-02. [2025-10-20T18:33:52.813324] [DEBUG] [LOCAL ] ENR: Loaded 44323 existing records from primary storage. [2025-10-20T18:33:52.816118] [INFO ] [MAIN ] ENR: Loaded 44323 existing records. [2025-10-20T18:33:53.106477] [WARN ] [MAIN ] ENR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:53.109119] [INFO ] [MAIN ] ENR: Wiping existing primary data before full refresh. [2025-10-20T18:33:53.109468] [INFO ] [LOCAL ] ENR: Starting deletion of primary data. [2025-10-20T18:33:53.110349] [DEBUG] [LOCAL ] ENR: Deleted primary data file. [2025-10-20T18:33:53.111214] [INFO ] [MAIN ] ENR: Starting full primary refresh. [2025-10-20T18:33:53.572463] [INFO ] [REMOTE ] ENR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:53.573398] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-01. [2025-10-20T18:33:53.611282] [DEBUG] [LOCAL ] EMR: Primary data saved (53176 records). [2025-10-20T18:33:53.612144] [INFO ] [MAIN ] EMR: Full history saved. [2025-10-20T18:33:53.613326] [INFO ] [MAIN ] Update Progress: 737/2367 symbols completed. EMR took 49.0s. Estimated remaining: 3h 27m 30s. Rate: 7.9 symbols/min [2025-10-20T18:33:53.727173] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-06 (2.27 seconds). [2025-10-20T18:33:53.868866] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-07. [2025-10-20T18:33:53.870871] [INFO ] [MAIN ] ENS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:53.872154] [DEBUG] [LOCAL ] ENS: No existing data file found in primary storage. [2025-10-20T18:33:53.872217] [WARN ] [MAIN ] ENS: No existing data found, performing full refresh instead. [2025-10-20T18:33:53.872347] [INFO ] [MAIN ] ENS: Wiping existing primary data before full refresh. [2025-10-20T18:33:53.872391] [INFO ] [LOCAL ] ENS: Starting deletion of primary data. [2025-10-20T18:33:53.872434] [DEBUG] [LOCAL ] ENS: No primary data file found to delete. [2025-10-20T18:33:53.872469] [INFO ] [MAIN ] ENS: Starting full primary refresh. [2025-10-20T18:33:54.094185] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T18:33:54.284896] [INFO ] [REMOTE ] ENS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:54.651410] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-01. [2025-10-20T18:33:55.082058] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T18:33:55.451380] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-03. [2025-10-20T18:33:55.986211] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-10-20T18:33:56.198881] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T18:33:56.251441] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-08. [2025-10-20T18:33:57.051381] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-02. [2025-10-20T18:33:57.119129] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T18:33:57.851410] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-04. [2025-10-20T18:33:58.139396] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T18:33:58.477249] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T18:33:58.651356] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-03. [2025-10-20T18:33:59.109104] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T18:33:59.451443] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-09. [2025-10-20T18:34:00.075792] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T18:34:00.251412] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-04. [2025-10-20T18:34:01.051362] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-05. [2025-10-20T18:34:01.397299] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T18:34:01.808556] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T18:34:01.851367] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-10. [2025-10-20T18:34:02.656787] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-05. [2025-10-20T18:34:02.673994] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:34:03.451341] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-02. [2025-10-20T18:34:04.059543] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T18:34:04.067820] [INFO ] [REMOTE ] ENB: Full fetch complete (77656 records). [2025-10-20T18:34:04.067972] [DEBUG] [VALIDATOR ] ENB: Validating primary data. [2025-10-20T18:34:04.270844] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-06. [2025-10-20T18:34:04.478926] [DEBUG] [VALIDATOR ] ENB: All expected trading days are present. [2025-10-20T18:34:04.479066] [DEBUG] [VALIDATOR ] ENB: Primary data validation passed. [2025-10-20T18:34:04.480469] [INFO ] [MAIN ] ENB: Fetched primary data is valid. [2025-10-20T18:34:04.480681] [DEBUG] [LOCAL ] ENB: Saving primary data. [2025-10-20T18:34:04.508990] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:34:04.518610] [DEBUG] [LOCAL ] ENB: Trimmed 77656 records to 53511 (keeping 200 days of data) [2025-10-20T18:34:04.639064] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T18:34:04.892225] [DEBUG] [LOCAL ] ENB: Primary data saved (53511 records). [2025-10-20T18:34:04.892425] [INFO ] [MAIN ] ENB: Full history saved. [2025-10-20T18:34:04.892531] [INFO ] [MAIN ] Update Progress: 738/2367 symbols completed. ENB took 41.1s. Estimated remaining: 3h 27m 30s. Rate: 7.9 symbols/min [2025-10-20T18:34:05.056570] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-06. [2025-10-20T18:34:05.092458] [INFO ] [MAIN ] ENVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:05.121076] [DEBUG] [LOCAL ] ENVA: Loaded 25790 existing records from primary storage. [2025-10-20T18:34:05.121413] [INFO ] [MAIN ] ENVA: Loaded 25790 existing records. [2025-10-20T18:34:05.339423] [WARN ] [MAIN ] ENVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:05.339672] [INFO ] [MAIN ] ENVA: Wiping existing primary data before full refresh. [2025-10-20T18:34:05.339739] [INFO ] [LOCAL ] ENVA: Starting deletion of primary data. [2025-10-20T18:34:05.339996] [DEBUG] [LOCAL ] ENVA: Deleted primary data file. [2025-10-20T18:34:05.340054] [INFO ] [MAIN ] ENVA: Starting full primary refresh. [2025-10-20T18:34:05.756554] [INFO ] [REMOTE ] ENVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:05.851384] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-01. [2025-10-20T18:34:05.894511] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:34:06.651376] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-07. [2025-10-20T18:34:06.950254] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T18:34:07.055712] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T18:34:07.451441] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-03. [2025-10-20T18:34:08.251378] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-02. [2025-10-20T18:34:08.408158] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T18:34:08.849053] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:34:09.051406] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-07. [2025-10-20T18:34:09.538168] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T18:34:09.851403] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-03. [2025-10-20T18:34:10.572577] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T18:34:10.651365] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-08. [2025-10-20T18:34:10.981934] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T18:34:11.451372] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-04. [2025-10-20T18:34:12.251357] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-08. [2025-10-20T18:34:12.298224] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T18:34:12.766170] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T18:34:13.051385] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-09. [2025-10-20T18:34:13.851383] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-05. [2025-10-20T18:34:13.978681] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:34:14.651369] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-09. [2025-10-20T18:34:14.867201] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:34:15.371754] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T18:34:15.451380] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-06. [2025-10-20T18:34:16.108420] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T18:34:16.251413] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-10. [2025-10-20T18:34:17.035878] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T18:34:17.051347] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-07. [2025-10-20T18:34:17.851364] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-10. [2025-10-20T18:34:18.079989] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:34:18.089430] [INFO ] [REMOTE ] ENR: Full fetch complete (59540 records). [2025-10-20T18:34:18.089620] [DEBUG] [VALIDATOR ] ENR: Validating primary data. [2025-10-20T18:34:18.485077] [DEBUG] [VALIDATOR ] ENR: All expected trading days are present. [2025-10-20T18:34:18.491417] [DEBUG] [VALIDATOR ] ENR: Primary data validation passed. [2025-10-20T18:34:18.503562] [INFO ] [MAIN ] ENR: Fetched primary data is valid. [2025-10-20T18:34:18.503703] [DEBUG] [LOCAL ] ENR: Saving primary data. [2025-10-20T18:34:18.505559] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T18:34:18.513147] [DEBUG] [LOCAL ] ENR: Trimmed 59540 records to 43825 (keeping 200 days of data) [2025-10-20T18:34:18.690936] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-04. [2025-10-20T18:34:18.796027] [DEBUG] [LOCAL ] ENR: Primary data saved (43825 records). [2025-10-20T18:34:18.796202] [INFO ] [MAIN ] ENR: Full history saved. [2025-10-20T18:34:18.796310] [INFO ] [MAIN ] Update Progress: 739/2367 symbols completed. ENR took 26.4s. Estimated remaining: 3h 27m 36s. Rate: 7.8 symbols/min [2025-10-20T18:34:18.976555] [INFO ] [MAIN ] EOD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:18.992747] [DEBUG] [LOCAL ] EOD: Loaded 15256 existing records from primary storage. [2025-10-20T18:34:18.992990] [INFO ] [MAIN ] EOD: Loaded 15256 existing records. [2025-10-20T18:34:19.172148] [WARN ] [MAIN ] EOD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:19.172359] [INFO ] [MAIN ] EOD: Wiping existing primary data before full refresh. [2025-10-20T18:34:19.172405] [INFO ] [LOCAL ] EOD: Starting deletion of primary data. [2025-10-20T18:34:19.172573] [DEBUG] [LOCAL ] EOD: Deleted primary data file. [2025-10-20T18:34:19.172608] [INFO ] [MAIN ] EOD: Starting full primary refresh. [2025-10-20T18:34:19.453580] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-08. [2025-10-20T18:34:19.486905] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T18:34:19.530524] [INFO ] [REMOTE ] ENOV: Full fetch complete (63455 records). [2025-10-20T18:34:19.537670] [DEBUG] [VALIDATOR ] ENOV: Validating primary data. [2025-10-20T18:34:19.557725] [INFO ] [REMOTE ] EOD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:19.958406] [DEBUG] [VALIDATOR ] ENOV: All expected trading days are present. [2025-10-20T18:34:19.958484] [DEBUG] [VALIDATOR ] ENOV: Primary data validation passed. [2025-10-20T18:34:19.959290] [INFO ] [MAIN ] ENOV: Fetched primary data is valid. [2025-10-20T18:34:19.959352] [DEBUG] [LOCAL ] ENOV: Saving primary data. [2025-10-20T18:34:19.960552] [DEBUG] [LOCAL ] ENOV: Trimmed 63455 records to 44779 (keeping 200 days of data) [2025-10-20T18:34:20.215035] [DEBUG] [LOCAL ] ENOV: Primary data saved (44779 records). [2025-10-20T18:34:20.215165] [INFO ] [MAIN ] ENOV: Full history saved. [2025-10-20T18:34:20.215244] [INFO ] [MAIN ] Update Progress: 740/2367 symbols completed. ENOV took 33.7s. Estimated remaining: 3h 27m 15s. Rate: 7.9 symbols/min [2025-10-20T18:34:20.283478] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-01. [2025-10-20T18:34:20.338769] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:34:20.415131] [INFO ] [MAIN ] EOG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:20.469399] [DEBUG] [LOCAL ] EOG: Loaded 54152 existing records from primary storage. [2025-10-20T18:34:20.469857] [INFO ] [MAIN ] EOG: Loaded 54152 existing records. [2025-10-20T18:34:20.714108] [WARN ] [MAIN ] EOG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:20.714314] [INFO ] [MAIN ] EOG: Wiping existing primary data before full refresh. [2025-10-20T18:34:20.714359] [INFO ] [LOCAL ] EOG: Starting deletion of primary data. [2025-10-20T18:34:20.714542] [DEBUG] [LOCAL ] EOG: Deleted primary data file. [2025-10-20T18:34:20.714580] [INFO ] [MAIN ] EOG: Starting full primary refresh. [2025-10-20T18:34:21.020124] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:34:21.066796] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-09. [2025-10-20T18:34:21.113776] [INFO ] [REMOTE ] EOG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:21.518947] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:34:21.851373] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-01. [2025-10-20T18:34:22.534207] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T18:34:22.651432] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-05. [2025-10-20T18:34:23.451372] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-10. [2025-10-20T18:34:23.727812] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T18:34:23.729357] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T18:34:24.251438] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-02. [2025-10-20T18:34:24.557212] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:34:24.562966] [INFO ] [REMOTE ] ENS: Full fetch complete (42567 records). [2025-10-20T18:34:24.563106] [DEBUG] [VALIDATOR ] ENS: Validating primary data. [2025-10-20T18:34:24.564671] [FATAL] [VALIDATOR ] ENS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:34:24.564887] [FATAL] [MAIN ] ENS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:34:24.564956] [WARN ] [MAIN ] ENS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:34:24.565031] [INFO ] [LOCAL ] ENS: Starting deletion of primary data. [2025-10-20T18:34:24.565103] [DEBUG] [LOCAL ] ENS: No primary data file found to delete. [2025-10-20T18:34:24.565141] [INFO ] [LOCAL ] ENS: Starting deletion of derivative data. [2025-10-20T18:34:24.565180] [DEBUG] [LOCAL ] ENS: No derivative data file found to delete. [2025-10-20T18:34:24.565249] [INFO ] [MAIN ] Update Progress: 741/2367 symbols completed. ENS took 30.9s. Estimated remaining: 3h 27m 0s. Rate: 7.9 symbols/min [2025-10-20T18:34:24.740530] [INFO ] [MAIN ] EOI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:24.752393] [DEBUG] [LOCAL ] EOI: Loaded 10809 existing records from primary storage. [2025-10-20T18:34:24.752632] [INFO ] [MAIN ] EOI: Loaded 10809 existing records. [2025-10-20T18:34:24.950687] [WARN ] [MAIN ] EOI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:24.951112] [INFO ] [MAIN ] EOI: Wiping existing primary data before full refresh. [2025-10-20T18:34:24.951236] [INFO ] [LOCAL ] EOI: Starting deletion of primary data. [2025-10-20T18:34:24.951505] [DEBUG] [LOCAL ] EOI: Deleted primary data file. [2025-10-20T18:34:24.951563] [INFO ] [MAIN ] EOI: Starting full primary refresh. [2025-10-20T18:34:25.052433] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-06. [2025-10-20T18:34:25.204766] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T18:34:25.333572] [INFO ] [REMOTE ] EOI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:25.851401] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-03. [2025-10-20T18:34:26.234090] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T18:34:26.373118] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T18:34:26.651464] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-04. [2025-10-20T18:34:27.392623] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T18:34:27.451365] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-05. [2025-10-20T18:34:28.252363] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-07. [2025-10-20T18:34:28.255626] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T18:34:29.051407] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-02. [2025-10-20T18:34:29.281166] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T18:34:29.851380] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-06. [2025-10-20T18:34:30.651357] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-08. [2025-10-20T18:34:30.777739] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T18:34:30.864536] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T18:34:31.451413] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-01. [2025-10-20T18:34:31.627748] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:34:32.251402] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-09. [2025-10-20T18:34:32.391166] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:34:33.051350] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-03. [2025-10-20T18:34:33.421583] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:34:33.851382] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-10. [2025-10-20T18:34:34.651349] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-07. [2025-10-20T18:34:34.944456] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:34:34.977499] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T18:34:34.981249] [INFO ] [REMOTE ] ENVA: Full fetch complete (36606 records). [2025-10-20T18:34:34.981324] [DEBUG] [VALIDATOR ] ENVA: Validating primary data. [2025-10-20T18:34:35.349361] [DEBUG] [VALIDATOR ] ENVA: All expected trading days are present. [2025-10-20T18:34:35.349437] [DEBUG] [VALIDATOR ] ENVA: Primary data validation passed. [2025-10-20T18:34:35.350008] [INFO ] [MAIN ] ENVA: Fetched primary data is valid. [2025-10-20T18:34:35.350070] [DEBUG] [LOCAL ] ENVA: Saving primary data. [2025-10-20T18:34:35.350893] [DEBUG] [LOCAL ] ENVA: Trimmed 36606 records to 25413 (keeping 200 days of data) [2025-10-20T18:34:35.454719] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-02. [2025-10-20T18:34:35.501570] [DEBUG] [LOCAL ] ENVA: Primary data saved (25413 records). [2025-10-20T18:34:35.501734] [INFO ] [MAIN ] ENVA: Full history saved. [2025-10-20T18:34:35.501819] [INFO ] [MAIN ] Update Progress: 742/2367 symbols completed. ENVA took 30.6s. Estimated remaining: 3h 26m 60s. Rate: 7.9 symbols/min [2025-10-20T18:34:35.686613] [INFO ] [MAIN ] EOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:35.701335] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T18:34:35.719585] [DEBUG] [LOCAL ] EOS: Loaded 15294 existing records from primary storage. [2025-10-20T18:34:35.719870] [INFO ] [MAIN ] EOS: Loaded 15294 existing records. [2025-10-20T18:34:35.899017] [WARN ] [MAIN ] EOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:35.899267] [INFO ] [MAIN ] EOS: Wiping existing primary data before full refresh. [2025-10-20T18:34:35.899316] [INFO ] [LOCAL ] EOS: Starting deletion of primary data. [2025-10-20T18:34:35.899504] [DEBUG] [LOCAL ] EOS: Deleted primary data file. [2025-10-20T18:34:35.899559] [INFO ] [MAIN ] EOS: Starting full primary refresh. [2025-10-20T18:34:36.266929] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-04. [2025-10-20T18:34:36.271537] [INFO ] [REMOTE ] EOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:36.316142] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T18:34:37.051394] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-03. [2025-10-20T18:34:37.851348] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-08. [2025-10-20T18:34:37.937037] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T18:34:38.135489] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T18:34:38.651400] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-04. [2025-10-20T18:34:38.779874] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:34:39.451335] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-05. [2025-10-20T18:34:39.499102] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T18:34:40.251419] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-05. [2025-10-20T18:34:41.051409] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-09. [2025-10-20T18:34:41.081993] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:34:41.399471] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T18:34:41.851415] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-01. [2025-10-20T18:34:41.951862] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T18:34:42.651389] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-06. [2025-10-20T18:34:43.189607] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T18:34:43.451386] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-10. [2025-10-20T18:34:44.259815] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-02. [2025-10-20T18:34:44.286399] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:34:44.288801] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:34:44.303029] [INFO ] [REMOTE ] EOD: Full fetch complete (20984 records). [2025-10-20T18:34:44.303144] [DEBUG] [VALIDATOR ] EOD: Validating primary data. [2025-10-20T18:34:44.659571] [DEBUG] [VALIDATOR ] EOD: All expected trading days are present. [2025-10-20T18:34:44.659664] [DEBUG] [VALIDATOR ] EOD: Primary data validation passed. [2025-10-20T18:34:44.660099] [INFO ] [MAIN ] EOD: Fetched primary data is valid. [2025-10-20T18:34:44.660138] [DEBUG] [LOCAL ] EOD: Saving primary data. [2025-10-20T18:34:44.660695] [DEBUG] [LOCAL ] EOD: Trimmed 20984 records to 15056 (keeping 200 days of data) [2025-10-20T18:34:44.742510] [DEBUG] [LOCAL ] EOD: Primary data saved (15056 records). [2025-10-20T18:34:44.742631] [INFO ] [MAIN ] EOD: Full history saved. [2025-10-20T18:34:44.742746] [INFO ] [MAIN ] Update Progress: 743/2367 symbols completed. EOD took 25.9s. Estimated remaining: 3h 26m 56s. Rate: 7.8 symbols/min [2025-10-20T18:34:44.912776] [INFO ] [MAIN ] EOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:44.921755] [DEBUG] [LOCAL ] EOT: Loaded 7100 existing records from primary storage. [2025-10-20T18:34:44.921936] [INFO ] [MAIN ] EOT: Loaded 7100 existing records. [2025-10-20T18:34:45.060777] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-07. [2025-10-20T18:34:45.121218] [WARN ] [MAIN ] EOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:45.121481] [INFO ] [MAIN ] EOT: Wiping existing primary data before full refresh. [2025-10-20T18:34:45.121535] [INFO ] [LOCAL ] EOT: Starting deletion of primary data. [2025-10-20T18:34:45.121721] [DEBUG] [LOCAL ] EOT: Deleted primary data file. [2025-10-20T18:34:45.121784] [INFO ] [MAIN ] EOT: Starting full primary refresh. [2025-10-20T18:34:45.282821] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:34:45.490482] [INFO ] [REMOTE ] EOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:45.851380] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-01. [2025-10-20T18:34:46.651398] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-03. [2025-10-20T18:34:46.907920] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T18:34:47.181498] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-10-20T18:34:47.451402] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-02. [2025-10-20T18:34:47.514079] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T18:34:48.114631] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T18:34:48.251382] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-04. [2025-10-20T18:34:49.051352] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-06. [2025-10-20T18:34:49.229726] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T18:34:49.851444] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-05. [2025-10-20T18:34:49.986723] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:34:50.651425] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-08. [2025-10-20T18:34:50.853310] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T18:34:51.451462] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-06. [2025-10-20T18:34:52.251414] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-03. [2025-10-20T18:34:52.345228] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T18:34:52.457172] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:34:52.655496] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T18:34:53.051416] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-09. [2025-10-20T18:34:53.851394] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-04. [2025-10-20T18:34:54.651363] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-07. [2025-10-20T18:34:54.681823] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T18:34:54.974234] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T18:34:55.208858] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T18:34:55.451382] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-07. [2025-10-20T18:34:56.251372] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-05. [2025-10-20T18:34:56.553074] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:34:57.011107] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T18:34:57.051432] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-10. [2025-10-20T18:34:57.851376] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-08. [2025-10-20T18:34:57.886487] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:34:57.894460] [INFO ] [REMOTE ] EOG: Full fetch complete (77466 records). [2025-10-20T18:34:57.894621] [DEBUG] [VALIDATOR ] EOG: Validating primary data. [2025-10-20T18:34:58.288627] [DEBUG] [VALIDATOR ] EOG: All expected trading days are present. [2025-10-20T18:34:58.288782] [DEBUG] [VALIDATOR ] EOG: Primary data validation passed. [2025-10-20T18:34:58.290041] [INFO ] [MAIN ] EOG: Fetched primary data is valid. [2025-10-20T18:34:58.290129] [DEBUG] [LOCAL ] EOG: Saving primary data. [2025-10-20T18:34:58.291724] [DEBUG] [LOCAL ] EOG: Trimmed 77466 records to 53391 (keeping 200 days of data) [2025-10-20T18:34:58.633256] [DEBUG] [LOCAL ] EOG: Primary data saved (53391 records). [2025-10-20T18:34:58.633420] [INFO ] [MAIN ] EOG: Full history saved. [2025-10-20T18:34:58.633505] [INFO ] [MAIN ] Update Progress: 744/2367 symbols completed. EOG took 38.4s. Estimated remaining: 3h 27m 2s. Rate: 7.8 symbols/min [2025-10-20T18:34:58.656598] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-06. [2025-10-20T18:34:58.812316] [INFO ] [MAIN ] EPAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:58.838818] [DEBUG] [LOCAL ] EPAC: Loaded 27001 existing records from primary storage. [2025-10-20T18:34:58.839090] [INFO ] [MAIN ] EPAC: Loaded 27001 existing records. [2025-10-20T18:34:58.954688] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:34:59.031526] [WARN ] [MAIN ] EPAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:59.031782] [INFO ] [MAIN ] EPAC: Wiping existing primary data before full refresh. [2025-10-20T18:34:59.031862] [INFO ] [LOCAL ] EPAC: Starting deletion of primary data. [2025-10-20T18:34:59.032123] [DEBUG] [LOCAL ] EPAC: Deleted primary data file. [2025-10-20T18:34:59.032182] [INFO ] [MAIN ] EPAC: Starting full primary refresh. [2025-10-20T18:34:59.335773] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T18:34:59.408222] [INFO ] [REMOTE ] EPAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:59.451339] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-07. [2025-10-20T18:35:00.178545] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:35:00.251379] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-09. [2025-10-20T18:35:01.051375] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-01. [2025-10-20T18:35:01.617044] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T18:35:01.851388] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-08. [2025-10-20T18:35:02.651387] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-10. [2025-10-20T18:35:03.037719] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T18:35:03.117937] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:35:03.451424] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-02. [2025-10-20T18:35:04.220769] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T18:35:04.223116] [INFO ] [REMOTE ] EOI: Full fetch complete (16173 records). [2025-10-20T18:35:04.223185] [DEBUG] [VALIDATOR ] EOI: Validating primary data. [2025-10-20T18:35:04.256500] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-09. [2025-10-20T18:35:04.586843] [DEBUG] [VALIDATOR ] EOI: All expected trading days are present. [2025-10-20T18:35:04.586927] [DEBUG] [VALIDATOR ] EOI: Primary data validation passed. [2025-10-20T18:35:04.587242] [INFO ] [MAIN ] EOI: Fetched primary data is valid. [2025-10-20T18:35:04.587281] [DEBUG] [LOCAL ] EOI: Saving primary data. [2025-10-20T18:35:04.587810] [DEBUG] [LOCAL ] EOI: Trimmed 16173 records to 10601 (keeping 200 days of data) [2025-10-20T18:35:04.650602] [DEBUG] [LOCAL ] EOI: Primary data saved (10601 records). [2025-10-20T18:35:04.650740] [INFO ] [MAIN ] EOI: Full history saved. [2025-10-20T18:35:04.650819] [INFO ] [MAIN ] Update Progress: 745/2367 symbols completed. EOI took 40.1s. Estimated remaining: 3h 26m 50s. Rate: 7.8 symbols/min [2025-10-20T18:35:04.817283] [INFO ] [MAIN ] EPAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:04.858050] [DEBUG] [LOCAL ] EPAM: Loaded 39978 existing records from primary storage. [2025-10-20T18:35:04.858334] [INFO ] [MAIN ] EPAM: Loaded 39978 existing records. [2025-10-20T18:35:05.038304] [WARN ] [MAIN ] EPAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:05.038860] [INFO ] [MAIN ] EPAM: Wiping existing primary data before full refresh. [2025-10-20T18:35:05.039022] [INFO ] [LOCAL ] EPAM: Starting deletion of primary data. [2025-10-20T18:35:05.052070] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-08. [2025-10-20T18:35:05.044416] [DEBUG] [LOCAL ] EPAM: Deleted primary data file. [2025-10-20T18:35:05.052561] [INFO ] [MAIN ] EPAM: Starting full primary refresh. [2025-10-20T18:35:05.113824] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:35:05.160512] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T18:35:05.415755] [INFO ] [REMOTE ] EPAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:05.851348] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-01. [2025-10-20T18:35:05.940381] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T18:35:06.651391] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-10. [2025-10-20T18:35:07.451993] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-03. [2025-10-20T18:35:07.456342] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T18:35:07.654752] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T18:35:07.656504] [INFO ] [REMOTE ] EOT: Full fetch complete (10627 records). [2025-10-20T18:35:07.656695] [DEBUG] [VALIDATOR ] EOT: Validating primary data. [2025-10-20T18:35:07.657087] [FATAL] [VALIDATOR ] EOT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:35:07.657167] [FATAL] [MAIN ] EOT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:35:07.657209] [WARN ] [MAIN ] EOT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:35:07.657272] [INFO ] [LOCAL ] EOT: Starting deletion of primary data. [2025-10-20T18:35:07.657333] [DEBUG] [LOCAL ] EOT: No primary data file found to delete. [2025-10-20T18:35:07.657368] [INFO ] [LOCAL ] EOT: Starting deletion of derivative data. [2025-10-20T18:35:07.657518] [DEBUG] [LOCAL ] EOT: Deleted derivative data file. [2025-10-20T18:35:07.657599] [INFO ] [MAIN ] Update Progress: 746/2367 symbols completed. EOT took 22.9s. Estimated remaining: 3h 26m 33s. Rate: 7.8 symbols/min [2025-10-20T18:35:07.829520] [INFO ] [MAIN ] EPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:07.868690] [DEBUG] [LOCAL ] EPC: Loaded 40316 existing records from primary storage. [2025-10-20T18:35:07.869094] [INFO ] [MAIN ] EPC: Loaded 40316 existing records. [2025-10-20T18:35:08.042314] [WARN ] [MAIN ] EPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:08.042601] [INFO ] [MAIN ] EPC: Wiping existing primary data before full refresh. [2025-10-20T18:35:08.042684] [INFO ] [LOCAL ] EPC: Starting deletion of primary data. [2025-10-20T18:35:08.043156] [DEBUG] [LOCAL ] EPC: Deleted primary data file. [2025-10-20T18:35:08.043214] [INFO ] [MAIN ] EPC: Starting full primary refresh. [2025-10-20T18:35:08.259821] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-02. [2025-10-20T18:35:08.397928] [INFO ] [REMOTE ] EPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:08.714402] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T18:35:09.051365] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-04. [2025-10-20T18:35:09.851397] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-09. [2025-10-20T18:35:09.901556] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T18:35:10.447083] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T18:35:10.651366] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-05. [2025-10-20T18:35:10.726619] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T18:35:11.451338] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-01. [2025-10-20T18:35:11.776273] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T18:35:12.251436] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-03. [2025-10-20T18:35:13.051435] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-10. [2025-10-20T18:35:13.322695] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T18:35:13.851374] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-02. [2025-10-20T18:35:14.005288] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:35:14.251599] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:35:14.254340] [INFO ] [REMOTE ] EOS: Full fetch complete (22914 records). [2025-10-20T18:35:14.254412] [DEBUG] [VALIDATOR ] EOS: Validating primary data. [2025-10-20T18:35:14.620631] [DEBUG] [VALIDATOR ] EOS: All expected trading days are present. [2025-10-20T18:35:14.620738] [DEBUG] [VALIDATOR ] EOS: Primary data validation passed. [2025-10-20T18:35:14.621210] [INFO ] [MAIN ] EOS: Fetched primary data is valid. [2025-10-20T18:35:14.621248] [DEBUG] [LOCAL ] EOS: Saving primary data. [2025-10-20T18:35:14.621871] [DEBUG] [LOCAL ] EOS: Trimmed 22914 records to 14852 (keeping 200 days of data) [2025-10-20T18:35:14.659744] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-06. [2025-10-20T18:35:14.714399] [DEBUG] [LOCAL ] EOS: Primary data saved (14852 records). [2025-10-20T18:35:14.714526] [INFO ] [MAIN ] EOS: Full history saved. [2025-10-20T18:35:14.714604] [INFO ] [MAIN ] Update Progress: 747/2367 symbols completed. EOS took 39.2s. Estimated remaining: 3h 26m 24s. Rate: 7.8 symbols/min [2025-10-20T18:35:14.881744] [INFO ] [MAIN ] EPD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:14.930803] [DEBUG] [LOCAL ] EPD: Loaded 54391 existing records from primary storage. [2025-10-20T18:35:14.931165] [INFO ] [MAIN ] EPD: Loaded 54391 existing records. [2025-10-20T18:35:15.111635] [WARN ] [MAIN ] EPD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:15.112154] [INFO ] [MAIN ] EPD: Wiping existing primary data before full refresh. [2025-10-20T18:35:15.112214] [INFO ] [LOCAL ] EPD: Starting deletion of primary data. [2025-10-20T18:35:15.112542] [DEBUG] [LOCAL ] EPD: Deleted primary data file. [2025-10-20T18:35:15.112581] [INFO ] [MAIN ] EPD: Starting full primary refresh. [2025-10-20T18:35:15.458402] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-04. [2025-10-20T18:35:15.497052] [INFO ] [REMOTE ] EPD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:15.519347] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:35:16.027268] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:35:16.251356] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-03. [2025-10-20T18:35:17.051352] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-01. [2025-10-20T18:35:17.143174] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T18:35:17.548227] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T18:35:17.851333] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-05. [2025-10-20T18:35:18.651363] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-04. [2025-10-20T18:35:18.957583] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T18:35:19.232631] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T18:35:19.451380] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-07. [2025-10-20T18:35:20.142774] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:35:20.251381] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-02. [2025-10-20T18:35:20.795010] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T18:35:21.051365] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-06. [2025-10-20T18:35:21.851386] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-08. [2025-10-20T18:35:21.900662] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:35:22.532282] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:35:22.651339] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-07. [2025-10-20T18:35:22.678055] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T18:35:23.451395] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-09. [2025-10-20T18:35:24.180586] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T18:35:24.251358] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-08. [2025-10-20T18:35:24.488806] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T18:35:25.051427] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-03. [2025-10-20T18:35:25.775000] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T18:35:25.851434] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-09. [2025-10-20T18:35:26.651398] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-05. [2025-10-20T18:35:26.983531] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T18:35:27.307294] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T18:35:27.451402] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-04. [2025-10-20T18:35:27.821813] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:35:28.251420] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-10. [2025-10-20T18:35:29.051587] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-10. [2025-10-20T18:35:29.613243] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T18:35:29.696558] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T18:35:29.702781] [INFO ] [REMOTE ] EPAM: Full fetch complete (56431 records). [2025-10-20T18:35:29.702947] [DEBUG] [VALIDATOR ] EPAM: Validating primary data. [2025-10-20T18:35:29.859144] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-05. [2025-10-20T18:35:30.124049] [DEBUG] [VALIDATOR ] EPAM: All expected trading days are present. [2025-10-20T18:35:30.124192] [DEBUG] [VALIDATOR ] EPAM: Primary data validation passed. [2025-10-20T18:35:30.125383] [INFO ] [MAIN ] EPAM: Fetched primary data is valid. [2025-10-20T18:35:30.125474] [DEBUG] [LOCAL ] EPAM: Saving primary data. [2025-10-20T18:35:30.127784] [DEBUG] [LOCAL ] EPAM: Trimmed 56431 records to 39399 (keeping 200 days of data) [2025-10-20T18:35:30.374787] [DEBUG] [LOCAL ] EPAM: Primary data saved (39399 records). [2025-10-20T18:35:30.375128] [INFO ] [MAIN ] EPAM: Full history saved. [2025-10-20T18:35:30.375723] [INFO ] [MAIN ] Update Progress: 748/2367 symbols completed. EPAM took 25.7s. Estimated remaining: 3h 26m 34s. Rate: 7.8 symbols/min [2025-10-20T18:35:30.514515] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:35:30.552002] [INFO ] [REMOTE ] EPAC: Full fetch complete (38888 records). [2025-10-20T18:35:30.554560] [DEBUG] [VALIDATOR ] EPAC: Validating primary data. [2025-10-20T18:35:30.604873] [INFO ] [MAIN ] EPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:30.654385] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-06. [2025-10-20T18:35:30.974661] [DEBUG] [LOCAL ] EPR: Loaded 44473 existing records from primary storage. [2025-10-20T18:35:30.988499] [DEBUG] [VALIDATOR ] EPAC: All expected trading days are present. [2025-10-20T18:35:30.988943] [INFO ] [MAIN ] EPR: Loaded 44473 existing records. [2025-10-20T18:35:30.988790] [DEBUG] [VALIDATOR ] EPAC: Primary data validation passed. [2025-10-20T18:35:30.994934] [INFO ] [MAIN ] EPAC: Fetched primary data is valid. [2025-10-20T18:35:31.008895] [DEBUG] [LOCAL ] EPAC: Saving primary data. [2025-10-20T18:35:31.093743] [DEBUG] [LOCAL ] EPAC: Trimmed 38888 records to 26516 (keeping 200 days of data) [2025-10-20T18:35:31.296851] [WARN ] [MAIN ] EPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:31.299980] [INFO ] [MAIN ] EPR: Wiping existing primary data before full refresh. [2025-10-20T18:35:31.300035] [INFO ] [LOCAL ] EPR: Starting deletion of primary data. [2025-10-20T18:35:31.315906] [DEBUG] [LOCAL ] EPR: Deleted primary data file. [2025-10-20T18:35:31.327281] [INFO ] [MAIN ] EPR: Starting full primary refresh. [2025-10-20T18:35:31.338275] [DEBUG] [LOCAL ] EPAC: Primary data saved (26516 records). [2025-10-20T18:35:31.371588] [INFO ] [MAIN ] EPAC: Full history saved. [2025-10-20T18:35:31.388757] [INFO ] [MAIN ] Update Progress: 749/2367 symbols completed. EPAC took 32.8s. Estimated remaining: 3h 26m 12s. Rate: 7.8 symbols/min [2025-10-20T18:35:31.731869] [INFO ] [MAIN ] EPRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:31.907109] [INFO ] [REMOTE ] EPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:31.907549] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-01. [2025-10-20T18:35:31.974709] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T18:35:31.986937] [DEBUG] [LOCAL ] EPRT: Loaded 49233 existing records from primary storage. [2025-10-20T18:35:31.988406] [INFO ] [MAIN ] EPRT: Loaded 49233 existing records. [2025-10-20T18:35:32.205854] [WARN ] [MAIN ] EPRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:32.206175] [INFO ] [MAIN ] EPRT: Wiping existing primary data before full refresh. [2025-10-20T18:35:32.206240] [INFO ] [LOCAL ] EPRT: Starting deletion of primary data. [2025-10-20T18:35:32.215205] [DEBUG] [LOCAL ] EPRT: Deleted primary data file. [2025-10-20T18:35:32.230371] [INFO ] [MAIN ] EPRT: Starting full primary refresh. [2025-10-20T18:35:32.258162] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-06. [2025-10-20T18:35:32.272279] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:35:32.616412] [INFO ] [REMOTE ] EPRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:33.051417] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-07. [2025-10-20T18:35:33.851362] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-01. [2025-10-20T18:35:34.132986] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T18:35:34.346804] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-01 (2.44 seconds). [2025-10-20T18:35:34.651433] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-07. [2025-10-20T18:35:34.951948] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T18:35:35.451507] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-02. [2025-10-20T18:35:35.682857] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:35:36.251399] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-02. [2025-10-20T18:35:36.665336] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T18:35:37.000294] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T18:35:37.051439] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-03. [2025-10-20T18:35:37.793366] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:35:37.851356] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-03. [2025-10-20T18:35:38.380964] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T18:35:38.651422] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-08. [2025-10-20T18:35:39.236689] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T18:35:39.451406] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-08. [2025-10-20T18:35:40.251479] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-04. [2025-10-20T18:35:40.281201] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T18:35:41.051413] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-09. [2025-10-20T18:35:41.422319] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T18:35:41.851421] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-09. [2025-10-20T18:35:41.878547] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:35:42.565896] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:35:42.651446] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-04. [2025-10-20T18:35:43.451367] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-10. [2025-10-20T18:35:43.844004] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T18:35:44.251421] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-10. [2025-10-20T18:35:44.336040] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T18:35:44.927235] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T18:35:44.937144] [INFO ] [REMOTE ] EPC: Full fetch complete (57300 records). [2025-10-20T18:35:44.937342] [DEBUG] [VALIDATOR ] EPC: Validating primary data. [2025-10-20T18:35:45.056586] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-05. [2025-10-20T18:35:45.347288] [DEBUG] [VALIDATOR ] EPC: All expected trading days are present. [2025-10-20T18:35:45.347372] [DEBUG] [VALIDATOR ] EPC: Primary data validation passed. [2025-10-20T18:35:45.348152] [INFO ] [MAIN ] EPC: Fetched primary data is valid. [2025-10-20T18:35:45.348214] [DEBUG] [LOCAL ] EPC: Saving primary data. [2025-10-20T18:35:45.349318] [DEBUG] [LOCAL ] EPC: Trimmed 57300 records to 39805 (keeping 200 days of data) [2025-10-20T18:35:45.592418] [DEBUG] [LOCAL ] EPC: Primary data saved (39805 records). [2025-10-20T18:35:45.592599] [INFO ] [MAIN ] EPC: Full history saved. [2025-10-20T18:35:45.592762] [INFO ] [MAIN ] Update Progress: 750/2367 symbols completed. EPC took 37.9s. Estimated remaining: 3h 26m 18s. Rate: 7.8 symbols/min [2025-10-20T18:35:45.776952] [INFO ] [MAIN ] EQBK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:45.788990] [DEBUG] [LOCAL ] EQBK: Loaded 11534 existing records from primary storage. [2025-10-20T18:35:45.789250] [INFO ] [MAIN ] EQBK: Loaded 11534 existing records. [2025-10-20T18:35:45.860339] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-05. [2025-10-20T18:35:45.976572] [WARN ] [MAIN ] EQBK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:45.976899] [INFO ] [MAIN ] EQBK: Wiping existing primary data before full refresh. [2025-10-20T18:35:45.976978] [INFO ] [LOCAL ] EQBK: Starting deletion of primary data. [2025-10-20T18:35:45.977181] [DEBUG] [LOCAL ] EQBK: Deleted primary data file. [2025-10-20T18:35:45.977257] [INFO ] [MAIN ] EQBK: Starting full primary refresh. [2025-10-20T18:35:46.340126] [INFO ] [REMOTE ] EQBK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:46.472385] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-10 (2.22 seconds). [2025-10-20T18:35:46.480358] [INFO ] [REMOTE ] EPD: Full fetch complete (77780 records). [2025-10-20T18:35:46.480499] [DEBUG] [VALIDATOR ] EPD: Validating primary data. [2025-10-20T18:35:46.658384] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-01. [2025-10-20T18:35:46.893529] [DEBUG] [VALIDATOR ] EPD: All expected trading days are present. [2025-10-20T18:35:46.893682] [DEBUG] [VALIDATOR ] EPD: Primary data validation passed. [2025-10-20T18:35:46.894668] [INFO ] [MAIN ] EPD: Fetched primary data is valid. [2025-10-20T18:35:46.894893] [DEBUG] [LOCAL ] EPD: Saving primary data. [2025-10-20T18:35:46.922617] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T18:35:46.932675] [DEBUG] [LOCAL ] EPD: Trimmed 77780 records to 53612 (keeping 200 days of data) [2025-10-20T18:35:47.256447] [DEBUG] [LOCAL ] EPD: Primary data saved (53612 records). [2025-10-20T18:35:47.256611] [INFO ] [MAIN ] EPD: Full history saved. [2025-10-20T18:35:47.256760] [INFO ] [MAIN ] Update Progress: 751/2367 symbols completed. EPD took 32.5s. Estimated remaining: 3h 25m 57s. Rate: 7.8 symbols/min [2025-10-20T18:35:47.431453] [INFO ] [MAIN ] EQH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:47.457834] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-06. [2025-10-20T18:35:47.481238] [DEBUG] [LOCAL ] EQH: Loaded 53475 existing records from primary storage. [2025-10-20T18:35:47.481572] [INFO ] [MAIN ] EQH: Loaded 53475 existing records. [2025-10-20T18:35:47.661403] [WARN ] [MAIN ] EQH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:47.662058] [INFO ] [MAIN ] EQH: Wiping existing primary data before full refresh. [2025-10-20T18:35:47.662319] [INFO ] [LOCAL ] EQH: Starting deletion of primary data. [2025-10-20T18:35:47.662998] [DEBUG] [LOCAL ] EQH: Deleted primary data file. [2025-10-20T18:35:47.663062] [INFO ] [MAIN ] EQH: Starting full primary refresh. [2025-10-20T18:35:47.800364] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T18:35:47.818248] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:35:48.088201] [INFO ] [REMOTE ] EQH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:48.251383] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-06. [2025-10-20T18:35:48.854740] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:35:49.051381] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-07. [2025-10-20T18:35:49.852706] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-02. [2025-10-20T18:35:49.854950] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T18:35:50.435420] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T18:35:50.651412] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-01. [2025-10-20T18:35:50.683164] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T18:35:51.451393] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-07. [2025-10-20T18:35:52.251391] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-03. [2025-10-20T18:35:52.597608] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T18:35:52.682313] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-10-20T18:35:53.051360] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-04. [2025-10-20T18:35:53.209401] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T18:35:53.851394] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-02. [2025-10-20T18:35:54.116462] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:35:54.651377] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-08. [2025-10-20T18:35:55.451350] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-05. [2025-10-20T18:35:55.503243] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:35:56.005869] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T18:35:56.053814] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T18:35:56.251399] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-08. [2025-10-20T18:35:57.051388] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-09. [2025-10-20T18:35:57.851386] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-06. [2025-10-20T18:35:57.967154] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:35:58.463733] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T18:35:58.651384] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-09. [2025-10-20T18:35:58.733378] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:35:59.451389] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-07. [2025-10-20T18:36:00.043476] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T18:36:00.251373] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-10. [2025-10-20T18:36:00.389086] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T18:36:01.051382] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-10. [2025-10-20T18:36:01.851414] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-03. [2025-10-20T18:36:02.108952] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T18:36:02.116324] [INFO ] [REMOTE ] EPRT: Full fetch complete (69434 records). [2025-10-20T18:36:02.116435] [DEBUG] [VALIDATOR ] EPRT: Validating primary data. [2025-10-20T18:36:02.511205] [DEBUG] [VALIDATOR ] EPRT: All expected trading days are present. [2025-10-20T18:36:02.511292] [DEBUG] [VALIDATOR ] EPRT: Primary data validation passed. [2025-10-20T18:36:02.512124] [INFO ] [MAIN ] EPRT: Fetched primary data is valid. [2025-10-20T18:36:02.512190] [DEBUG] [LOCAL ] EPRT: Saving primary data. [2025-10-20T18:36:02.513509] [DEBUG] [LOCAL ] EPRT: Trimmed 69434 records to 48488 (keeping 200 days of data) [2025-10-20T18:36:02.693074] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-08. [2025-10-20T18:36:02.819010] [DEBUG] [LOCAL ] EPRT: Primary data saved (48488 records). [2025-10-20T18:36:02.819182] [INFO ] [MAIN ] EPRT: Full history saved. [2025-10-20T18:36:02.819285] [INFO ] [MAIN ] Update Progress: 752/2367 symbols completed. EPRT took 31.4s. Estimated remaining: 3h 26m 7s. Rate: 7.8 symbols/min [2025-10-20T18:36:02.995559] [INFO ] [MAIN ] EQNR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:03.045509] [DEBUG] [LOCAL ] EQNR: Loaded 53821 existing records from primary storage. [2025-10-20T18:36:03.045883] [INFO ] [MAIN ] EQNR: Loaded 53821 existing records. [2025-10-20T18:36:03.254671] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T18:36:03.252093] [WARN ] [MAIN ] EQNR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:36:03.262842] [INFO ] [MAIN ] EQNR: Wiping existing primary data before full refresh. [2025-10-20T18:36:03.263140] [INFO ] [LOCAL ] EQNR: Starting deletion of primary data. [2025-10-20T18:36:03.263585] [INFO ] [REMOTE ] EPR: Full fetch complete (63249 records). [2025-10-20T18:36:03.263831] [DEBUG] [VALIDATOR ] EPR: Validating primary data. [2025-10-20T18:36:03.263912] [DEBUG] [LOCAL ] EQNR: Deleted primary data file. [2025-10-20T18:36:03.264218] [INFO ] [MAIN ] EQNR: Starting full primary refresh. [2025-10-20T18:36:03.876394] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T18:36:03.945450] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-09. [2025-10-20T18:36:04.075133] [INFO ] [REMOTE ] EQNR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:04.071194] [DEBUG] [VALIDATOR ] EPR: All expected trading days are present. [2025-10-20T18:36:04.075711] [DEBUG] [VALIDATOR ] EPR: Primary data validation passed. [2025-10-20T18:36:04.076610] [INFO ] [MAIN ] EPR: Fetched primary data is valid. [2025-10-20T18:36:04.076710] [DEBUG] [LOCAL ] EPR: Saving primary data. [2025-10-20T18:36:04.077929] [DEBUG] [LOCAL ] EPR: Trimmed 63249 records to 43859 (keeping 200 days of data) [2025-10-20T18:36:04.227440] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-10-20T18:36:04.300752] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-04. [2025-10-20T18:36:04.401155] [DEBUG] [LOCAL ] EPR: Primary data saved (43859 records). [2025-10-20T18:36:04.401289] [INFO ] [MAIN ] EPR: Full history saved. [2025-10-20T18:36:04.401364] [INFO ] [MAIN ] Update Progress: 753/2367 symbols completed. EPR took 34.0s. Estimated remaining: 3h 25m 46s. Rate: 7.8 symbols/min [2025-10-20T18:36:04.571835] [INFO ] [MAIN ] EQR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:04.617438] [DEBUG] [LOCAL ] EQR: Loaded 52498 existing records from primary storage. [2025-10-20T18:36:04.617831] [INFO ] [MAIN ] EQR: Loaded 52498 existing records. [2025-10-20T18:36:04.802669] [WARN ] [MAIN ] EQR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:36:04.803861] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T18:36:04.804088] [INFO ] [MAIN ] EQR: Wiping existing primary data before full refresh. [2025-10-20T18:36:04.807269] [INFO ] [LOCAL ] EQR: Starting deletion of primary data. [2025-10-20T18:36:04.807937] [DEBUG] [LOCAL ] EQR: Deleted primary data file. [2025-10-20T18:36:04.807994] [INFO ] [MAIN ] EQR: Starting full primary refresh. [2025-10-20T18:36:05.056545] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-01. [2025-10-20T18:36:05.157088] [INFO ] [REMOTE ] EQR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:05.784574] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:36:05.851345] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-01. [2025-10-20T18:36:06.651375] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-10. [2025-10-20T18:36:06.865309] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T18:36:07.451354] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-02. [2025-10-20T18:36:07.635569] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T18:36:07.642691] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:36:07.641513] [INFO ] [REMOTE ] EQBK: Full fetch complete (15732 records). [2025-10-20T18:36:07.653813] [DEBUG] [VALIDATOR ] EQBK: Validating primary data. [2025-10-20T18:36:08.023809] [DEBUG] [VALIDATOR ] EQBK: All expected trading days are present. [2025-10-20T18:36:08.023886] [DEBUG] [VALIDATOR ] EQBK: Primary data validation passed. [2025-10-20T18:36:08.024251] [INFO ] [MAIN ] EQBK: Fetched primary data is valid. [2025-10-20T18:36:08.024291] [DEBUG] [LOCAL ] EQBK: Saving primary data. [2025-10-20T18:36:08.024890] [DEBUG] [LOCAL ] EQBK: Trimmed 15732 records to 11425 (keeping 200 days of data) [2025-10-20T18:36:08.093314] [DEBUG] [LOCAL ] EQBK: Primary data saved (11425 records). [2025-10-20T18:36:08.093451] [INFO ] [MAIN ] EQBK: Full history saved. [2025-10-20T18:36:08.093530] [INFO ] [MAIN ] Update Progress: 754/2367 symbols completed. EQBK took 22.5s. Estimated remaining: 3h 25m 30s. Rate: 7.8 symbols/min [2025-10-20T18:36:08.252138] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-02. [2025-10-20T18:36:08.272590] [INFO ] [MAIN ] EQS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:08.272854] [DEBUG] [LOCAL ] EQS: No existing data file found in primary storage. [2025-10-20T18:36:08.272901] [WARN ] [MAIN ] EQS: No existing data found, performing full refresh instead. [2025-10-20T18:36:08.273006] [INFO ] [MAIN ] EQS: Wiping existing primary data before full refresh. [2025-10-20T18:36:08.273047] [INFO ] [LOCAL ] EQS: Starting deletion of primary data. [2025-10-20T18:36:08.273089] [DEBUG] [LOCAL ] EQS: No primary data file found to delete. [2025-10-20T18:36:08.273159] [INFO ] [MAIN ] EQS: Starting full primary refresh. [2025-10-20T18:36:08.636973] [INFO ] [REMOTE ] EQS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:09.034868] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:36:09.051343] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-05. [2025-10-20T18:36:09.851378] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-01. [2025-10-20T18:36:10.053685] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T18:36:10.578926] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T18:36:10.651382] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-03. [2025-10-20T18:36:10.777357] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T18:36:11.451379] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-02. [2025-10-20T18:36:11.818402] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T18:36:12.251373] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-03. [2025-10-20T18:36:12.644363] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T18:36:13.051364] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-06. [2025-10-20T18:36:13.851366] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-03. [2025-10-20T18:36:13.886921] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T18:36:14.162445] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-10-20T18:36:14.655379] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-04. [2025-10-20T18:36:14.660893] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:36:15.451372] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-04. [2025-10-20T18:36:15.731689] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T18:36:16.251368] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-04. [2025-10-20T18:36:16.675606] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T18:36:17.051373] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-07. [2025-10-20T18:36:17.851400] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-05. [2025-10-20T18:36:18.108480] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T18:36:18.221621] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-10-20T18:36:18.651388] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-05. [2025-10-20T18:36:18.824068] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T18:36:19.451388] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-06. [2025-10-20T18:36:19.828863] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-10-20T18:36:20.251424] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-08. [2025-10-20T18:36:20.694142] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T18:36:21.051437] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-06. [2025-10-20T18:36:21.851384] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-07. [2025-10-20T18:36:22.031982] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:36:22.430763] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:36:22.445388] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T18:36:22.651383] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-07. [2025-10-20T18:36:23.451446] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-09. [2025-10-20T18:36:24.258693] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-05. [2025-10-20T18:36:24.295351] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:36:25.051351] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-08. [2025-10-20T18:36:25.169564] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:36:25.575908] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T18:36:25.851388] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-08. [2025-10-20T18:36:25.994853] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:36:26.651435] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-10. [2025-10-20T18:36:27.451397] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-09. [2025-10-20T18:36:27.739503] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T18:36:27.956533] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T18:36:27.965074] [INFO ] [REMOTE ] EQH: Full fetch complete (76265 records). [2025-10-20T18:36:27.965309] [DEBUG] [VALIDATOR ] EQH: Validating primary data. [2025-10-20T18:36:27.966927] [FATAL] [VALIDATOR ] EQH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:36:27.967213] [FATAL] [MAIN ] EQH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:36:27.967537] [WARN ] [MAIN ] EQH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:36:27.967597] [INFO ] [LOCAL ] EQH: Starting deletion of primary data. [2025-10-20T18:36:27.967700] [DEBUG] [LOCAL ] EQH: No primary data file found to delete. [2025-10-20T18:36:27.967759] [INFO ] [LOCAL ] EQH: Starting deletion of derivative data. [2025-10-20T18:36:27.968001] [DEBUG] [LOCAL ] EQH: Deleted derivative data file. [2025-10-20T18:36:27.968110] [INFO ] [MAIN ] Update Progress: 755/2367 symbols completed. EQH took 40.7s. Estimated remaining: 3h 25m 48s. Rate: 7.8 symbols/min [2025-10-20T18:36:28.160339] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T18:36:28.165130] [INFO ] [MAIN ] EQT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:28.224927] [DEBUG] [LOCAL ] EQT: Loaded 54465 existing records from primary storage. [2025-10-20T18:36:28.225364] [INFO ] [MAIN ] EQT: Loaded 54465 existing records. [2025-10-20T18:36:28.257322] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-06. [2025-10-20T18:36:28.427769] [WARN ] [MAIN ] EQT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:36:28.428008] [INFO ] [MAIN ] EQT: Wiping existing primary data before full refresh. [2025-10-20T18:36:28.428058] [INFO ] [LOCAL ] EQT: Starting deletion of primary data. [2025-10-20T18:36:28.428359] [DEBUG] [LOCAL ] EQT: Deleted primary data file. [2025-10-20T18:36:28.428400] [INFO ] [MAIN ] EQT: Starting full primary refresh. [2025-10-20T18:36:28.775546] [INFO ] [REMOTE ] EQT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:29.051425] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-09. [2025-10-20T18:36:29.851366] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-10. [2025-10-20T18:36:29.959605] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T18:36:30.275703] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T18:36:30.276711] [INFO ] [REMOTE ] EQS: Full fetch complete (2387 records). [2025-10-20T18:36:30.276781] [DEBUG] [VALIDATOR ] EQS: Validating primary data. [2025-10-20T18:36:30.475325] [WARN ] [VALIDATOR ] EQS: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-10-20T18:36:30.475670] [FATAL] [MAIN ] EQS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:36:30.475756] [WARN ] [MAIN ] EQS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:36:30.475845] [INFO ] [LOCAL ] EQS: Starting deletion of primary data. [2025-10-20T18:36:30.475940] [DEBUG] [LOCAL ] EQS: No primary data file found to delete. [2025-10-20T18:36:30.476010] [INFO ] [LOCAL ] EQS: Starting deletion of derivative data. [2025-10-20T18:36:30.476123] [DEBUG] [LOCAL ] EQS: No derivative data file found to delete. [2025-10-20T18:36:30.476237] [INFO ] [MAIN ] Update Progress: 756/2367 symbols completed. EQS took 22.4s. Estimated remaining: 3h 25m 30s. Rate: 7.8 symbols/min [2025-10-20T18:36:30.662059] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-01. [2025-10-20T18:36:30.666255] [INFO ] [MAIN ] EQV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:30.667765] [DEBUG] [LOCAL ] EQV: No existing data file found in primary storage. [2025-10-20T18:36:30.667821] [WARN ] [MAIN ] EQV: No existing data found, performing full refresh instead. [2025-10-20T18:36:30.667943] [INFO ] [MAIN ] EQV: Wiping existing primary data before full refresh. [2025-10-20T18:36:30.667986] [INFO ] [LOCAL ] EQV: Starting deletion of primary data. [2025-10-20T18:36:30.668028] [DEBUG] [LOCAL ] EQV: No primary data file found to delete. [2025-10-20T18:36:30.668059] [INFO ] [MAIN ] EQV: Starting full primary refresh. [2025-10-20T18:36:31.031334] [INFO ] [REMOTE ] EQV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:31.062257] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T18:36:31.451440] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-10. [2025-10-20T18:36:32.251434] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-01. [2025-10-20T18:36:33.003199] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T18:36:33.054347] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-07. [2025-10-20T18:36:33.060273] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-01 (2.40 seconds). [2025-10-20T18:36:33.182708] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:36:33.190410] [INFO ] [REMOTE ] EQNR: Full fetch complete (77011 records). [2025-10-20T18:36:33.190537] [DEBUG] [VALIDATOR ] EQNR: Validating primary data. [2025-10-20T18:36:33.588450] [DEBUG] [VALIDATOR ] EQNR: All expected trading days are present. [2025-10-20T18:36:33.588557] [DEBUG] [VALIDATOR ] EQNR: Primary data validation passed. [2025-10-20T18:36:33.589487] [INFO ] [MAIN ] EQNR: Fetched primary data is valid. [2025-10-20T18:36:33.589550] [DEBUG] [LOCAL ] EQNR: Saving primary data. [2025-10-20T18:36:33.590994] [DEBUG] [LOCAL ] EQNR: Trimmed 77011 records to 53058 (keeping 200 days of data) [2025-10-20T18:36:33.872398] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-02. [2025-10-20T18:36:33.918536] [DEBUG] [LOCAL ] EQNR: Primary data saved (53058 records). [2025-10-20T18:36:33.919200] [INFO ] [MAIN ] EQNR: Full history saved. [2025-10-20T18:36:33.919469] [INFO ] [MAIN ] Update Progress: 757/2367 symbols completed. EQNR took 31.1s. Estimated remaining: 3h 25m 13s. Rate: 7.8 symbols/min [2025-10-20T18:36:34.470159] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T18:36:34.506037] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T18:36:34.651335] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-08. [2025-10-20T18:36:35.451439] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-03. [2025-10-20T18:36:35.973600] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T18:36:36.256939] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-02. [2025-10-20T18:36:36.262497] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T18:36:37.051446] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-04. [2025-10-20T18:36:37.535592] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-10-20T18:36:37.851350] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-05. [2025-10-20T18:36:38.003305] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:36:38.260382] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T18:36:38.651426] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-06. [2025-10-20T18:36:39.218161] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T18:36:39.451360] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-03. [2025-10-20T18:36:40.251435] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-09. [2025-10-20T18:36:41.051368] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-07. [2025-10-20T18:36:41.607746] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-03 (2.16 seconds). [2025-10-20T18:36:41.851350] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-04. [2025-10-20T18:36:42.104507] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T18:36:42.651387] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-10. [2025-10-20T18:36:47.162867] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-07 (6.11 seconds). [2025-10-20T18:36:47.164336] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-08. [2025-10-20T18:36:49.657251] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-08 (2.49 seconds). [2025-10-20T18:36:49.659590] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-09. [2025-10-20T18:36:50.497109] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T18:36:50.498785] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-10. [2025-10-20T18:36:51.556944] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:36:51.558085] [INFO ] [REMOTE ] EQV: Full fetch complete (5061 records). [2025-10-20T18:36:51.558186] [DEBUG] [VALIDATOR ] EQV: Validating primary data. [2025-10-20T18:36:51.968895] [DEBUG] [VALIDATOR ] EQV: All expected trading days are present. [2025-10-20T18:36:51.969002] [DEBUG] [VALIDATOR ] EQV: Primary data validation passed. [2025-10-20T18:36:51.969367] [INFO ] [MAIN ] EQV: Fetched primary data is valid. [2025-10-20T18:36:51.969447] [DEBUG] [LOCAL ] EQV: Saving primary data. [2025-10-20T18:36:51.969916] [DEBUG] [LOCAL ] EQV: Trimmed 5061 records to 2745 (keeping 200 days of data) [2025-10-20T18:36:51.987136] [DEBUG] [LOCAL ] EQV: Primary data saved (2745 records). [2025-10-20T18:36:51.987242] [INFO ] [MAIN ] EQV: Full history saved. [2025-10-20T18:36:51.987325] [INFO ] [MAIN ] Update Progress: 758/2367 symbols completed. EQV took 21.5s. Estimated remaining: 3h 25m 28s. Rate: 7.8 symbols/min [2025-10-20T18:36:52.461453] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-10 (9.81 seconds). [2025-10-20T18:36:52.470307] [INFO ] [REMOTE ] EQR: Full fetch complete (74535 records). [2025-10-20T18:36:52.470475] [DEBUG] [VALIDATOR ] EQR: Validating primary data. [2025-10-20T18:36:52.647669] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-04 (10.80 seconds). [2025-10-20T18:36:52.695414] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-05. [2025-10-20T18:36:52.960263] [DEBUG] [VALIDATOR ] EQR: All expected trading days are present. [2025-10-20T18:36:52.960377] [DEBUG] [VALIDATOR ] EQR: Primary data validation passed. [2025-10-20T18:36:52.961354] [INFO ] [MAIN ] EQR: Fetched primary data is valid. [2025-10-20T18:36:52.961419] [DEBUG] [LOCAL ] EQR: Saving primary data. [2025-10-20T18:36:52.962923] [DEBUG] [LOCAL ] EQR: Trimmed 74535 records to 51795 (keeping 200 days of data) [2025-10-20T18:36:53.276771] [DEBUG] [LOCAL ] EQR: Primary data saved (51795 records). [2025-10-20T18:36:53.276909] [INFO ] [MAIN ] EQR: Full history saved. [2025-10-20T18:36:53.277432] [INFO ] [MAIN ] Update Progress: 759/2367 symbols completed. EQR took 48.9s. Estimated remaining: 3h 25m 7s. Rate: 7.8 symbols/min [2025-10-20T18:36:55.322152] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-05 (2.63 seconds). [2025-10-20T18:36:55.334934] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-06. [2025-10-20T18:36:58.203331] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-06 (2.87 seconds). [2025-10-20T18:36:58.214260] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-07. [2025-10-20T18:37:00.313908] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-10-20T18:37:00.327337] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-08. [2025-10-20T18:37:02.705345] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-08 (2.38 seconds). [2025-10-20T18:37:02.716905] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-09. [2025-10-20T18:37:04.890196] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T18:37:04.903110] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-10. [2025-10-20T18:37:07.253687] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-10 (2.35 seconds). [2025-10-20T18:37:07.261590] [INFO ] [REMOTE ] EQT: Full fetch complete (77860 records). [2025-10-20T18:37:07.261757] [DEBUG] [VALIDATOR ] EQT: Validating primary data. [2025-10-20T18:37:07.654793] [DEBUG] [VALIDATOR ] EQT: All expected trading days are present. [2025-10-20T18:37:07.654874] [DEBUG] [VALIDATOR ] EQT: Primary data validation passed. [2025-10-20T18:37:07.655776] [INFO ] [MAIN ] EQT: Fetched primary data is valid. [2025-10-20T18:37:07.655839] [DEBUG] [LOCAL ] EQT: Saving primary data. [2025-10-20T18:37:07.657296] [DEBUG] [LOCAL ] EQT: Trimmed 77860 records to 53685 (keeping 200 days of data) [2025-10-20T18:37:07.972444] [DEBUG] [LOCAL ] EQT: Primary data saved (53685 records). [2025-10-20T18:37:07.972571] [INFO ] [MAIN ] EQT: Full history saved. [2025-10-20T18:37:07.972669] [INFO ] [MAIN ] Update Progress: 760/2367 symbols completed. EQT took 40.0s. Estimated remaining: 3h 25m 14s. Rate: 7.8 symbols/min [2025-10-20T18:37:08.508480] [INFO ] [MAIN ] ES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:08.678458] [INFO ] [MAIN ] ESAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:08.682756] [INFO ] [MAIN ] ERJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:08.632289] [INFO ] [MAIN ] ERO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:08.722073] [DEBUG] [LOCAL ] ESAB: Loaded 26444 existing records from primary storage. [2025-10-20T18:37:08.722379] [INFO ] [MAIN ] ESAB: Loaded 26444 existing records. [2025-10-20T18:37:08.861877] [DEBUG] [LOCAL ] ERO: Loaded 42618 existing records from primary storage. [2025-10-20T18:37:08.873854] [INFO ] [MAIN ] ERO: Loaded 42618 existing records. [2025-10-20T18:37:09.073052] [DEBUG] [LOCAL ] ES: Loaded 53462 existing records from primary storage. [2025-10-20T18:37:09.078695] [WARN ] [MAIN ] ESAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:09.084977] [DEBUG] [LOCAL ] ERJ: Loaded 51052 existing records from primary storage. [2025-10-20T18:37:09.095780] [INFO ] [MAIN ] ES: Loaded 53462 existing records. [2025-10-20T18:37:09.085223] [INFO ] [MAIN ] ESAB: Wiping existing primary data before full refresh. [2025-10-20T18:37:09.110043] [INFO ] [MAIN ] ERJ: Loaded 51052 existing records. [2025-10-20T18:37:09.114784] [INFO ] [LOCAL ] ESAB: Starting deletion of primary data. [2025-10-20T18:37:09.226850] [DEBUG] [LOCAL ] ESAB: Deleted primary data file. [2025-10-20T18:37:09.292670] [INFO ] [MAIN ] ESAB: Starting full primary refresh. [2025-10-20T18:37:09.435032] [WARN ] [MAIN ] ERO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:09.466916] [INFO ] [MAIN ] ERO: Wiping existing primary data before full refresh. [2025-10-20T18:37:09.492824] [INFO ] [LOCAL ] ERO: Starting deletion of primary data. [2025-10-20T18:37:09.615698] [DEBUG] [LOCAL ] ERO: Deleted primary data file. [2025-10-20T18:37:09.641239] [WARN ] [MAIN ] ERJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:09.649893] [INFO ] [MAIN ] ERO: Starting full primary refresh. [2025-10-20T18:37:09.650220] [INFO ] [MAIN ] ERJ: Wiping existing primary data before full refresh. [2025-10-20T18:37:09.662588] [INFO ] [LOCAL ] ERJ: Starting deletion of primary data. [2025-10-20T18:37:09.718154] [DEBUG] [LOCAL ] ERJ: Deleted primary data file. [2025-10-20T18:37:09.732782] [WARN ] [MAIN ] ES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:09.718353] [INFO ] [MAIN ] ERJ: Starting full primary refresh. [2025-10-20T18:37:09.754111] [INFO ] [MAIN ] ES: Wiping existing primary data before full refresh. [2025-10-20T18:37:09.764704] [INFO ] [LOCAL ] ES: Starting deletion of primary data. [2025-10-20T18:37:09.880335] [DEBUG] [LOCAL ] ES: Deleted primary data file. [2025-10-20T18:37:09.914199] [INFO ] [MAIN ] ES: Starting full primary refresh. [2025-10-20T18:37:10.541050] [INFO ] [REMOTE ] ESAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:10.618590] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-01. [2025-10-20T18:37:11.037213] [INFO ] [REMOTE ] ERO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:11.061178] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-01. [2025-10-20T18:37:11.115390] [INFO ] [REMOTE ] ERJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:11.129543] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-01. [2025-10-20T18:37:11.157392] [INFO ] [REMOTE ] ES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:11.157823] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-01. [2025-10-20T18:37:11.952515] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T18:37:11.957192] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-02. [2025-10-20T18:37:12.856433] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:37:12.868166] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-02. [2025-10-20T18:37:13.025824] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T18:37:13.036445] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-02. [2025-10-20T18:37:13.078995] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T18:37:13.085420] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-03. [2025-10-20T18:37:13.126983] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-01 (2.07 seconds). [2025-10-20T18:37:13.136050] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-02. [2025-10-20T18:37:14.231115] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T18:37:14.237140] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-04. [2025-10-20T18:37:14.401541] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:37:14.411733] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-03. [2025-10-20T18:37:14.479365] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T18:37:14.487761] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-03. [2025-10-20T18:37:14.772450] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T18:37:14.781992] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-03. [2025-10-20T18:37:15.520132] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-10-20T18:37:15.526770] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-05. [2025-10-20T18:37:15.872426] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:37:15.882626] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-04. [2025-10-20T18:37:16.221168] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T18:37:16.232553] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-04. [2025-10-20T18:37:16.302533] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:37:16.313362] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-04. [2025-10-20T18:37:16.362894] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T18:37:16.368362] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-06. [2025-10-20T18:37:17.025056] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-10-20T18:37:17.036048] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-05. [2025-10-20T18:37:17.320006] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T18:37:17.325460] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-07. [2025-10-20T18:37:18.231565] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T18:37:18.240920] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-06. [2025-10-20T18:37:18.459381] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T18:37:18.470936] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-05. [2025-10-20T18:37:18.546906] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T18:37:18.558728] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-05. [2025-10-20T18:37:18.609190] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:37:18.614850] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-08. [2025-10-20T18:37:19.386249] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T18:37:19.394538] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-07. [2025-10-20T18:37:19.880763] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:37:19.890171] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-09. [2025-10-20T18:37:20.279063] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T18:37:20.290950] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-06. [2025-10-20T18:37:20.322553] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:37:20.333967] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-06. [2025-10-20T18:37:20.647215] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T18:37:20.662037] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-08. [2025-10-20T18:37:21.181263] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T18:37:21.188236] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-10. [2025-10-20T18:37:21.556564] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T18:37:21.566894] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-07. [2025-10-20T18:37:22.075247] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T18:37:22.086459] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-07. [2025-10-20T18:37:22.110685] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:37:22.119771] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-09. [2025-10-20T18:37:22.491941] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T18:37:22.496239] [INFO ] [REMOTE ] ESAB: Full fetch complete (36350 records). [2025-10-20T18:37:22.496346] [DEBUG] [VALIDATOR ] ESAB: Validating primary data. [2025-10-20T18:37:22.872906] [DEBUG] [VALIDATOR ] ESAB: All expected trading days are present. [2025-10-20T18:37:22.872984] [DEBUG] [VALIDATOR ] ESAB: Primary data validation passed. [2025-10-20T18:37:22.873551] [INFO ] [MAIN ] ESAB: Fetched primary data is valid. [2025-10-20T18:37:22.873590] [DEBUG] [LOCAL ] ESAB: Saving primary data. [2025-10-20T18:37:22.874436] [DEBUG] [LOCAL ] ESAB: Trimmed 36350 records to 26063 (keeping 200 days of data) [2025-10-20T18:37:23.034332] [DEBUG] [LOCAL ] ESAB: Primary data saved (26063 records). [2025-10-20T18:37:23.034461] [INFO ] [MAIN ] ESAB: Full history saved. [2025-10-20T18:37:23.034540] [INFO ] [MAIN ] Update Progress: 761/2367 symbols completed. ESAB took 15.1s. Estimated remaining: 3h 25m 22s. Rate: 7.8 symbols/min [2025-10-20T18:37:23.207113] [INFO ] [MAIN ] ESE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:23.244185] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T18:37:23.262111] [DEBUG] [LOCAL ] ESE: Loaded 16777 existing records from primary storage. [2025-10-20T18:37:23.261775] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-08. [2025-10-20T18:37:23.263106] [INFO ] [MAIN ] ESE: Loaded 16777 existing records. [2025-10-20T18:37:23.448857] [WARN ] [MAIN ] ESE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:23.449176] [INFO ] [MAIN ] ESE: Wiping existing primary data before full refresh. [2025-10-20T18:37:23.449257] [INFO ] [LOCAL ] ESE: Starting deletion of primary data. [2025-10-20T18:37:23.449461] [DEBUG] [LOCAL ] ESE: Deleted primary data file. [2025-10-20T18:37:23.449516] [INFO ] [MAIN ] ESE: Starting full primary refresh. [2025-10-20T18:37:23.823332] [INFO ] [REMOTE ] ESE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:23.823971] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-01. [2025-10-20T18:37:23.902416] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T18:37:23.912535] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-10. [2025-10-20T18:37:24.029511] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T18:37:24.040898] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-08. [2025-10-20T18:37:24.749457] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T18:37:24.753189] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-02. [2025-10-20T18:37:25.003207] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:37:25.014345] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-09. [2025-10-20T18:37:25.374116] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:37:25.381965] [INFO ] [REMOTE ] ERO: Full fetch complete (60563 records). [2025-10-20T18:37:25.382158] [DEBUG] [VALIDATOR ] ERO: Validating primary data. [2025-10-20T18:37:25.707388] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T18:37:25.749568] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-03. [2025-10-20T18:37:25.808619] [DEBUG] [VALIDATOR ] ERO: All expected trading days are present. [2025-10-20T18:37:25.808743] [DEBUG] [VALIDATOR ] ERO: Primary data validation passed. [2025-10-20T18:37:25.809565] [INFO ] [MAIN ] ERO: Fetched primary data is valid. [2025-10-20T18:37:25.809666] [DEBUG] [LOCAL ] ERO: Saving primary data. [2025-10-20T18:37:25.811134] [DEBUG] [LOCAL ] ERO: Trimmed 60563 records to 42144 (keeping 200 days of data) [2025-10-20T18:37:25.946066] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T18:37:25.964876] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-09. [2025-10-20T18:37:26.099506] [DEBUG] [LOCAL ] ERO: Primary data saved (42144 records). [2025-10-20T18:37:26.099712] [INFO ] [MAIN ] ERO: Full history saved. [2025-10-20T18:37:26.099810] [INFO ] [MAIN ] Update Progress: 762/2367 symbols completed. ERO took 18.1s. Estimated remaining: 3h 25m 4s. Rate: 7.8 symbols/min [2025-10-20T18:37:26.271419] [INFO ] [MAIN ] ESI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:26.320521] [DEBUG] [LOCAL ] ESI: Loaded 51407 existing records from primary storage. [2025-10-20T18:37:26.320882] [INFO ] [MAIN ] ESI: Loaded 51407 existing records. [2025-10-20T18:37:26.506232] [WARN ] [MAIN ] ESI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:26.506721] [INFO ] [MAIN ] ESI: Wiping existing primary data before full refresh. [2025-10-20T18:37:26.506909] [INFO ] [LOCAL ] ESI: Starting deletion of primary data. [2025-10-20T18:37:26.507585] [DEBUG] [LOCAL ] ESI: Deleted primary data file. [2025-10-20T18:37:26.507671] [INFO ] [MAIN ] ESI: Starting full primary refresh. [2025-10-20T18:37:26.679294] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T18:37:26.713058] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-04. [2025-10-20T18:37:26.904541] [INFO ] [REMOTE ] ESI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:26.904889] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-01. [2025-10-20T18:37:26.932847] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T18:37:26.944050] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-10. [2025-10-20T18:37:27.781513] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:37:27.792931] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-10. [2025-10-20T18:37:27.914146] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T18:37:27.918109] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-05. [2025-10-20T18:37:28.447930] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:37:28.457424] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-02. [2025-10-20T18:37:28.812858] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:37:28.820626] [INFO ] [REMOTE ] ERJ: Full fetch complete (72962 records). [2025-10-20T18:37:28.820988] [DEBUG] [VALIDATOR ] ERJ: Validating primary data. [2025-10-20T18:37:28.911148] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T18:37:28.970746] [INFO ] [REMOTE ] ES: Full fetch complete (76475 records). [2025-10-20T18:37:28.976380] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:37:28.976048] [DEBUG] [VALIDATOR ] ES: Validating primary data. [2025-10-20T18:37:28.991446] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-06. [2025-10-20T18:37:29.595979] [DEBUG] [VALIDATOR ] ERJ: All expected trading days are present. [2025-10-20T18:37:29.619385] [DEBUG] [VALIDATOR ] ERJ: Primary data validation passed. [2025-10-20T18:37:29.676288] [DEBUG] [VALIDATOR ] ES: All expected trading days are present. [2025-10-20T18:37:29.677480] [DEBUG] [VALIDATOR ] ES: Primary data validation passed. [2025-10-20T18:37:29.677398] [INFO ] [MAIN ] ERJ: Fetched primary data is valid. [2025-10-20T18:37:29.678233] [INFO ] [MAIN ] ES: Fetched primary data is valid. [2025-10-20T18:37:29.678406] [DEBUG] [LOCAL ] ES: Saving primary data. [2025-10-20T18:37:29.678608] [DEBUG] [LOCAL ] ERJ: Saving primary data. [2025-10-20T18:37:29.680838] [DEBUG] [LOCAL ] ES: Trimmed 76475 records to 52707 (keeping 200 days of data) [2025-10-20T18:37:29.681670] [DEBUG] [LOCAL ] ERJ: Trimmed 72962 records to 50301 (keeping 200 days of data) [2025-10-20T18:37:29.905938] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T18:37:29.939254] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-03. [2025-10-20T18:37:30.042860] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:37:30.078591] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-07. [2025-10-20T18:37:30.272293] [DEBUG] [LOCAL ] ERJ: Primary data saved (50301 records). [2025-10-20T18:37:30.278496] [INFO ] [MAIN ] ERJ: Full history saved. [2025-10-20T18:37:30.283407] [INFO ] [MAIN ] Update Progress: 763/2367 symbols completed. ERJ took 22.3s. Estimated remaining: 3h 24m 49s. Rate: 7.8 symbols/min [2025-10-20T18:37:30.534178] [INFO ] [MAIN ] ESNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:30.541233] [DEBUG] [LOCAL ] ES: Primary data saved (52707 records). [2025-10-20T18:37:30.541700] [INFO ] [MAIN ] ES: Full history saved. [2025-10-20T18:37:30.542260] [INFO ] [MAIN ] Update Progress: 764/2367 symbols completed. ES took 22.5s. Estimated remaining: 3h 24m 26s. Rate: 7.8 symbols/min [2025-10-20T18:37:30.712065] [INFO ] [MAIN ] ESRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:30.741573] [DEBUG] [LOCAL ] ESNT: Loaded 38819 existing records from primary storage. [2025-10-20T18:37:30.741965] [INFO ] [MAIN ] ESNT: Loaded 38819 existing records. [2025-10-20T18:37:30.948955] [WARN ] [MAIN ] ESNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:30.965115] [DEBUG] [LOCAL ] ESRT: Loaded 47197 existing records from primary storage. [2025-10-20T18:37:30.965315] [INFO ] [MAIN ] ESNT: Wiping existing primary data before full refresh. [2025-10-20T18:37:30.965623] [INFO ] [MAIN ] ESRT: Loaded 47197 existing records. [2025-10-20T18:37:30.965734] [INFO ] [LOCAL ] ESNT: Starting deletion of primary data. [2025-10-20T18:37:31.010238] [DEBUG] [LOCAL ] ESNT: Deleted primary data file. [2025-10-20T18:37:31.010334] [INFO ] [MAIN ] ESNT: Starting full primary refresh. [2025-10-20T18:37:31.311288] [WARN ] [MAIN ] ESRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:31.322913] [INFO ] [MAIN ] ESRT: Wiping existing primary data before full refresh. [2025-10-20T18:37:31.324380] [INFO ] [LOCAL ] ESRT: Starting deletion of primary data. [2025-10-20T18:37:31.346150] [DEBUG] [LOCAL ] ESRT: Deleted primary data file. [2025-10-20T18:37:31.393283] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T18:37:31.387710] [INFO ] [MAIN ] ESRT: Starting full primary refresh. [2025-10-20T18:37:31.485006] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-08. [2025-10-20T18:37:31.651161] [INFO ] [REMOTE ] ESNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:31.658040] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-01. [2025-10-20T18:37:31.940294] [INFO ] [REMOTE ] ESRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:32.251353] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-01. [2025-10-20T18:37:32.409457] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-03 (2.47 seconds). [2025-10-20T18:37:33.051372] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-04. [2025-10-20T18:37:33.130382] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T18:37:33.672354] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T18:37:33.753749] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:37:33.851347] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-02. [2025-10-20T18:37:34.651433] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-02. [2025-10-20T18:37:34.926559] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T18:37:35.319424] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T18:37:35.451373] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-03. [2025-10-20T18:37:35.846684] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:37:36.251400] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-05. [2025-10-20T18:37:36.650180] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T18:37:37.051438] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-03. [2025-10-20T18:37:37.851432] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-04. [2025-10-20T18:37:37.900031] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T18:37:38.391543] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:37:38.651395] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-09. [2025-10-20T18:37:39.358630] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:37:39.451423] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-06. [2025-10-20T18:37:39.781776] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T18:37:40.251408] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-04. [2025-10-20T18:37:40.915217] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T18:37:41.051382] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-10. [2025-10-20T18:37:41.757501] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:37:41.851404] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-05. [2025-10-20T18:37:42.039292] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T18:37:42.042050] [INFO ] [REMOTE ] ESE: Full fetch complete (22811 records). [2025-10-20T18:37:42.042131] [DEBUG] [VALIDATOR ] ESE: Validating primary data. [2025-10-20T18:37:42.414964] [DEBUG] [VALIDATOR ] ESE: All expected trading days are present. [2025-10-20T18:37:42.415057] [DEBUG] [VALIDATOR ] ESE: Primary data validation passed. [2025-10-20T18:37:42.415599] [INFO ] [MAIN ] ESE: Fetched primary data is valid. [2025-10-20T18:37:42.415686] [DEBUG] [LOCAL ] ESE: Saving primary data. [2025-10-20T18:37:42.416379] [DEBUG] [LOCAL ] ESE: Trimmed 22811 records to 16467 (keeping 200 days of data) [2025-10-20T18:37:42.526725] [DEBUG] [LOCAL ] ESE: Primary data saved (16467 records). [2025-10-20T18:37:42.526867] [INFO ] [MAIN ] ESE: Full history saved. [2025-10-20T18:37:42.526947] [INFO ] [MAIN ] Update Progress: 765/2367 symbols completed. ESE took 19.5s. Estimated remaining: 3h 24m 28s. Rate: 7.8 symbols/min [2025-10-20T18:37:42.656539] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-07. [2025-10-20T18:37:42.715236] [INFO ] [MAIN ] ESS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:42.747350] [DEBUG] [LOCAL ] ESS: Loaded 30619 existing records from primary storage. [2025-10-20T18:37:42.747721] [INFO ] [MAIN ] ESS: Loaded 30619 existing records. [2025-10-20T18:37:42.935984] [WARN ] [MAIN ] ESS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:42.936252] [INFO ] [MAIN ] ESS: Wiping existing primary data before full refresh. [2025-10-20T18:37:42.936400] [INFO ] [LOCAL ] ESS: Starting deletion of primary data. [2025-10-20T18:37:42.936787] [DEBUG] [LOCAL ] ESS: Deleted primary data file. [2025-10-20T18:37:42.936845] [INFO ] [MAIN ] ESS: Starting full primary refresh. [2025-10-20T18:37:43.329552] [INFO ] [REMOTE ] ESS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:43.331426] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:37:43.451338] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-06. [2025-10-20T18:37:44.251429] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-05. [2025-10-20T18:37:44.408611] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:37:44.983137] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T18:37:45.051367] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-01. [2025-10-20T18:37:45.851497] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-07. [2025-10-20T18:37:45.889220] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T18:37:46.585266] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T18:37:46.651393] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-06. [2025-10-20T18:37:47.377389] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T18:37:47.451410] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-08. [2025-10-20T18:37:48.251394] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-08. [2025-10-20T18:37:48.426993] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T18:37:49.025630] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:37:49.051344] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-09. [2025-10-20T18:37:49.629812] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T18:37:49.851366] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-09. [2025-10-20T18:37:50.330276] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:37:50.651383] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-10. [2025-10-20T18:37:51.345532] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T18:37:51.451360] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-10. [2025-10-20T18:37:52.251363] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-02. [2025-10-20T18:37:52.466668] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T18:37:52.474256] [INFO ] [REMOTE ] ESI: Full fetch complete (73231 records). [2025-10-20T18:37:52.474371] [DEBUG] [VALIDATOR ] ESI: Validating primary data. [2025-10-20T18:37:52.852965] [DEBUG] [VALIDATOR ] ESI: All expected trading days are present. [2025-10-20T18:37:52.853045] [DEBUG] [VALIDATOR ] ESI: Primary data validation passed. [2025-10-20T18:37:52.853890] [INFO ] [MAIN ] ESI: Fetched primary data is valid. [2025-10-20T18:37:52.853953] [DEBUG] [LOCAL ] ESI: Saving primary data. [2025-10-20T18:37:52.855245] [DEBUG] [LOCAL ] ESI: Trimmed 73231 records to 50643 (keeping 200 days of data) [2025-10-20T18:37:53.062422] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-07. [2025-10-20T18:37:53.158952] [DEBUG] [LOCAL ] ESI: Primary data saved (50643 records). [2025-10-20T18:37:53.159100] [INFO ] [MAIN ] ESI: Full history saved. [2025-10-20T18:37:53.159237] [INFO ] [MAIN ] Update Progress: 766/2367 symbols completed. ESI took 27.1s. Estimated remaining: 3h 24m 26s. Rate: 7.8 symbols/min [2025-10-20T18:37:53.330843] [INFO ] [MAIN ] ESTC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:53.392531] [DEBUG] [LOCAL ] ESTC: Loaded 51091 existing records from primary storage. [2025-10-20T18:37:53.393974] [INFO ] [MAIN ] ESTC: Loaded 51091 existing records. [2025-10-20T18:37:53.393821] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T18:37:53.498624] [INFO ] [REMOTE ] ESNT: Full fetch complete (57227 records). [2025-10-20T18:37:53.498898] [DEBUG] [VALIDATOR ] ESNT: Validating primary data. [2025-10-20T18:37:53.673537] [WARN ] [MAIN ] ESTC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:53.691973] [INFO ] [MAIN ] ESTC: Wiping existing primary data before full refresh. [2025-10-20T18:37:53.746011] [INFO ] [LOCAL ] ESTC: Starting deletion of primary data. [2025-10-20T18:37:53.769050] [DEBUG] [LOCAL ] ESTC: Deleted primary data file. [2025-10-20T18:37:53.766540] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:37:53.769498] [INFO ] [MAIN ] ESTC: Starting full primary refresh. [2025-10-20T18:37:53.925690] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-03. [2025-10-20T18:37:54.239299] [DEBUG] [VALIDATOR ] ESNT: All expected trading days are present. [2025-10-20T18:37:54.276993] [DEBUG] [VALIDATOR ] ESNT: Primary data validation passed. [2025-10-20T18:37:54.302957] [INFO ] [MAIN ] ESNT: Fetched primary data is valid. [2025-10-20T18:37:54.313709] [DEBUG] [LOCAL ] ESNT: Saving primary data. [2025-10-20T18:37:54.382802] [DEBUG] [LOCAL ] ESNT: Trimmed 57227 records to 38177 (keeping 200 days of data) [2025-10-20T18:37:54.414210] [INFO ] [REMOTE ] ESTC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:54.537237] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T18:37:54.654066] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-08. [2025-10-20T18:37:54.683153] [DEBUG] [LOCAL ] ESNT: Primary data saved (38177 records). [2025-10-20T18:37:54.683265] [INFO ] [MAIN ] ESNT: Full history saved. [2025-10-20T18:37:54.683338] [INFO ] [MAIN ] Update Progress: 767/2367 symbols completed. ESNT took 24.4s. Estimated remaining: 3h 24m 6s. Rate: 7.8 symbols/min [2025-10-20T18:37:54.848696] [INFO ] [MAIN ] ET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:54.900450] [DEBUG] [LOCAL ] ET: Loaded 54569 existing records from primary storage. [2025-10-20T18:37:54.900785] [INFO ] [MAIN ] ET: Loaded 54569 existing records. [2025-10-20T18:37:55.082497] [WARN ] [MAIN ] ET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:55.082756] [INFO ] [MAIN ] ET: Wiping existing primary data before full refresh. [2025-10-20T18:37:55.082856] [INFO ] [LOCAL ] ET: Starting deletion of primary data. [2025-10-20T18:37:55.083081] [DEBUG] [LOCAL ] ET: Deleted primary data file. [2025-10-20T18:37:55.083150] [INFO ] [MAIN ] ET: Starting full primary refresh. [2025-10-20T18:37:55.399317] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:37:55.456532] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-01. [2025-10-20T18:37:55.497570] [INFO ] [REMOTE ] ET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:56.251391] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-04. [2025-10-20T18:37:56.337021] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T18:37:57.051380] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-09. [2025-10-20T18:37:57.350917] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:37:57.735665] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:37:57.851389] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-02. [2025-10-20T18:37:58.646178] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:37:58.655468] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-05. [2025-10-20T18:37:59.451389] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-10. [2025-10-20T18:37:59.488510] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T18:37:59.836273] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:38:00.251408] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-01. [2025-10-20T18:38:01.051419] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-06. [2025-10-20T18:38:01.156464] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T18:38:01.163755] [INFO ] [REMOTE ] ESRT: Full fetch complete (66652 records). [2025-10-20T18:38:01.163917] [DEBUG] [VALIDATOR ] ESRT: Validating primary data. [2025-10-20T18:38:01.559248] [DEBUG] [VALIDATOR ] ESRT: All expected trading days are present. [2025-10-20T18:38:01.559338] [DEBUG] [VALIDATOR ] ESRT: Primary data validation passed. [2025-10-20T18:38:01.560158] [INFO ] [MAIN ] ESRT: Fetched primary data is valid. [2025-10-20T18:38:01.560219] [DEBUG] [LOCAL ] ESRT: Saving primary data. [2025-10-20T18:38:01.561489] [DEBUG] [LOCAL ] ESRT: Trimmed 66652 records to 46564 (keeping 200 days of data) [2025-10-20T18:38:01.827211] [DEBUG] [LOCAL ] ESRT: Primary data saved (46564 records). [2025-10-20T18:38:01.827331] [INFO ] [MAIN ] ESRT: Full history saved. [2025-10-20T18:38:01.827946] [INFO ] [MAIN ] Update Progress: 768/2367 symbols completed. ESRT took 31.3s. Estimated remaining: 3h 23m 57s. Rate: 7.8 symbols/min [2025-10-20T18:38:01.860034] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-03. [2025-10-20T18:38:02.009303] [INFO ] [MAIN ] ETB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:02.035385] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T18:38:02.053365] [DEBUG] [LOCAL ] ETB: Loaded 12085 existing records from primary storage. [2025-10-20T18:38:02.053637] [INFO ] [MAIN ] ETB: Loaded 12085 existing records. [2025-10-20T18:38:02.248340] [WARN ] [MAIN ] ETB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:02.248873] [INFO ] [MAIN ] ETB: Wiping existing primary data before full refresh. [2025-10-20T18:38:02.248961] [INFO ] [LOCAL ] ETB: Starting deletion of primary data. [2025-10-20T18:38:02.249339] [DEBUG] [LOCAL ] ETB: Deleted primary data file. [2025-10-20T18:38:02.249426] [INFO ] [MAIN ] ETB: Starting full primary refresh. [2025-10-20T18:38:02.607962] [INFO ] [REMOTE ] ETB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:02.649495] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-01 (2.40 seconds). [2025-10-20T18:38:02.661254] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-01. [2025-10-20T18:38:03.451441] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-02. [2025-10-20T18:38:03.494885] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T18:38:03.842004] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T18:38:04.251415] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-02. [2025-10-20T18:38:05.051395] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-07. [2025-10-20T18:38:05.298273] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:38:05.507979] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T18:38:05.851376] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-03. [2025-10-20T18:38:06.369543] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T18:38:06.482367] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-10-20T18:38:06.651355] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-08. [2025-10-20T18:38:07.451378] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-03. [2025-10-20T18:38:08.017625] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T18:38:08.251351] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-09. [2025-10-20T18:38:09.051381] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-04. [2025-10-20T18:38:09.349185] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T18:38:09.467752] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T18:38:09.763269] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T18:38:09.851359] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-05. [2025-10-20T18:38:10.651345] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-04. [2025-10-20T18:38:10.728703] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T18:38:11.451381] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-06. [2025-10-20T18:38:12.235516] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T18:38:12.251320] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-07. [2025-10-20T18:38:12.350286] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:38:13.057827] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-04. [2025-10-20T18:38:13.058897] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T18:38:13.851364] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-05. [2025-10-20T18:38:14.651435] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-08. [2025-10-20T18:38:15.226625] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T18:38:15.451415] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-10. [2025-10-20T18:38:15.536053] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-04 (2.48 seconds). [2025-10-20T18:38:15.555751] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T18:38:16.251383] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-05. [2025-10-20T18:38:16.907065] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:38:16.911523] [INFO ] [REMOTE ] ESS: Full fetch complete (43527 records). [2025-10-20T18:38:16.911610] [DEBUG] [VALIDATOR ] ESS: Validating primary data. [2025-10-20T18:38:16.912213] [FATAL] [VALIDATOR ] ESS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:38:16.912295] [FATAL] [MAIN ] ESS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:38:16.912338] [WARN ] [MAIN ] ESS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:38:16.912374] [INFO ] [LOCAL ] ESS: Starting deletion of primary data. [2025-10-20T18:38:16.912450] [DEBUG] [LOCAL ] ESS: No primary data file found to delete. [2025-10-20T18:38:16.912519] [INFO ] [LOCAL ] ESS: Starting deletion of derivative data. [2025-10-20T18:38:16.912784] [DEBUG] [LOCAL ] ESS: Deleted derivative data file. [2025-10-20T18:38:16.912882] [INFO ] [MAIN ] Update Progress: 769/2367 symbols completed. ESS took 34.4s. Estimated remaining: 3h 24m 5s. Rate: 7.8 symbols/min [2025-10-20T18:38:17.056520] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-09. [2025-10-20T18:38:17.081085] [INFO ] [MAIN ] ETD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:17.106523] [DEBUG] [LOCAL ] ETD: Loaded 27663 existing records from primary storage. [2025-10-20T18:38:17.106809] [INFO ] [MAIN ] ETD: Loaded 27663 existing records. [2025-10-20T18:38:17.280517] [WARN ] [MAIN ] ETD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:17.280708] [INFO ] [MAIN ] ETD: Wiping existing primary data before full refresh. [2025-10-20T18:38:17.280762] [INFO ] [LOCAL ] ETD: Starting deletion of primary data. [2025-10-20T18:38:17.280948] [DEBUG] [LOCAL ] ETD: Deleted primary data file. [2025-10-20T18:38:17.280985] [INFO ] [MAIN ] ETD: Starting full primary refresh. [2025-10-20T18:38:17.630304] [INFO ] [REMOTE ] ETD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:17.834942] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T18:38:17.851332] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-10. [2025-10-20T18:38:18.310516] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-05 (2.06 seconds). [2025-10-20T18:38:18.651454] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-06. [2025-10-20T18:38:18.767088] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T18:38:18.769784] [INFO ] [REMOTE ] ETB: Full fetch complete (18466 records). [2025-10-20T18:38:18.769863] [DEBUG] [VALIDATOR ] ETB: Validating primary data. [2025-10-20T18:38:19.145250] [DEBUG] [VALIDATOR ] ETB: All expected trading days are present. [2025-10-20T18:38:19.145336] [DEBUG] [VALIDATOR ] ETB: Primary data validation passed. [2025-10-20T18:38:19.145897] [INFO ] [MAIN ] ETB: Fetched primary data is valid. [2025-10-20T18:38:19.145960] [DEBUG] [LOCAL ] ETB: Saving primary data. [2025-10-20T18:38:19.146690] [DEBUG] [LOCAL ] ETB: Trimmed 18466 records to 11975 (keeping 200 days of data) [2025-10-20T18:38:19.220153] [DEBUG] [LOCAL ] ETB: Primary data saved (11975 records). [2025-10-20T18:38:19.220290] [INFO ] [MAIN ] ETB: Full history saved. [2025-10-20T18:38:19.220369] [INFO ] [MAIN ] Update Progress: 770/2367 symbols completed. ETB took 17.4s. Estimated remaining: 3h 23m 46s. Rate: 7.8 symbols/min [2025-10-20T18:38:19.390762] [INFO ] [MAIN ] ETG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:19.408897] [DEBUG] [LOCAL ] ETG: Loaded 16892 existing records from primary storage. [2025-10-20T18:38:19.409132] [INFO ] [MAIN ] ETG: Loaded 16892 existing records. [2025-10-20T18:38:19.456616] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-01. [2025-10-20T18:38:19.595460] [WARN ] [MAIN ] ETG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:19.595699] [INFO ] [MAIN ] ETG: Wiping existing primary data before full refresh. [2025-10-20T18:38:19.595800] [INFO ] [LOCAL ] ETG: Starting deletion of primary data. [2025-10-20T18:38:19.595968] [DEBUG] [LOCAL ] ETG: Deleted primary data file. [2025-10-20T18:38:19.596008] [INFO ] [MAIN ] ETG: Starting full primary refresh. [2025-10-20T18:38:19.951334] [INFO ] [REMOTE ] ETG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:20.032271] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:38:20.251377] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-01. [2025-10-20T18:38:20.518590] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T18:38:21.051394] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-02. [2025-10-20T18:38:21.415128] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:38:21.851397] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-02. [2025-10-20T18:38:22.134140] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T18:38:22.597146] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:38:22.651350] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-03. [2025-10-20T18:38:23.451405] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-06. [2025-10-20T18:38:23.748617] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T18:38:24.251433] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-04. [2025-10-20T18:38:25.051365] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-07. [2025-10-20T18:38:25.622975] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T18:38:25.693594] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T18:38:25.851392] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-05. [2025-10-20T18:38:26.625202] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T18:38:26.651331] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-08. [2025-10-20T18:38:27.109548] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T18:38:27.451440] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-03. [2025-10-20T18:38:28.251346] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-07. [2025-10-20T18:38:28.482471] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T18:38:28.532037] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T18:38:29.051425] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-04. [2025-10-20T18:38:29.851444] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-09. [2025-10-20T18:38:30.055027] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T18:38:30.656582] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-05. [2025-10-20T18:38:30.664977] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-07 (2.41 seconds). [2025-10-20T18:38:31.451338] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-08. [2025-10-20T18:38:31.484785] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:38:31.695571] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:38:32.251376] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-06. [2025-10-20T18:38:33.051360] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-06. [2025-10-20T18:38:33.090156] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T18:38:33.800538] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-08 (2.35 seconds). [2025-10-20T18:38:33.851380] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-09. [2025-10-20T18:38:34.107035] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:38:34.651449] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-07. [2025-10-20T18:38:35.451421] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-07. [2025-10-20T18:38:35.875347] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T18:38:36.059228] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T18:38:36.112304] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T18:38:36.251379] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-08. [2025-10-20T18:38:37.051419] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-08. [2025-10-20T18:38:37.722602] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T18:38:37.851334] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-09. [2025-10-20T18:38:38.229383] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T18:38:38.651415] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-10. [2025-10-20T18:38:39.031002] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:38:39.451384] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-10. [2025-10-20T18:38:40.251409] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-09. [2025-10-20T18:38:40.687243] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T18:38:40.694690] [INFO ] [REMOTE ] ESTC: Full fetch complete (72485 records). [2025-10-20T18:38:40.694803] [DEBUG] [VALIDATOR ] ESTC: Validating primary data. [2025-10-20T18:38:40.811308] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T18:38:40.854522] [INFO ] [REMOTE ] ETD: Full fetch complete (38642 records). [2025-10-20T18:38:40.886790] [DEBUG] [VALIDATOR ] ETD: Validating primary data. [2025-10-20T18:38:41.062364] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-10. [2025-10-20T18:38:41.078252] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:38:41.399754] [DEBUG] [VALIDATOR ] ESTC: All expected trading days are present. [2025-10-20T18:38:41.405776] [DEBUG] [VALIDATOR ] ESTC: Primary data validation passed. [2025-10-20T18:38:41.406814] [INFO ] [MAIN ] ESTC: Fetched primary data is valid. [2025-10-20T18:38:41.417824] [DEBUG] [LOCAL ] ESTC: Saving primary data. [2025-10-20T18:38:41.478777] [DEBUG] [LOCAL ] ESTC: Trimmed 72485 records to 50306 (keeping 200 days of data) [2025-10-20T18:38:41.586624] [DEBUG] [VALIDATOR ] ETD: All expected trading days are present. [2025-10-20T18:38:41.586829] [DEBUG] [VALIDATOR ] ETD: Primary data validation passed. [2025-10-20T18:38:41.612311] [INFO ] [MAIN ] ETD: Fetched primary data is valid. [2025-10-20T18:38:41.612726] [DEBUG] [LOCAL ] ETD: Saving primary data. [2025-10-20T18:38:41.631718] [DEBUG] [LOCAL ] ETD: Trimmed 38642 records to 27370 (keeping 200 days of data) [2025-10-20T18:38:41.858682] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-10. [2025-10-20T18:38:41.951597] [DEBUG] [LOCAL ] ETD: Primary data saved (27370 records). [2025-10-20T18:38:41.955922] [INFO ] [MAIN ] ETD: Full history saved. [2025-10-20T18:38:41.971845] [INFO ] [MAIN ] Update Progress: 771/2367 symbols completed. ETD took 25.1s. Estimated remaining: 3h 24m 10s. Rate: 7.8 symbols/min [2025-10-20T18:38:42.024558] [DEBUG] [LOCAL ] ESTC: Primary data saved (50306 records). [2025-10-20T18:38:42.036903] [INFO ] [MAIN ] ESTC: Full history saved. [2025-10-20T18:38:42.072200] [INFO ] [MAIN ] Update Progress: 772/2367 symbols completed. ESTC took 48.9s. Estimated remaining: 3h 23m 46s. Rate: 7.8 symbols/min [2025-10-20T18:38:42.318686] [INFO ] [MAIN ] ETJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:42.371913] [INFO ] [MAIN ] ETN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:42.401159] [DEBUG] [LOCAL ] ETJ: Loaded 17031 existing records from primary storage. [2025-10-20T18:38:42.401447] [INFO ] [MAIN ] ETJ: Loaded 17031 existing records. [2025-10-20T18:38:42.622082] [WARN ] [MAIN ] ETJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:42.623822] [INFO ] [MAIN ] ETJ: Wiping existing primary data before full refresh. [2025-10-20T18:38:42.623903] [INFO ] [LOCAL ] ETJ: Starting deletion of primary data. [2025-10-20T18:38:42.624939] [DEBUG] [LOCAL ] ETJ: Deleted primary data file. [2025-10-20T18:38:42.624869] [DEBUG] [LOCAL ] ETN: Loaded 53392 existing records from primary storage. [2025-10-20T18:38:42.625106] [INFO ] [MAIN ] ETJ: Starting full primary refresh. [2025-10-20T18:38:42.630630] [INFO ] [MAIN ] ETN: Loaded 53392 existing records. [2025-10-20T18:38:43.073755] [WARN ] [MAIN ] ETN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:43.084884] [INFO ] [MAIN ] ETN: Wiping existing primary data before full refresh. [2025-10-20T18:38:43.102832] [INFO ] [LOCAL ] ETN: Starting deletion of primary data. [2025-10-20T18:38:43.132045] [DEBUG] [LOCAL ] ETN: Deleted primary data file. [2025-10-20T18:38:43.154857] [INFO ] [MAIN ] ETN: Starting full primary refresh. [2025-10-20T18:38:43.155638] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T18:38:43.218025] [INFO ] [REMOTE ] ETG: Full fetch complete (24799 records). [2025-10-20T18:38:43.274484] [DEBUG] [VALIDATOR ] ETG: Validating primary data. [2025-10-20T18:38:43.357823] [INFO ] [REMOTE ] ETJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:43.379926] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-01. [2025-10-20T18:38:44.032770] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-10 (2.97 seconds). [2025-10-20T18:38:44.104901] [INFO ] [REMOTE ] ETN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:44.106993] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-01. [2025-10-20T18:38:44.113410] [INFO ] [REMOTE ] ET: Full fetch complete (77969 records). [2025-10-20T18:38:44.120133] [DEBUG] [VALIDATOR ] ET: Validating primary data. [2025-10-20T18:38:44.284965] [DEBUG] [VALIDATOR ] ETG: All expected trading days are present. [2025-10-20T18:38:44.299200] [DEBUG] [VALIDATOR ] ETG: Primary data validation passed. [2025-10-20T18:38:44.319474] [INFO ] [MAIN ] ETG: Fetched primary data is valid. [2025-10-20T18:38:44.327227] [DEBUG] [LOCAL ] ETG: Saving primary data. [2025-10-20T18:38:44.349913] [DEBUG] [LOCAL ] ETG: Trimmed 24799 records to 16667 (keeping 200 days of data) [2025-10-20T18:38:44.680384] [DEBUG] [LOCAL ] ETG: Primary data saved (16667 records). [2025-10-20T18:38:44.709890] [INFO ] [MAIN ] ETG: Full history saved. [2025-10-20T18:38:44.719130] [INFO ] [MAIN ] Update Progress: 773/2367 symbols completed. ETG took 25.5s. Estimated remaining: 3h 23m 28s. Rate: 7.8 symbols/min [2025-10-20T18:38:44.711357] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T18:38:44.798049] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-02. [2025-10-20T18:38:44.946858] [DEBUG] [VALIDATOR ] ET: All expected trading days are present. [2025-10-20T18:38:44.947353] [DEBUG] [VALIDATOR ] ET: Primary data validation passed. [2025-10-20T18:38:44.964409] [INFO ] [MAIN ] ET: Fetched primary data is valid. [2025-10-20T18:38:44.991580] [DEBUG] [LOCAL ] ET: Saving primary data. [2025-10-20T18:38:45.026103] [INFO ] [MAIN ] ETO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:45.027798] [DEBUG] [LOCAL ] ET: Trimmed 77969 records to 53789 (keeping 200 days of data) [2025-10-20T18:38:45.167495] [DEBUG] [LOCAL ] ETO: Loaded 7052 existing records from primary storage. [2025-10-20T18:38:45.180375] [INFO ] [MAIN ] ETO: Loaded 7052 existing records. [2025-10-20T18:38:45.629082] [WARN ] [MAIN ] ETO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:45.642762] [INFO ] [MAIN ] ETO: Wiping existing primary data before full refresh. [2025-10-20T18:38:45.642910] [INFO ] [LOCAL ] ETO: Starting deletion of primary data. [2025-10-20T18:38:45.643145] [DEBUG] [LOCAL ] ETO: Deleted primary data file. [2025-10-20T18:38:45.643246] [INFO ] [MAIN ] ETO: Starting full primary refresh. [2025-10-20T18:38:45.643070] [DEBUG] [LOCAL ] ET: Primary data saved (53789 records). [2025-10-20T18:38:45.710862] [INFO ] [MAIN ] ET: Full history saved. [2025-10-20T18:38:45.724060] [INFO ] [MAIN ] Update Progress: 774/2367 symbols completed. ET took 51.0s. Estimated remaining: 3h 23m 7s. Rate: 7.8 symbols/min [2025-10-20T18:38:46.081432] [INFO ] [MAIN ] ETR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:46.176405] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:38:46.232597] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-03. [2025-10-20T18:38:46.255933] [INFO ] [REMOTE ] ETO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:46.257564] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-01. [2025-10-20T18:38:46.300184] [DEBUG] [LOCAL ] ETR: Loaded 53984 existing records from primary storage. [2025-10-20T18:38:46.300693] [INFO ] [MAIN ] ETR: Loaded 53984 existing records. [2025-10-20T18:38:46.444445] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-01 (2.34 seconds). [2025-10-20T18:38:46.530543] [WARN ] [MAIN ] ETR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:46.530791] [INFO ] [MAIN ] ETR: Wiping existing primary data before full refresh. [2025-10-20T18:38:46.530843] [INFO ] [LOCAL ] ETR: Starting deletion of primary data. [2025-10-20T18:38:46.531050] [DEBUG] [LOCAL ] ETR: Deleted primary data file. [2025-10-20T18:38:46.531088] [INFO ] [MAIN ] ETR: Starting full primary refresh. [2025-10-20T18:38:46.666476] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-02. [2025-10-20T18:38:46.882347] [INFO ] [REMOTE ] ETR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:47.143691] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T18:38:47.317405] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T18:38:47.451357] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-04. [2025-10-20T18:38:48.251354] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-02. [2025-10-20T18:38:48.558535] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T18:38:48.743671] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-02 (2.08 seconds). [2025-10-20T18:38:48.761574] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T18:38:49.051394] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-05. [2025-10-20T18:38:49.851395] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-01. [2025-10-20T18:38:49.885166] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:38:50.651375] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-03. [2025-10-20T18:38:51.451363] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-03. [2025-10-20T18:38:51.691169] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:38:52.090541] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:38:52.251368] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-02. [2025-10-20T18:38:52.686311] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T18:38:53.051420] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-04. [2025-10-20T18:38:53.600056] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T18:38:53.851411] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-04. [2025-10-20T18:38:53.979490] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T18:38:54.651412] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-06. [2025-10-20T18:38:55.451396] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-03. [2025-10-20T18:38:55.600820] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T18:38:55.874024] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T18:38:56.251392] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-07. [2025-10-20T18:38:57.051356] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-05. [2025-10-20T18:38:57.137917] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T18:38:57.320609] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T18:38:57.851383] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-04. [2025-10-20T18:38:57.866324] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T18:38:58.651359] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-08. [2025-10-20T18:38:59.451389] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-05. [2025-10-20T18:38:59.635017] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:38:59.670180] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T18:39:00.251377] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-05. [2025-10-20T18:39:01.051424] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-09. [2025-10-20T18:39:01.707582] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-10-20T18:39:01.851339] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-06. [2025-10-20T18:39:02.133483] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T18:39:02.376546] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T18:39:02.651363] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-06. [2025-10-20T18:39:03.451368] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-10. [2025-10-20T18:39:03.576960] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T18:39:04.251350] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-06. [2025-10-20T18:39:04.426832] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T18:39:04.473768] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T18:39:04.476660] [INFO ] [REMOTE ] ETJ: Full fetch complete (25632 records). [2025-10-20T18:39:04.476736] [DEBUG] [VALIDATOR ] ETJ: Validating primary data. [2025-10-20T18:39:04.818541] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T18:39:04.865353] [DEBUG] [VALIDATOR ] ETJ: All expected trading days are present. [2025-10-20T18:39:04.865431] [DEBUG] [VALIDATOR ] ETJ: Primary data validation passed. [2025-10-20T18:39:04.866026] [INFO ] [MAIN ] ETJ: Fetched primary data is valid. [2025-10-20T18:39:04.866089] [DEBUG] [LOCAL ] ETJ: Saving primary data. [2025-10-20T18:39:04.866732] [DEBUG] [LOCAL ] ETJ: Trimmed 25632 records to 16676 (keeping 200 days of data) [2025-10-20T18:39:04.982468] [DEBUG] [LOCAL ] ETJ: Primary data saved (16676 records). [2025-10-20T18:39:04.982613] [INFO ] [MAIN ] ETJ: Full history saved. [2025-10-20T18:39:04.982721] [INFO ] [MAIN ] Update Progress: 775/2367 symbols completed. ETJ took 23.0s. Estimated remaining: 3h 23m 23s. Rate: 7.8 symbols/min [2025-10-20T18:39:05.056465] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-07. [2025-10-20T18:39:05.151203] [INFO ] [MAIN ] ETSY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:05.199771] [DEBUG] [LOCAL ] ETSY: Loaded 54353 existing records from primary storage. [2025-10-20T18:39:05.200107] [INFO ] [MAIN ] ETSY: Loaded 54353 existing records. [2025-10-20T18:39:05.370745] [WARN ] [MAIN ] ETSY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:05.370919] [INFO ] [MAIN ] ETSY: Wiping existing primary data before full refresh. [2025-10-20T18:39:05.370966] [INFO ] [LOCAL ] ETSY: Starting deletion of primary data. [2025-10-20T18:39:05.371262] [DEBUG] [LOCAL ] ETSY: Deleted primary data file. [2025-10-20T18:39:05.371318] [INFO ] [MAIN ] ETSY: Starting full primary refresh. [2025-10-20T18:39:05.722124] [INFO ] [REMOTE ] ETSY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:05.821303] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T18:39:05.851335] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-08. [2025-10-20T18:39:06.518113] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T18:39:06.651343] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-07. [2025-10-20T18:39:07.451393] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-01. [2025-10-20T18:39:08.251356] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-09. [2025-10-20T18:39:08.500295] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:39:09.051337] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-08. [2025-10-20T18:39:09.057583] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T18:39:09.245903] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:39:09.851377] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-10. [2025-10-20T18:39:10.651389] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-02. [2025-10-20T18:39:10.660250] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T18:39:10.662112] [INFO ] [REMOTE ] ETO: Full fetch complete (9731 records). [2025-10-20T18:39:10.662182] [DEBUG] [VALIDATOR ] ETO: Validating primary data. [2025-10-20T18:39:11.055181] [DEBUG] [VALIDATOR ] ETO: All expected trading days are present. [2025-10-20T18:39:11.055346] [DEBUG] [VALIDATOR ] ETO: Primary data validation passed. [2025-10-20T18:39:11.055630] [INFO ] [MAIN ] ETO: Fetched primary data is valid. [2025-10-20T18:39:11.055765] [DEBUG] [LOCAL ] ETO: Saving primary data. [2025-10-20T18:39:11.056207] [DEBUG] [LOCAL ] ETO: Trimmed 9731 records to 7007 (keeping 200 days of data) [2025-10-20T18:39:11.098376] [DEBUG] [LOCAL ] ETO: Primary data saved (7007 records). [2025-10-20T18:39:11.098466] [INFO ] [MAIN ] ETO: Full history saved. [2025-10-20T18:39:11.098539] [INFO ] [MAIN ] Update Progress: 776/2367 symbols completed. ETO took 26.4s. Estimated remaining: 3h 23m 12s. Rate: 7.8 symbols/min [2025-10-20T18:39:11.267208] [INFO ] [MAIN ] ETV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:11.289746] [DEBUG] [LOCAL ] ETV: Loaded 22515 existing records from primary storage. [2025-10-20T18:39:11.290000] [INFO ] [MAIN ] ETV: Loaded 22515 existing records. [2025-10-20T18:39:11.464297] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-07. [2025-10-20T18:39:11.469158] [WARN ] [MAIN ] ETV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:11.469437] [INFO ] [MAIN ] ETV: Wiping existing primary data before full refresh. [2025-10-20T18:39:11.470329] [INFO ] [LOCAL ] ETV: Starting deletion of primary data. [2025-10-20T18:39:11.470589] [DEBUG] [LOCAL ] ETV: Deleted primary data file. [2025-10-20T18:39:11.470693] [INFO ] [MAIN ] ETV: Starting full primary refresh. [2025-10-20T18:39:11.654030] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-08 (2.60 seconds). [2025-10-20T18:39:11.888275] [INFO ] [REMOTE ] ETV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:12.251375] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-01. [2025-10-20T18:39:12.404978] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:39:13.051385] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-09. [2025-10-20T18:39:13.113749] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:39:13.528497] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T18:39:13.851379] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-08. [2025-10-20T18:39:14.652304] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-02. [2025-10-20T18:39:15.106775] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T18:39:15.341749] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T18:39:15.451356] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-10. [2025-10-20T18:39:15.631307] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:39:16.251384] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-09. [2025-10-20T18:39:17.051398] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-03. [2025-10-20T18:39:17.650094] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T18:39:17.657915] [INFO ] [REMOTE ] ETN: Full fetch complete (76644 records). [2025-10-20T18:39:17.658105] [DEBUG] [VALIDATOR ] ETN: Validating primary data. [2025-10-20T18:39:17.867898] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-03. [2025-10-20T18:39:18.060907] [DEBUG] [VALIDATOR ] ETN: All expected trading days are present. [2025-10-20T18:39:18.061166] [DEBUG] [VALIDATOR ] ETN: Primary data validation passed. [2025-10-20T18:39:18.062117] [INFO ] [MAIN ] ETN: Fetched primary data is valid. [2025-10-20T18:39:18.062206] [DEBUG] [LOCAL ] ETN: Saving primary data. [2025-10-20T18:39:18.063824] [DEBUG] [LOCAL ] ETN: Trimmed 76644 records to 52584 (keeping 200 days of data) [2025-10-20T18:39:18.188359] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T18:39:18.227299] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T18:39:18.459511] [DEBUG] [LOCAL ] ETN: Primary data saved (52584 records). [2025-10-20T18:39:18.459680] [INFO ] [MAIN ] ETN: Full history saved. [2025-10-20T18:39:18.459777] [INFO ] [MAIN ] Update Progress: 777/2367 symbols completed. ETN took 36.4s. Estimated remaining: 3h 23m 4s. Rate: 7.8 symbols/min [2025-10-20T18:39:18.628574] [INFO ] [MAIN ] ETW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:18.653438] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-10. [2025-10-20T18:39:18.654062] [DEBUG] [LOCAL ] ETW: Loaded 25887 existing records from primary storage. [2025-10-20T18:39:18.655181] [INFO ] [MAIN ] ETW: Loaded 25887 existing records. [2025-10-20T18:39:18.839547] [WARN ] [MAIN ] ETW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:18.839846] [INFO ] [MAIN ] ETW: Wiping existing primary data before full refresh. [2025-10-20T18:39:18.839895] [INFO ] [LOCAL ] ETW: Starting deletion of primary data. [2025-10-20T18:39:18.840491] [DEBUG] [LOCAL ] ETW: Deleted primary data file. [2025-10-20T18:39:18.840551] [INFO ] [MAIN ] ETW: Starting full primary refresh. [2025-10-20T18:39:19.202545] [INFO ] [REMOTE ] ETW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:19.451360] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-01. [2025-10-20T18:39:20.020846] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-10-20T18:39:20.106818] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T18:39:20.114839] [INFO ] [REMOTE ] ETR: Full fetch complete (77276 records). [2025-10-20T18:39:20.114997] [DEBUG] [VALIDATOR ] ETR: Validating primary data. [2025-10-20T18:39:20.252880] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-04. [2025-10-20T18:39:20.297127] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T18:39:20.549831] [DEBUG] [VALIDATOR ] ETR: All expected trading days are present. [2025-10-20T18:39:20.549918] [DEBUG] [VALIDATOR ] ETR: Primary data validation passed. [2025-10-20T18:39:20.550839] [INFO ] [MAIN ] ETR: Fetched primary data is valid. [2025-10-20T18:39:20.550902] [DEBUG] [LOCAL ] ETR: Saving primary data. [2025-10-20T18:39:20.552328] [DEBUG] [LOCAL ] ETR: Trimmed 77276 records to 53196 (keeping 200 days of data) [2025-10-20T18:39:20.884098] [DEBUG] [LOCAL ] ETR: Primary data saved (53196 records). [2025-10-20T18:39:20.884243] [INFO ] [MAIN ] ETR: Full history saved. [2025-10-20T18:39:20.884323] [INFO ] [MAIN ] Update Progress: 778/2367 symbols completed. ETR took 35.2s. Estimated remaining: 3h 22m 46s. Rate: 7.8 symbols/min [2025-10-20T18:39:21.068715] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-04. [2025-10-20T18:39:21.072293] [INFO ] [MAIN ] ETX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:21.079235] [DEBUG] [LOCAL ] ETX: Loaded 5300 existing records from primary storage. [2025-10-20T18:39:21.079397] [INFO ] [MAIN ] ETX: Loaded 5300 existing records. [2025-10-20T18:39:21.254408] [WARN ] [MAIN ] ETX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:21.254617] [INFO ] [MAIN ] ETX: Wiping existing primary data before full refresh. [2025-10-20T18:39:21.254693] [INFO ] [LOCAL ] ETX: Starting deletion of primary data. [2025-10-20T18:39:21.254889] [DEBUG] [LOCAL ] ETX: Deleted primary data file. [2025-10-20T18:39:21.254944] [INFO ] [MAIN ] ETX: Starting full primary refresh. [2025-10-20T18:39:21.426842] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T18:39:21.667245] [INFO ] [REMOTE ] ETX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:21.851366] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-05. [2025-10-20T18:39:22.476415] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T18:39:22.651377] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-01. [2025-10-20T18:39:23.051018] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T18:39:23.424307] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:39:23.451439] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-02. [2025-10-20T18:39:24.028280] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T18:39:24.251409] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-05. [2025-10-20T18:39:25.051385] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-06. [2025-10-20T18:39:25.851376] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-02. [2025-10-20T18:39:26.000062] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T18:39:26.199551] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T18:39:26.651358] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-03. [2025-10-20T18:39:26.900983] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:39:27.199401] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T18:39:27.451371] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-07. [2025-10-20T18:39:28.251420] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-03. [2025-10-20T18:39:28.519322] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:39:29.051382] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-04. [2025-10-20T18:39:29.355436] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T18:39:29.593300] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T18:39:29.851370] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-04. [2025-10-20T18:39:30.651399] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-08. [2025-10-20T18:39:31.168514] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T18:39:31.451378] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-05. [2025-10-20T18:39:31.730029] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:39:32.251362] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-09. [2025-10-20T18:39:32.571119] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T18:39:33.051360] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-06. [2025-10-20T18:39:33.417271] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:39:33.851394] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-06. [2025-10-20T18:39:34.131057] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T18:39:34.651367] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-05. [2025-10-20T18:39:35.300732] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T18:39:35.451359] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-06. [2025-10-20T18:39:35.625634] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:39:36.180027] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:39:36.251396] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-07. [2025-10-20T18:39:37.051392] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-07. [2025-10-20T18:39:37.640197] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T18:39:37.851418] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-08. [2025-10-20T18:39:38.176937] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T18:39:38.547404] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T18:39:38.651394] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-07. [2025-10-20T18:39:39.451349] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-09. [2025-10-20T18:39:39.723139] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:39:40.028908] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T18:39:40.251392] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-10. [2025-10-20T18:39:40.959463] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T18:39:40.960708] [INFO ] [REMOTE ] ETX: Full fetch complete (6789 records). [2025-10-20T18:39:40.960776] [DEBUG] [VALIDATOR ] ETX: Validating primary data. [2025-10-20T18:39:41.059504] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-08. [2025-10-20T18:39:41.316004] [DEBUG] [VALIDATOR ] ETX: All expected trading days are present. [2025-10-20T18:39:41.316084] [DEBUG] [VALIDATOR ] ETX: Primary data validation passed. [2025-10-20T18:39:41.316299] [INFO ] [MAIN ] ETX: Fetched primary data is valid. [2025-10-20T18:39:41.316336] [DEBUG] [LOCAL ] ETX: Saving primary data. [2025-10-20T18:39:41.316737] [DEBUG] [LOCAL ] ETX: Trimmed 6789 records to 5277 (keeping 200 days of data) [2025-10-20T18:39:41.348258] [DEBUG] [LOCAL ] ETX: Primary data saved (5277 records). [2025-10-20T18:39:41.348361] [INFO ] [MAIN ] ETX: Full history saved. [2025-10-20T18:39:41.348432] [INFO ] [MAIN ] Update Progress: 779/2367 symbols completed. ETX took 20.5s. Estimated remaining: 3h 23m 4s. Rate: 7.8 symbols/min [2025-10-20T18:39:41.523314] [INFO ] [MAIN ] ETY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:41.545979] [DEBUG] [LOCAL ] ETY: Loaded 23210 existing records from primary storage. [2025-10-20T18:39:41.546227] [INFO ] [MAIN ] ETY: Loaded 23210 existing records. [2025-10-20T18:39:41.719785] [WARN ] [MAIN ] ETY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:41.720015] [INFO ] [MAIN ] ETY: Wiping existing primary data before full refresh. [2025-10-20T18:39:41.720062] [INFO ] [LOCAL ] ETY: Starting deletion of primary data. [2025-10-20T18:39:41.720218] [DEBUG] [LOCAL ] ETY: Deleted primary data file. [2025-10-20T18:39:41.720255] [INFO ] [MAIN ] ETY: Starting full primary refresh. [2025-10-20T18:39:41.862082] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-08. [2025-10-20T18:39:42.100011] [INFO ] [REMOTE ] ETY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:42.651418] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-10. [2025-10-20T18:39:43.224871] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T18:39:43.268374] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T18:39:43.451396] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-01. [2025-10-20T18:39:43.620517] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T18:39:43.624882] [INFO ] [REMOTE ] ETV: Full fetch complete (32456 records). [2025-10-20T18:39:43.624978] [DEBUG] [VALIDATOR ] ETV: Validating primary data. [2025-10-20T18:39:44.013902] [DEBUG] [VALIDATOR ] ETV: All expected trading days are present. [2025-10-20T18:39:44.013987] [DEBUG] [VALIDATOR ] ETV: Primary data validation passed. [2025-10-20T18:39:44.014538] [INFO ] [MAIN ] ETV: Fetched primary data is valid. [2025-10-20T18:39:44.014598] [DEBUG] [LOCAL ] ETV: Saving primary data. [2025-10-20T18:39:44.015454] [DEBUG] [LOCAL ] ETV: Trimmed 32456 records to 22127 (keeping 200 days of data) [2025-10-20T18:39:44.146496] [DEBUG] [LOCAL ] ETV: Primary data saved (22127 records). [2025-10-20T18:39:44.146617] [INFO ] [MAIN ] ETV: Full history saved. [2025-10-20T18:39:44.146722] [INFO ] [MAIN ] Update Progress: 780/2367 symbols completed. ETV took 33.0s. Estimated remaining: 3h 22m 46s. Rate: 7.8 symbols/min [2025-10-20T18:39:44.256450] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-09. [2025-10-20T18:39:44.318663] [INFO ] [MAIN ] EVC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:44.337214] [DEBUG] [LOCAL ] EVC: Loaded 18307 existing records from primary storage. [2025-10-20T18:39:44.337421] [INFO ] [MAIN ] EVC: Loaded 18307 existing records. [2025-10-20T18:39:44.521993] [WARN ] [MAIN ] EVC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:44.522234] [INFO ] [MAIN ] EVC: Wiping existing primary data before full refresh. [2025-10-20T18:39:44.522283] [INFO ] [LOCAL ] EVC: Starting deletion of primary data. [2025-10-20T18:39:44.522450] [DEBUG] [LOCAL ] EVC: Deleted primary data file. [2025-10-20T18:39:44.522489] [INFO ] [MAIN ] EVC: Starting full primary refresh. [2025-10-20T18:39:44.890442] [INFO ] [REMOTE ] EVC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:44.904747] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:39:45.051459] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-01. [2025-10-20T18:39:45.851418] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-09. [2025-10-20T18:39:46.071826] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:39:46.542620] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T18:39:46.651443] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-02. [2025-10-20T18:39:46.933061] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T18:39:47.451413] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-02. [2025-10-20T18:39:47.820878] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T18:39:48.251396] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-10. [2025-10-20T18:39:48.451174] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T18:39:49.051396] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-03. [2025-10-20T18:39:49.619018] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:39:49.623954] [INFO ] [REMOTE ] ETW: Full fetch complete (36669 records). [2025-10-20T18:39:49.624081] [DEBUG] [VALIDATOR ] ETW: Validating primary data. [2025-10-20T18:39:49.854224] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-10. [2025-10-20T18:39:49.999694] [DEBUG] [VALIDATOR ] ETW: All expected trading days are present. [2025-10-20T18:39:49.999794] [DEBUG] [VALIDATOR ] ETW: Primary data validation passed. [2025-10-20T18:39:50.000459] [INFO ] [MAIN ] ETW: Fetched primary data is valid. [2025-10-20T18:39:50.000500] [DEBUG] [LOCAL ] ETW: Saving primary data. [2025-10-20T18:39:50.001449] [DEBUG] [LOCAL ] ETW: Trimmed 36669 records to 25419 (keeping 200 days of data) [2025-10-20T18:39:50.149667] [DEBUG] [LOCAL ] ETW: Primary data saved (25419 records). [2025-10-20T18:39:50.149979] [INFO ] [MAIN ] ETW: Full history saved. [2025-10-20T18:39:50.150095] [INFO ] [MAIN ] Update Progress: 781/2367 symbols completed. ETW took 31.7s. Estimated remaining: 3h 22m 35s. Rate: 7.8 symbols/min [2025-10-20T18:39:50.268127] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T18:39:50.345007] [INFO ] [MAIN ] EVEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:50.375684] [DEBUG] [LOCAL ] EVEX: Loaded 33091 existing records from primary storage. [2025-10-20T18:39:50.375956] [INFO ] [MAIN ] EVEX: Loaded 33091 existing records. [2025-10-20T18:39:50.555800] [WARN ] [MAIN ] EVEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:50.556013] [INFO ] [MAIN ] EVEX: Wiping existing primary data before full refresh. [2025-10-20T18:39:50.556060] [INFO ] [LOCAL ] EVEX: Starting deletion of primary data. [2025-10-20T18:39:50.556232] [DEBUG] [LOCAL ] EVEX: Deleted primary data file. [2025-10-20T18:39:50.556269] [INFO ] [MAIN ] EVEX: Starting full primary refresh. [2025-10-20T18:39:50.654483] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-03. [2025-10-20T18:39:50.910201] [INFO ] [REMOTE ] EVEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:51.451352] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-04. [2025-10-20T18:39:51.931439] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T18:39:51.939687] [INFO ] [REMOTE ] ETSY: Full fetch complete (77662 records). [2025-10-20T18:39:51.939861] [DEBUG] [VALIDATOR ] ETSY: Validating primary data. [2025-10-20T18:39:52.070283] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T18:39:52.256492] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-01. [2025-10-20T18:39:52.367293] [DEBUG] [VALIDATOR ] ETSY: All expected trading days are present. [2025-10-20T18:39:52.367409] [DEBUG] [VALIDATOR ] ETSY: Primary data validation passed. [2025-10-20T18:39:52.368405] [INFO ] [MAIN ] ETSY: Fetched primary data is valid. [2025-10-20T18:39:52.368584] [DEBUG] [LOCAL ] ETSY: Saving primary data. [2025-10-20T18:39:52.370413] [DEBUG] [LOCAL ] ETSY: Trimmed 77662 records to 53569 (keeping 200 days of data) [2025-10-20T18:39:52.501478] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T18:39:52.698451] [DEBUG] [LOCAL ] ETSY: Primary data saved (53569 records). [2025-10-20T18:39:52.698582] [INFO ] [MAIN ] ETSY: Full history saved. [2025-10-20T18:39:52.698678] [INFO ] [MAIN ] Update Progress: 782/2367 symbols completed. ETSY took 47.7s. Estimated remaining: 3h 22m 17s. Rate: 7.8 symbols/min [2025-10-20T18:39:52.864338] [INFO ] [MAIN ] EVF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:52.874430] [DEBUG] [LOCAL ] EVF: Loaded 9913 existing records from primary storage. [2025-10-20T18:39:52.874629] [INFO ] [MAIN ] EVF: Loaded 9913 existing records. [2025-10-20T18:39:53.052351] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-04. [2025-10-20T18:39:53.052238] [WARN ] [MAIN ] EVF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:53.052613] [INFO ] [MAIN ] EVF: Wiping existing primary data before full refresh. [2025-10-20T18:39:53.052703] [INFO ] [LOCAL ] EVF: Starting deletion of primary data. [2025-10-20T18:39:53.053587] [DEBUG] [LOCAL ] EVF: Deleted primary data file. [2025-10-20T18:39:53.053670] [INFO ] [MAIN ] EVF: Starting full primary refresh. [2025-10-20T18:39:53.429967] [INFO ] [REMOTE ] EVF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:53.683514] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:39:53.851431] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-01. [2025-10-20T18:39:54.314134] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T18:39:54.607180] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T18:39:54.651383] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-02. [2025-10-20T18:39:55.451356] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-05. [2025-10-20T18:39:55.813777] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T18:39:56.252846] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-03. [2025-10-20T18:39:56.322590] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T18:39:57.051422] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-02. [2025-10-20T18:39:57.422174] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T18:39:57.476286] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T18:39:57.851437] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-05. [2025-10-20T18:39:58.651389] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-06. [2025-10-20T18:39:59.189731] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:39:59.451387] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-04. [2025-10-20T18:39:59.926124] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T18:40:00.251359] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-06. [2025-10-20T18:40:00.479228] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T18:40:01.051474] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-05. [2025-10-20T18:40:01.316175] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:40:01.851362] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-07. [2025-10-20T18:40:01.879601] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:40:02.651365] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-07. [2025-10-20T18:40:02.971768] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:40:03.451348] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-08. [2025-10-20T18:40:03.766831] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:40:04.251423] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-03. [2025-10-20T18:40:04.487392] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:40:05.051389] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-08. [2025-10-20T18:40:05.281464] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T18:40:05.851370] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-04. [2025-10-20T18:40:06.146722] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:40:06.651379] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-09. [2025-10-20T18:40:06.817792] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T18:40:07.319096] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T18:40:07.451376] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-10. [2025-10-20T18:40:08.251401] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-09. [2025-10-20T18:40:08.418407] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T18:40:08.421561] [INFO ] [REMOTE ] EVC: Full fetch complete (28839 records). [2025-10-20T18:40:08.421674] [DEBUG] [VALIDATOR ] EVC: Validating primary data. [2025-10-20T18:40:08.814852] [DEBUG] [VALIDATOR ] EVC: All expected trading days are present. [2025-10-20T18:40:08.814935] [DEBUG] [VALIDATOR ] EVC: Primary data validation passed. [2025-10-20T18:40:08.815491] [INFO ] [MAIN ] EVC: Fetched primary data is valid. [2025-10-20T18:40:08.815530] [DEBUG] [LOCAL ] EVC: Saving primary data. [2025-10-20T18:40:08.816269] [DEBUG] [LOCAL ] EVC: Trimmed 28839 records to 18043 (keeping 200 days of data) [2025-10-20T18:40:08.919792] [DEBUG] [LOCAL ] EVC: Primary data saved (18043 records). [2025-10-20T18:40:08.919982] [INFO ] [MAIN ] EVC: Full history saved. [2025-10-20T18:40:08.920077] [INFO ] [MAIN ] Update Progress: 783/2367 symbols completed. EVC took 24.8s. Estimated remaining: 3h 22m 27s. Rate: 7.8 symbols/min [2025-10-20T18:40:09.056488] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-05. [2025-10-20T18:40:09.089581] [INFO ] [MAIN ] EVG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:09.097505] [DEBUG] [LOCAL ] EVG: Loaded 6021 existing records from primary storage. [2025-10-20T18:40:09.097826] [INFO ] [MAIN ] EVG: Loaded 6021 existing records. [2025-10-20T18:40:09.283264] [WARN ] [MAIN ] EVG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:09.283549] [INFO ] [MAIN ] EVG: Wiping existing primary data before full refresh. [2025-10-20T18:40:09.283599] [INFO ] [LOCAL ] EVG: Starting deletion of primary data. [2025-10-20T18:40:09.283811] [DEBUG] [LOCAL ] EVG: Deleted primary data file. [2025-10-20T18:40:09.283868] [INFO ] [MAIN ] EVG: Starting full primary refresh. [2025-10-20T18:40:09.584562] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T18:40:09.660838] [INFO ] [REMOTE ] EVG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:09.667199] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T18:40:09.851363] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-01. [2025-10-20T18:40:10.533689] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T18:40:10.651342] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-02. [2025-10-20T18:40:11.301049] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T18:40:11.451358] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-06. [2025-10-20T18:40:12.165660] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T18:40:12.251423] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-10. [2025-10-20T18:40:13.051410] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-07. [2025-10-20T18:40:13.851361] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-03. [2025-10-20T18:40:13.875901] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T18:40:14.024998] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T18:40:14.028809] [INFO ] [REMOTE ] ETY: Full fetch complete (34178 records). [2025-10-20T18:40:14.028892] [DEBUG] [VALIDATOR ] ETY: Validating primary data. [2025-10-20T18:40:14.394356] [DEBUG] [VALIDATOR ] ETY: All expected trading days are present. [2025-10-20T18:40:14.394474] [DEBUG] [VALIDATOR ] ETY: Primary data validation passed. [2025-10-20T18:40:14.395231] [INFO ] [MAIN ] ETY: Fetched primary data is valid. [2025-10-20T18:40:14.395313] [DEBUG] [LOCAL ] ETY: Saving primary data. [2025-10-20T18:40:14.396406] [DEBUG] [LOCAL ] ETY: Trimmed 34178 records to 22685 (keeping 200 days of data) [2025-10-20T18:40:14.495482] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:40:14.540403] [DEBUG] [LOCAL ] ETY: Primary data saved (22685 records). [2025-10-20T18:40:14.540529] [INFO ] [MAIN ] ETY: Full history saved. [2025-10-20T18:40:14.540602] [INFO ] [MAIN ] Update Progress: 784/2367 symbols completed. ETY took 33.2s. Estimated remaining: 3h 22m 15s. Rate: 7.8 symbols/min [2025-10-20T18:40:14.654076] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-08. [2025-10-20T18:40:14.722496] [INFO ] [MAIN ] EVH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:14.768909] [DEBUG] [LOCAL ] EVH: Loaded 50335 existing records from primary storage. [2025-10-20T18:40:14.769288] [INFO ] [MAIN ] EVH: Loaded 50335 existing records. [2025-10-20T18:40:14.953430] [WARN ] [MAIN ] EVH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:14.953662] [INFO ] [MAIN ] EVH: Wiping existing primary data before full refresh. [2025-10-20T18:40:14.953723] [INFO ] [LOCAL ] EVH: Starting deletion of primary data. [2025-10-20T18:40:14.953926] [DEBUG] [LOCAL ] EVH: Deleted primary data file. [2025-10-20T18:40:14.953983] [INFO ] [MAIN ] EVH: Starting full primary refresh. [2025-10-20T18:40:15.314552] [INFO ] [REMOTE ] EVH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:15.451338] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-06. [2025-10-20T18:40:15.581906] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:40:16.251382] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-01. [2025-10-20T18:40:16.736589] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:40:17.051358] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-07. [2025-10-20T18:40:17.851364] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-09. [2025-10-20T18:40:17.997284] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T18:40:18.450262] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T18:40:18.472585] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T18:40:18.651397] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-10. [2025-10-20T18:40:19.451397] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-04. [2025-10-20T18:40:19.656487] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T18:40:19.658907] [INFO ] [REMOTE ] EVF: Full fetch complete (12846 records). [2025-10-20T18:40:19.658989] [DEBUG] [VALIDATOR ] EVF: Validating primary data. [2025-10-20T18:40:20.036516] [DEBUG] [VALIDATOR ] EVF: All expected trading days are present. [2025-10-20T18:40:20.036598] [DEBUG] [VALIDATOR ] EVF: Primary data validation passed. [2025-10-20T18:40:20.036911] [INFO ] [MAIN ] EVF: Fetched primary data is valid. [2025-10-20T18:40:20.036971] [DEBUG] [LOCAL ] EVF: Saving primary data. [2025-10-20T18:40:20.037444] [DEBUG] [LOCAL ] EVF: Trimmed 12846 records to 9820 (keeping 200 days of data) [2025-10-20T18:40:20.094921] [DEBUG] [LOCAL ] EVF: Primary data saved (9820 records). [2025-10-20T18:40:20.095049] [INFO ] [MAIN ] EVF: Full history saved. [2025-10-20T18:40:20.095128] [INFO ] [MAIN ] Update Progress: 785/2367 symbols completed. EVF took 27.4s. Estimated remaining: 3h 22m 3s. Rate: 7.8 symbols/min [2025-10-20T18:40:20.267305] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-08. [2025-10-20T18:40:20.271337] [INFO ] [MAIN ] EVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:20.285238] [DEBUG] [LOCAL ] EVN: Loaded 10887 existing records from primary storage. [2025-10-20T18:40:20.285462] [INFO ] [MAIN ] EVN: Loaded 10887 existing records. [2025-10-20T18:40:20.392858] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T18:40:20.475493] [WARN ] [MAIN ] EVN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:20.475820] [INFO ] [MAIN ] EVN: Wiping existing primary data before full refresh. [2025-10-20T18:40:20.475875] [INFO ] [LOCAL ] EVN: Starting deletion of primary data. [2025-10-20T18:40:20.476111] [DEBUG] [LOCAL ] EVN: Deleted primary data file. [2025-10-20T18:40:20.476196] [INFO ] [MAIN ] EVN: Starting full primary refresh. [2025-10-20T18:40:20.826104] [INFO ] [REMOTE ] EVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:21.051347] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-01. [2025-10-20T18:40:21.857176] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-02. [2025-10-20T18:40:21.868239] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T18:40:22.161502] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:40:22.651367] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-05. [2025-10-20T18:40:23.245155] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:40:23.430010] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:40:23.451323] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-03. [2025-10-20T18:40:24.251413] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-09. [2025-10-20T18:40:25.051361] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-06. [2025-10-20T18:40:25.365150] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T18:40:25.648154] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T18:40:25.851374] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-04. [2025-10-20T18:40:25.940535] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:40:26.651406] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-10. [2025-10-20T18:40:27.352083] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T18:40:27.451372] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-05. [2025-10-20T18:40:28.251364] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-02. [2025-10-20T18:40:28.325505] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T18:40:28.332524] [INFO ] [REMOTE ] EVEX: Full fetch complete (43835 records). [2025-10-20T18:40:28.332633] [DEBUG] [VALIDATOR ] EVEX: Validating primary data. [2025-10-20T18:40:28.698843] [DEBUG] [VALIDATOR ] EVEX: All expected trading days are present. [2025-10-20T18:40:28.698978] [DEBUG] [VALIDATOR ] EVEX: Primary data validation passed. [2025-10-20T18:40:28.699736] [INFO ] [MAIN ] EVEX: Fetched primary data is valid. [2025-10-20T18:40:28.699804] [DEBUG] [LOCAL ] EVEX: Saving primary data. [2025-10-20T18:40:28.700782] [DEBUG] [LOCAL ] EVEX: Trimmed 43835 records to 32891 (keeping 200 days of data) [2025-10-20T18:40:28.879922] [DEBUG] [LOCAL ] EVEX: Primary data saved (32891 records). [2025-10-20T18:40:28.880090] [INFO ] [MAIN ] EVEX: Full history saved. [2025-10-20T18:40:28.880250] [INFO ] [MAIN ] Update Progress: 786/2367 symbols completed. EVEX took 38.7s. Estimated remaining: 3h 21m 58s. Rate: 7.8 symbols/min [2025-10-20T18:40:29.081486] [INFO ] [MAIN ] EVR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:29.079118] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-07. [2025-10-20T18:40:29.085179] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T18:40:29.103759] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T18:40:29.136838] [DEBUG] [LOCAL ] EVR: Loaded 34821 existing records from primary storage. [2025-10-20T18:40:29.137139] [INFO ] [MAIN ] EVR: Loaded 34821 existing records. [2025-10-20T18:40:29.315848] [WARN ] [MAIN ] EVR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:29.316097] [INFO ] [MAIN ] EVR: Wiping existing primary data before full refresh. [2025-10-20T18:40:29.316146] [INFO ] [LOCAL ] EVR: Starting deletion of primary data. [2025-10-20T18:40:29.316351] [DEBUG] [LOCAL ] EVR: Deleted primary data file. [2025-10-20T18:40:29.316389] [INFO ] [MAIN ] EVR: Starting full primary refresh. [2025-10-20T18:40:29.679551] [INFO ] [REMOTE ] EVR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:29.691633] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:40:29.851370] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-01. [2025-10-20T18:40:30.651382] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-06. [2025-10-20T18:40:31.134084] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T18:40:31.451431] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-03. [2025-10-20T18:40:32.263737] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-08. [2025-10-20T18:40:32.270820] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:40:32.466076] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T18:40:32.999273] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T18:40:33.051429] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-07. [2025-10-20T18:40:33.851481] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-02. [2025-10-20T18:40:34.511755] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:40:34.651412] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-08. [2025-10-20T18:40:35.395972] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:40:35.451400] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-03. [2025-10-20T18:40:35.859195] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T18:40:36.251407] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-04. [2025-10-20T18:40:36.562997] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T18:40:37.051425] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-04. [2025-10-20T18:40:37.142539] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T18:40:37.851435] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-05. [2025-10-20T18:40:38.437004] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:40:38.651371] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-09. [2025-10-20T18:40:38.773870] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T18:40:39.451394] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-06. [2025-10-20T18:40:40.256823] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-09. [2025-10-20T18:40:40.276702] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:40:40.305456] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T18:40:40.950402] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T18:40:41.051372] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-10. [2025-10-20T18:40:41.851387] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-05. [2025-10-20T18:40:42.651434] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-07. [2025-10-20T18:40:42.877513] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:40:42.886084] [INFO ] [REMOTE ] EVH: Full fetch complete (71841 records). [2025-10-20T18:40:42.886346] [DEBUG] [VALIDATOR ] EVH: Validating primary data. [2025-10-20T18:40:43.341023] [DEBUG] [VALIDATOR ] EVH: All expected trading days are present. [2025-10-20T18:40:43.341258] [DEBUG] [VALIDATOR ] EVH: Primary data validation passed. [2025-10-20T18:40:43.343019] [INFO ] [MAIN ] EVH: Fetched primary data is valid. [2025-10-20T18:40:43.343292] [DEBUG] [LOCAL ] EVH: Saving primary data. [2025-10-20T18:40:43.346178] [DEBUG] [LOCAL ] EVH: Trimmed 71841 records to 49646 (keeping 200 days of data) [2025-10-20T18:40:43.452135] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-10. [2025-10-20T18:40:43.547276] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:40:43.625973] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T18:40:43.687714] [DEBUG] [LOCAL ] EVH: Primary data saved (49646 records). [2025-10-20T18:40:43.687885] [INFO ] [MAIN ] EVH: Full history saved. [2025-10-20T18:40:43.687981] [INFO ] [MAIN ] Update Progress: 787/2367 symbols completed. EVH took 29.1s. Estimated remaining: 3h 22m 5s. Rate: 7.8 symbols/min [2025-10-20T18:40:43.853083] [INFO ] [MAIN ] EVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:43.870258] [DEBUG] [LOCAL ] EVT: Loaded 17008 existing records from primary storage. [2025-10-20T18:40:43.870477] [INFO ] [MAIN ] EVT: Loaded 17008 existing records. [2025-10-20T18:40:44.042788] [WARN ] [MAIN ] EVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:44.043016] [INFO ] [MAIN ] EVT: Wiping existing primary data before full refresh. [2025-10-20T18:40:44.043061] [INFO ] [LOCAL ] EVT: Starting deletion of primary data. [2025-10-20T18:40:44.043255] [DEBUG] [LOCAL ] EVT: Deleted primary data file. [2025-10-20T18:40:44.043310] [INFO ] [MAIN ] EVT: Starting full primary refresh. [2025-10-20T18:40:44.256627] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-08. [2025-10-20T18:40:44.332469] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T18:40:44.345526] [INFO ] [REMOTE ] EVG: Full fetch complete (8146 records). [2025-10-20T18:40:44.361146] [DEBUG] [VALIDATOR ] EVG: Validating primary data. [2025-10-20T18:40:44.519704] [INFO ] [REMOTE ] EVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:44.803577] [DEBUG] [VALIDATOR ] EVG: All expected trading days are present. [2025-10-20T18:40:44.803677] [DEBUG] [VALIDATOR ] EVG: Primary data validation passed. [2025-10-20T18:40:44.803889] [INFO ] [MAIN ] EVG: Fetched primary data is valid. [2025-10-20T18:40:44.803930] [DEBUG] [LOCAL ] EVG: Saving primary data. [2025-10-20T18:40:44.804384] [DEBUG] [LOCAL ] EVG: Trimmed 8146 records to 5867 (keeping 200 days of data) [2025-10-20T18:40:44.840223] [DEBUG] [LOCAL ] EVG: Primary data saved (5867 records). [2025-10-20T18:40:44.840321] [INFO ] [MAIN ] EVG: Full history saved. [2025-10-20T18:40:44.840399] [INFO ] [MAIN ] Update Progress: 788/2367 symbols completed. EVG took 35.9s. Estimated remaining: 3h 21m 44s. Rate: 7.8 symbols/min [2025-10-20T18:40:45.006106] [INFO ] [MAIN ] EVTC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:45.032076] [DEBUG] [LOCAL ] EVTC: Loaded 27241 existing records from primary storage. [2025-10-20T18:40:45.032368] [INFO ] [MAIN ] EVTC: Loaded 27241 existing records. [2025-10-20T18:40:45.061280] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-01. [2025-10-20T18:40:45.217367] [WARN ] [MAIN ] EVTC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:45.217622] [INFO ] [MAIN ] EVTC: Wiping existing primary data before full refresh. [2025-10-20T18:40:45.217709] [INFO ] [LOCAL ] EVTC: Starting deletion of primary data. [2025-10-20T18:40:45.217891] [DEBUG] [LOCAL ] EVTC: Deleted primary data file. [2025-10-20T18:40:45.217930] [INFO ] [MAIN ] EVTC: Starting full primary refresh. [2025-10-20T18:40:45.498098] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T18:40:45.587805] [INFO ] [REMOTE ] EVTC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:45.851380] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-01. [2025-10-20T18:40:46.505196] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:40:46.651381] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-02. [2025-10-20T18:40:47.110022] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:40:47.451398] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-02. [2025-10-20T18:40:47.630921] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:40:48.251378] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-03. [2025-10-20T18:40:48.490302] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:40:49.051389] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-03. [2025-10-20T18:40:49.229184] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T18:40:49.851395] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-06. [2025-10-20T18:40:50.369457] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:40:50.651367] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-04. [2025-10-20T18:40:51.451342] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-04. [2025-10-20T18:40:51.778735] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T18:40:52.251390] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-09. [2025-10-20T18:40:52.370887] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T18:40:52.675185] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T18:40:53.051390] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-05. [2025-10-20T18:40:53.723602] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T18:40:53.851396] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-05. [2025-10-20T18:40:54.105909] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T18:40:54.651412] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-07. [2025-10-20T18:40:54.784938] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:40:55.451369] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-06. [2025-10-20T18:40:56.040361] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:40:56.251401] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-10. [2025-10-20T18:40:56.373136] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T18:40:57.051377] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-06. [2025-10-20T18:40:57.092734] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:40:57.094782] [INFO ] [REMOTE ] EVN: Full fetch complete (17068 records). [2025-10-20T18:40:57.094856] [DEBUG] [VALIDATOR ] EVN: Validating primary data. [2025-10-20T18:40:57.464112] [DEBUG] [VALIDATOR ] EVN: All expected trading days are present. [2025-10-20T18:40:57.464188] [DEBUG] [VALIDATOR ] EVN: Primary data validation passed. [2025-10-20T18:40:57.464540] [INFO ] [MAIN ] EVN: Fetched primary data is valid. [2025-10-20T18:40:57.464579] [DEBUG] [LOCAL ] EVN: Saving primary data. [2025-10-20T18:40:57.465114] [DEBUG] [LOCAL ] EVN: Trimmed 17068 records to 10724 (keeping 200 days of data) [2025-10-20T18:40:57.528001] [DEBUG] [LOCAL ] EVN: Primary data saved (10724 records). [2025-10-20T18:40:57.528128] [INFO ] [MAIN ] EVN: Full history saved. [2025-10-20T18:40:57.528205] [INFO ] [MAIN ] Update Progress: 789/2367 symbols completed. EVN took 37.4s. Estimated remaining: 3h 21m 46s. Rate: 7.8 symbols/min [2025-10-20T18:40:57.697836] [INFO ] [MAIN ] EVTL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:57.735485] [DEBUG] [LOCAL ] EVTL: Loaded 41058 existing records from primary storage. [2025-10-20T18:40:57.735796] [INFO ] [MAIN ] EVTL: Loaded 41058 existing records. [2025-10-20T18:40:57.851704] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-07. [2025-10-20T18:40:57.921033] [WARN ] [MAIN ] EVTL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:57.921255] [INFO ] [MAIN ] EVTL: Wiping existing primary data before full refresh. [2025-10-20T18:40:57.921311] [INFO ] [LOCAL ] EVTL: Starting deletion of primary data. [2025-10-20T18:40:57.921699] [DEBUG] [LOCAL ] EVTL: Deleted primary data file. [2025-10-20T18:40:57.921755] [INFO ] [MAIN ] EVTL: Starting full primary refresh. [2025-10-20T18:40:58.245341] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T18:40:58.319706] [INFO ] [REMOTE ] EVTL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:58.651341] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-01. [2025-10-20T18:40:58.954334] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:40:59.451381] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-08. [2025-10-20T18:41:00.128773] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:41:00.251408] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-08. [2025-10-20T18:41:00.638181] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T18:41:00.836057] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T18:41:01.051375] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-07. [2025-10-20T18:41:01.851407] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-09. [2025-10-20T18:41:02.337800] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T18:41:02.651389] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-09. [2025-10-20T18:41:03.112024] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T18:41:03.442528] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:41:03.451328] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-10. [2025-10-20T18:41:04.251400] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-02. [2025-10-20T18:41:04.475112] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T18:41:04.477694] [INFO ] [REMOTE ] EVT: Full fetch complete (25197 records). [2025-10-20T18:41:04.477763] [DEBUG] [VALIDATOR ] EVT: Validating primary data. [2025-10-20T18:41:04.847079] [DEBUG] [VALIDATOR ] EVT: All expected trading days are present. [2025-10-20T18:41:04.847216] [DEBUG] [VALIDATOR ] EVT: Primary data validation passed. [2025-10-20T18:41:04.847760] [INFO ] [MAIN ] EVT: Fetched primary data is valid. [2025-10-20T18:41:04.847822] [DEBUG] [LOCAL ] EVT: Saving primary data. [2025-10-20T18:41:04.848806] [DEBUG] [LOCAL ] EVT: Trimmed 25197 records to 16762 (keeping 200 days of data) [2025-10-20T18:41:04.948855] [DEBUG] [LOCAL ] EVT: Primary data saved (16762 records). [2025-10-20T18:41:04.949004] [INFO ] [MAIN ] EVT: Full history saved. [2025-10-20T18:41:04.949082] [INFO ] [MAIN ] Update Progress: 790/2367 symbols completed. EVT took 21.3s. Estimated remaining: 3h 21m 38s. Rate: 7.8 symbols/min [2025-10-20T18:41:05.056571] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-08. [2025-10-20T18:41:05.116951] [INFO ] [MAIN ] EVTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:05.154680] [DEBUG] [LOCAL ] EVTR: Loaded 25284 existing records from primary storage. [2025-10-20T18:41:05.154956] [INFO ] [MAIN ] EVTR: Loaded 25284 existing records. [2025-10-20T18:41:05.331764] [WARN ] [MAIN ] EVTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:05.332029] [INFO ] [MAIN ] EVTR: Wiping existing primary data before full refresh. [2025-10-20T18:41:05.332075] [INFO ] [LOCAL ] EVTR: Starting deletion of primary data. [2025-10-20T18:41:05.332320] [DEBUG] [LOCAL ] EVTR: Deleted primary data file. [2025-10-20T18:41:05.332375] [INFO ] [MAIN ] EVTR: Starting full primary refresh. [2025-10-20T18:41:05.696287] [INFO ] [REMOTE ] EVTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:05.717307] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T18:41:05.827976] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T18:41:05.851338] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-01. [2025-10-20T18:41:06.651376] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-03. [2025-10-20T18:41:07.055742] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T18:41:07.451378] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-02. [2025-10-20T18:41:07.968318] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:41:08.251343] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-04. [2025-10-20T18:41:08.410348] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:41:09.051358] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-09. [2025-10-20T18:41:09.302300] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T18:41:09.851340] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-05. [2025-10-20T18:41:10.309625] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T18:41:10.651359] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-10. [2025-10-20T18:41:11.314884] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T18:41:11.451429] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-03. [2025-10-20T18:41:11.886346] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T18:41:11.891108] [INFO ] [REMOTE ] EVTC: Full fetch complete (38420 records). [2025-10-20T18:41:11.891194] [DEBUG] [VALIDATOR ] EVTC: Validating primary data. [2025-10-20T18:41:12.252896] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-06. [2025-10-20T18:41:12.286156] [DEBUG] [VALIDATOR ] EVTC: All expected trading days are present. [2025-10-20T18:41:12.286301] [DEBUG] [VALIDATOR ] EVTC: Primary data validation passed. [2025-10-20T18:41:12.287066] [INFO ] [MAIN ] EVTC: Fetched primary data is valid. [2025-10-20T18:41:12.287129] [DEBUG] [LOCAL ] EVTC: Saving primary data. [2025-10-20T18:41:12.287992] [DEBUG] [LOCAL ] EVTC: Trimmed 38420 records to 26895 (keeping 200 days of data) [2025-10-20T18:41:12.472011] [DEBUG] [LOCAL ] EVTC: Primary data saved (26895 records). [2025-10-20T18:41:12.472754] [INFO ] [MAIN ] EVTC: Full history saved. [2025-10-20T18:41:12.473002] [INFO ] [MAIN ] Update Progress: 791/2367 symbols completed. EVTC took 27.6s. Estimated remaining: 3h 21m 30s. Rate: 7.8 symbols/min [2025-10-20T18:41:12.473510] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T18:41:12.694587] [INFO ] [MAIN ] EW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:12.747958] [DEBUG] [LOCAL ] EW: Loaded 54179 existing records from primary storage. [2025-10-20T18:41:12.748276] [INFO ] [MAIN ] EW: Loaded 54179 existing records. [2025-10-20T18:41:12.920769] [WARN ] [MAIN ] EW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:12.920964] [INFO ] [MAIN ] EW: Wiping existing primary data before full refresh. [2025-10-20T18:41:12.921008] [INFO ] [LOCAL ] EW: Starting deletion of primary data. [2025-10-20T18:41:12.921211] [DEBUG] [LOCAL ] EW: Deleted primary data file. [2025-10-20T18:41:12.921267] [INFO ] [MAIN ] EW: Starting full primary refresh. [2025-10-20T18:41:13.056614] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-04. [2025-10-20T18:41:13.268363] [INFO ] [REMOTE ] EW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:13.803542] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T18:41:13.851358] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-10. [2025-10-20T18:41:14.071466] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T18:41:14.651386] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-07. [2025-10-20T18:41:15.345583] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T18:41:15.350333] [INFO ] [REMOTE ] EVR: Full fetch complete (50835 records). [2025-10-20T18:41:15.350456] [DEBUG] [VALIDATOR ] EVR: Validating primary data. [2025-10-20T18:41:15.456580] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-01. [2025-10-20T18:41:15.733606] [DEBUG] [VALIDATOR ] EVR: All expected trading days are present. [2025-10-20T18:41:15.733952] [DEBUG] [VALIDATOR ] EVR: Primary data validation passed. [2025-10-20T18:41:15.734692] [INFO ] [MAIN ] EVR: Fetched primary data is valid. [2025-10-20T18:41:15.734755] [DEBUG] [LOCAL ] EVR: Saving primary data. [2025-10-20T18:41:15.735922] [DEBUG] [LOCAL ] EVR: Trimmed 50835 records to 34068 (keeping 200 days of data) [2025-10-20T18:41:15.941668] [DEBUG] [LOCAL ] EVR: Primary data saved (34068 records). [2025-10-20T18:41:15.941800] [INFO ] [MAIN ] EVR: Full history saved. [2025-10-20T18:41:15.941874] [INFO ] [MAIN ] Update Progress: 792/2367 symbols completed. EVR took 47.1s. Estimated remaining: 3h 21m 14s. Rate: 7.8 symbols/min [2025-10-20T18:41:16.106363] [INFO ] [MAIN ] EXG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:16.138192] [DEBUG] [LOCAL ] EXG: Loaded 34518 existing records from primary storage. [2025-10-20T18:41:16.138509] [INFO ] [MAIN ] EXG: Loaded 34518 existing records. [2025-10-20T18:41:16.257305] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-05. [2025-10-20T18:41:16.316431] [WARN ] [MAIN ] EXG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:16.316623] [INFO ] [MAIN ] EXG: Wiping existing primary data before full refresh. [2025-10-20T18:41:16.316700] [INFO ] [LOCAL ] EXG: Starting deletion of primary data. [2025-10-20T18:41:16.316979] [DEBUG] [LOCAL ] EXG: Deleted primary data file. [2025-10-20T18:41:16.317036] [INFO ] [MAIN ] EXG: Starting full primary refresh. [2025-10-20T18:41:16.688735] [INFO ] [REMOTE ] EXG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:16.758129] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T18:41:17.051343] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-08. [2025-10-20T18:41:17.217143] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T18:41:17.413014] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T18:41:17.851389] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-02. [2025-10-20T18:41:18.651528] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-06. [2025-10-20T18:41:18.819753] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:41:19.451341] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-09. [2025-10-20T18:41:19.569010] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T18:41:19.603223] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T18:41:20.251396] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-07. [2025-10-20T18:41:21.051406] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-01. [2025-10-20T18:41:21.243412] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T18:41:21.277234] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T18:41:21.851405] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-10. [2025-10-20T18:41:22.067347] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:41:22.651440] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-03. [2025-10-20T18:41:23.451421] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-08. [2025-10-20T18:41:24.078584] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-10 (2.23 seconds). [2025-10-20T18:41:24.090538] [INFO ] [REMOTE ] EVTL: Full fetch complete (54651 records). [2025-10-20T18:41:24.090718] [DEBUG] [VALIDATOR ] EVTL: Validating primary data. [2025-10-20T18:41:24.256040] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-02. [2025-10-20T18:41:24.529088] [DEBUG] [VALIDATOR ] EVTL: All expected trading days are present. [2025-10-20T18:41:24.529676] [DEBUG] [VALIDATOR ] EVTL: Primary data validation passed. [2025-10-20T18:41:24.532114] [INFO ] [MAIN ] EVTL: Fetched primary data is valid. [2025-10-20T18:41:24.531144] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T18:41:24.532358] [DEBUG] [LOCAL ] EVTL: Saving primary data. [2025-10-20T18:41:24.561117] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T18:41:24.568089] [DEBUG] [LOCAL ] EVTL: Trimmed 54651 records to 41005 (keeping 200 days of data) [2025-10-20T18:41:24.802930] [DEBUG] [LOCAL ] EVTL: Primary data saved (41005 records). [2025-10-20T18:41:24.803050] [INFO ] [MAIN ] EVTL: Full history saved. [2025-10-20T18:41:24.803121] [INFO ] [MAIN ] Update Progress: 793/2367 symbols completed. EVTL took 27.3s. Estimated remaining: 3h 21m 9s. Rate: 7.8 symbols/min [2025-10-20T18:41:24.975572] [INFO ] [MAIN ] EXK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:25.026700] [DEBUG] [LOCAL ] EXK: Loaded 53951 existing records from primary storage. [2025-10-20T18:41:25.027021] [INFO ] [MAIN ] EXK: Loaded 53951 existing records. [2025-10-20T18:41:25.057317] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-04. [2025-10-20T18:41:25.221313] [WARN ] [MAIN ] EXK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:25.221540] [INFO ] [MAIN ] EXK: Wiping existing primary data before full refresh. [2025-10-20T18:41:25.221590] [INFO ] [LOCAL ] EXK: Starting deletion of primary data. [2025-10-20T18:41:25.222137] [DEBUG] [LOCAL ] EXK: Deleted primary data file. [2025-10-20T18:41:25.222195] [INFO ] [MAIN ] EXK: Starting full primary refresh. [2025-10-20T18:41:25.735840] [INFO ] [REMOTE ] EXK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:25.817935] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:41:25.851420] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-03. [2025-10-20T18:41:26.651400] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-09. [2025-10-20T18:41:26.962775] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T18:41:27.042083] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T18:41:27.451413] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-05. [2025-10-20T18:41:28.052155] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T18:41:28.251423] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-10. [2025-10-20T18:41:29.051427] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-04. [2025-10-20T18:41:29.284723] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T18:41:29.649334] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T18:41:29.654627] [INFO ] [REMOTE ] EVTR: Full fetch complete (33630 records). [2025-10-20T18:41:29.654763] [DEBUG] [VALIDATOR ] EVTR: Validating primary data. [2025-10-20T18:41:29.851510] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-01. [2025-10-20T18:41:30.049888] [DEBUG] [VALIDATOR ] EVTR: All expected trading days are present. [2025-10-20T18:41:30.049969] [DEBUG] [VALIDATOR ] EVTR: Primary data validation passed. [2025-10-20T18:41:30.050534] [INFO ] [MAIN ] EVTR: Fetched primary data is valid. [2025-10-20T18:41:30.050572] [DEBUG] [LOCAL ] EVTR: Saving primary data. [2025-10-20T18:41:30.051501] [DEBUG] [LOCAL ] EVTR: Trimmed 33630 records to 25188 (keeping 200 days of data) [2025-10-20T18:41:30.202963] [DEBUG] [LOCAL ] EVTR: Primary data saved (25188 records). [2025-10-20T18:41:30.203102] [INFO ] [MAIN ] EVTR: Full history saved. [2025-10-20T18:41:30.203182] [INFO ] [MAIN ] Update Progress: 794/2367 symbols completed. EVTR took 25.3s. Estimated remaining: 3h 20m 57s. Rate: 7.8 symbols/min [2025-10-20T18:41:30.380423] [INFO ] [MAIN ] EXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:30.405472] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T18:41:30.439343] [DEBUG] [LOCAL ] EXP: Loaded 29985 existing records from primary storage. [2025-10-20T18:41:30.439603] [INFO ] [MAIN ] EXP: Loaded 29985 existing records. [2025-10-20T18:41:30.620238] [WARN ] [MAIN ] EXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:30.620410] [INFO ] [MAIN ] EXP: Wiping existing primary data before full refresh. [2025-10-20T18:41:30.620459] [INFO ] [LOCAL ] EXP: Starting deletion of primary data. [2025-10-20T18:41:30.620628] [DEBUG] [LOCAL ] EXP: Deleted primary data file. [2025-10-20T18:41:30.620713] [INFO ] [MAIN ] EXP: Starting full primary refresh. [2025-10-20T18:41:30.656529] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-05. [2025-10-20T18:41:30.973884] [INFO ] [REMOTE ] EXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:31.451372] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-06. [2025-10-20T18:41:31.824989] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T18:41:32.067882] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T18:41:32.251384] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-06. [2025-10-20T18:41:33.051382] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-02. [2025-10-20T18:41:33.287568] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T18:41:33.795450] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T18:41:33.851380] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-07. [2025-10-20T18:41:34.651363] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-07. [2025-10-20T18:41:34.896778] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T18:41:34.972980] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:41:35.451383] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-08. [2025-10-20T18:41:36.251396] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-03. [2025-10-20T18:41:36.520053] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:41:36.701408] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T18:41:37.051364] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-09. [2025-10-20T18:41:37.851365] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-08. [2025-10-20T18:41:38.142061] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:41:38.207507] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T18:41:38.651384] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-04. [2025-10-20T18:41:39.451358] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-01. [2025-10-20T18:41:39.734098] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T18:41:40.251404] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-10. [2025-10-20T18:41:40.476277] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T18:41:40.990426] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:41:41.051339] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-02. [2025-10-20T18:41:41.613428] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T18:41:41.618290] [INFO ] [REMOTE ] EXG: Full fetch complete (49749 records). [2025-10-20T18:41:41.618383] [DEBUG] [VALIDATOR ] EXG: Validating primary data. [2025-10-20T18:41:41.858017] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-05. [2025-10-20T18:41:41.994118] [DEBUG] [VALIDATOR ] EXG: All expected trading days are present. [2025-10-20T18:41:41.994196] [DEBUG] [VALIDATOR ] EXG: Primary data validation passed. [2025-10-20T18:41:41.994974] [INFO ] [MAIN ] EXG: Fetched primary data is valid. [2025-10-20T18:41:41.995038] [DEBUG] [LOCAL ] EXG: Saving primary data. [2025-10-20T18:41:41.996103] [DEBUG] [LOCAL ] EXG: Trimmed 49749 records to 33887 (keeping 200 days of data) [2025-10-20T18:41:42.199298] [DEBUG] [LOCAL ] EXG: Primary data saved (33887 records). [2025-10-20T18:41:42.199453] [INFO ] [MAIN ] EXG: Full history saved. [2025-10-20T18:41:42.199538] [INFO ] [MAIN ] Update Progress: 795/2367 symbols completed. EXG took 26.3s. Estimated remaining: 3h 20m 58s. Rate: 7.8 symbols/min [2025-10-20T18:41:42.377664] [INFO ] [MAIN ] EXPD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:42.421773] [DEBUG] [LOCAL ] EXPD: Loaded 48595 existing records from primary storage. [2025-10-20T18:41:42.422077] [INFO ] [MAIN ] EXPD: Loaded 48595 existing records. [2025-10-20T18:41:42.602784] [WARN ] [MAIN ] EXPD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:42.609218] [INFO ] [MAIN ] EXPD: Wiping existing primary data before full refresh. [2025-10-20T18:41:42.618217] [INFO ] [LOCAL ] EXPD: Starting deletion of primary data. [2025-10-20T18:41:42.618954] [DEBUG] [LOCAL ] EXPD: Deleted primary data file. [2025-10-20T18:41:42.619695] [INFO ] [MAIN ] EXPD: Starting full primary refresh. [2025-10-20T18:41:42.619602] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:41:42.653694] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-09. [2025-10-20T18:41:42.985732] [INFO ] [REMOTE ] EXPD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:43.451399] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-01. [2025-10-20T18:41:43.768455] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T18:41:44.251359] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-06. [2025-10-20T18:41:44.297782] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T18:41:45.051373] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-03. [2025-10-20T18:41:45.113581] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T18:41:45.851362] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-10. [2025-10-20T18:41:46.070491] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T18:41:46.492344] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T18:41:46.587127] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T18:41:46.598503] [INFO ] [REMOTE ] EW: Full fetch complete (77515 records). [2025-10-20T18:41:46.598715] [DEBUG] [VALIDATOR ] EW: Validating primary data. [2025-10-20T18:41:46.656446] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-04. [2025-10-20T18:41:46.998439] [DEBUG] [VALIDATOR ] EW: All expected trading days are present. [2025-10-20T18:41:46.998521] [DEBUG] [VALIDATOR ] EW: Primary data validation passed. [2025-10-20T18:41:46.999451] [INFO ] [MAIN ] EW: Fetched primary data is valid. [2025-10-20T18:41:46.999522] [DEBUG] [LOCAL ] EW: Saving primary data. [2025-10-20T18:41:47.000986] [DEBUG] [LOCAL ] EW: Trimmed 77515 records to 53400 (keeping 200 days of data) [2025-10-20T18:41:47.313103] [DEBUG] [LOCAL ] EW: Primary data saved (53400 records). [2025-10-20T18:41:47.313250] [INFO ] [MAIN ] EW: Full history saved. [2025-10-20T18:41:47.313603] [INFO ] [MAIN ] Update Progress: 796/2367 symbols completed. EW took 34.8s. Estimated remaining: 3h 20m 45s. Rate: 7.8 symbols/min [2025-10-20T18:41:47.456466] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-07. [2025-10-20T18:41:47.511962] [INFO ] [MAIN ] EXR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:47.556572] [DEBUG] [LOCAL ] EXR: Loaded 46249 existing records from primary storage. [2025-10-20T18:41:47.556906] [INFO ] [MAIN ] EXR: Loaded 46249 existing records. [2025-10-20T18:41:47.737327] [WARN ] [MAIN ] EXR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:47.737672] [INFO ] [MAIN ] EXR: Wiping existing primary data before full refresh. [2025-10-20T18:41:47.737779] [INFO ] [LOCAL ] EXR: Starting deletion of primary data. [2025-10-20T18:41:47.738063] [DEBUG] [LOCAL ] EXR: Deleted primary data file. [2025-10-20T18:41:47.738117] [INFO ] [MAIN ] EXR: Starting full primary refresh. [2025-10-20T18:41:47.933902] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-10-20T18:41:48.119210] [INFO ] [REMOTE ] EXR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:48.251369] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-01. [2025-10-20T18:41:49.051408] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-02. [2025-10-20T18:41:49.526358] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T18:41:49.851419] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-08. [2025-10-20T18:41:49.967318] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T18:41:50.545126] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:41:50.651356] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-02. [2025-10-20T18:41:51.451428] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-03. [2025-10-20T18:41:51.708410] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T18:41:52.186278] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:41:52.251339] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-03. [2025-10-20T18:41:53.051404] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-09. [2025-10-20T18:41:53.359976] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T18:41:53.851378] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-04. [2025-10-20T18:41:53.904257] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:41:54.651425] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-05. [2025-10-20T18:41:54.728169] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:41:55.296962] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T18:41:55.451380] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-04. [2025-10-20T18:41:56.090156] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:41:56.251421] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-10. [2025-10-20T18:41:57.051399] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-05. [2025-10-20T18:41:57.152605] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:41:57.851351] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-06. [2025-10-20T18:41:58.651421] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-05. [2025-10-20T18:41:58.677430] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-10 (2.43 seconds). [2025-10-20T18:41:58.685423] [INFO ] [REMOTE ] EXK: Full fetch complete (76809 records). [2025-10-20T18:41:58.685626] [DEBUG] [VALIDATOR ] EXK: Validating primary data. [2025-10-20T18:41:59.043158] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T18:41:59.128929] [DEBUG] [VALIDATOR ] EXK: All expected trading days are present. [2025-10-20T18:41:59.129075] [DEBUG] [VALIDATOR ] EXK: Primary data validation passed. [2025-10-20T18:41:59.130083] [INFO ] [MAIN ] EXK: Fetched primary data is valid. [2025-10-20T18:41:59.130148] [DEBUG] [LOCAL ] EXK: Saving primary data. [2025-10-20T18:41:59.131635] [DEBUG] [LOCAL ] EXK: Trimmed 76809 records to 53185 (keeping 200 days of data) [2025-10-20T18:41:59.342282] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T18:41:59.451924] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-07. [2025-10-20T18:41:59.464582] [DEBUG] [LOCAL ] EXK: Primary data saved (53185 records). [2025-10-20T18:41:59.464790] [INFO ] [MAIN ] EXK: Full history saved. [2025-10-20T18:41:59.465885] [INFO ] [MAIN ] Update Progress: 797/2367 symbols completed. EXK took 34.7s. Estimated remaining: 3h 20m 46s. Rate: 7.8 symbols/min [2025-10-20T18:42:00.251356] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-06. [2025-10-20T18:42:00.390685] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:42:00.729605] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:42:01.051366] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-08. [2025-10-20T18:42:01.851505] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-06. [2025-10-20T18:42:01.967769] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:42:02.034485] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:42:02.651377] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-09. [2025-10-20T18:42:03.406974] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T18:42:03.451366] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-07. [2025-10-20T18:42:04.102875] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T18:42:04.251367] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-10. [2025-10-20T18:42:05.051418] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-07. [2025-10-20T18:42:05.298549] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:42:05.525267] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T18:42:05.529704] [INFO ] [REMOTE ] EXP: Full fetch complete (42951 records). [2025-10-20T18:42:05.529849] [DEBUG] [VALIDATOR ] EXP: Validating primary data. [2025-10-20T18:42:05.856519] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-08. [2025-10-20T18:42:05.928209] [DEBUG] [VALIDATOR ] EXP: All expected trading days are present. [2025-10-20T18:42:05.928284] [DEBUG] [VALIDATOR ] EXP: Primary data validation passed. [2025-10-20T18:42:05.928942] [INFO ] [MAIN ] EXP: Fetched primary data is valid. [2025-10-20T18:42:05.929025] [DEBUG] [LOCAL ] EXP: Saving primary data. [2025-10-20T18:42:05.930075] [DEBUG] [LOCAL ] EXP: Trimmed 42951 records to 29414 (keeping 200 days of data) [2025-10-20T18:42:06.107468] [DEBUG] [LOCAL ] EXP: Primary data saved (29414 records). [2025-10-20T18:42:06.107633] [INFO ] [MAIN ] EXP: Full history saved. [2025-10-20T18:42:06.107752] [INFO ] [MAIN ] Update Progress: 798/2367 symbols completed. EXP took 35.9s. Estimated remaining: 3h 20m 36s. Rate: 7.8 symbols/min [2025-10-20T18:42:06.795808] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T18:42:06.806037] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-08. [2025-10-20T18:42:07.625091] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:42:07.639895] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-09. [2025-10-20T18:42:08.240497] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T18:42:08.252541] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-09. [2025-10-20T18:42:09.354459] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T18:42:09.364601] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-10. [2025-10-20T18:42:09.742164] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-10-20T18:42:09.748801] [INFO ] [REMOTE ] EXPD: Full fetch complete (69216 records). [2025-10-20T18:42:09.748946] [DEBUG] [VALIDATOR ] EXPD: Validating primary data. [2025-10-20T18:42:09.749775] [FATAL] [VALIDATOR ] EXPD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:42:09.749872] [FATAL] [MAIN ] EXPD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:42:09.749916] [WARN ] [MAIN ] EXPD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:42:09.749985] [INFO ] [LOCAL ] EXPD: Starting deletion of primary data. [2025-10-20T18:42:09.750051] [DEBUG] [LOCAL ] EXPD: No primary data file found to delete. [2025-10-20T18:42:09.750088] [INFO ] [LOCAL ] EXPD: Starting deletion of derivative data. [2025-10-20T18:42:09.750302] [DEBUG] [LOCAL ] EXPD: Deleted derivative data file. [2025-10-20T18:42:09.750396] [INFO ] [MAIN ] Update Progress: 799/2367 symbols completed. EXPD took 27.6s. Estimated remaining: 3h 20m 21s. Rate: 7.8 symbols/min [2025-10-20T18:42:09.920166] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T18:42:09.930290] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-10. [2025-10-20T18:42:10.798070] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T18:42:10.804893] [INFO ] [REMOTE ] EXR: Full fetch complete (66237 records). [2025-10-20T18:42:10.805009] [DEBUG] [VALIDATOR ] EXR: Validating primary data. [2025-10-20T18:42:11.189297] [DEBUG] [VALIDATOR ] EXR: All expected trading days are present. [2025-10-20T18:42:11.189404] [DEBUG] [VALIDATOR ] EXR: Primary data validation passed. [2025-10-20T18:42:11.190327] [INFO ] [MAIN ] EXR: Fetched primary data is valid. [2025-10-20T18:42:11.190414] [DEBUG] [LOCAL ] EXR: Saving primary data. [2025-10-20T18:42:11.191787] [DEBUG] [LOCAL ] EXR: Trimmed 66237 records to 45611 (keeping 200 days of data) [2025-10-20T18:42:11.471045] [DEBUG] [LOCAL ] EXR: Primary data saved (45611 records). [2025-10-20T18:42:11.471169] [INFO ] [MAIN ] EXR: Full history saved. [2025-10-20T18:42:11.471614] [INFO ] [MAIN ] Update Progress: 800/2367 symbols completed. EXR took 24.2s. Estimated remaining: 3h 20m 1s. Rate: 7.8 symbols/min [2025-10-20T18:42:12.052555] [INFO ] [MAIN ] FBK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:12.131533] [INFO ] [MAIN ] F: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:12.167480] [INFO ] [MAIN ] FBIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:12.185491] [INFO ] [MAIN ] FAF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:12.213989] [DEBUG] [LOCAL ] FBK: Loaded 27962 existing records from primary storage. [2025-10-20T18:42:12.214701] [INFO ] [MAIN ] FBK: Loaded 27962 existing records. [2025-10-20T18:42:12.459709] [WARN ] [MAIN ] FBK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:12.491313] [INFO ] [MAIN ] FBK: Wiping existing primary data before full refresh. [2025-10-20T18:42:12.491483] [INFO ] [LOCAL ] FBK: Starting deletion of primary data. [2025-10-20T18:42:12.491855] [DEBUG] [LOCAL ] FBK: Deleted primary data file. [2025-10-20T18:42:12.491980] [INFO ] [MAIN ] FBK: Starting full primary refresh. [2025-10-20T18:42:12.527323] [DEBUG] [LOCAL ] F: Loaded 54600 existing records from primary storage. [2025-10-20T18:42:12.526683] [DEBUG] [LOCAL ] FBIN: Loaded 51068 existing records from primary storage. [2025-10-20T18:42:12.549199] [DEBUG] [LOCAL ] FAF: Loaded 42095 existing records from primary storage. [2025-10-20T18:42:12.575156] [INFO ] [MAIN ] FAF: Loaded 42095 existing records. [2025-10-20T18:42:12.575030] [INFO ] [MAIN ] F: Loaded 54600 existing records. [2025-10-20T18:42:12.569720] [INFO ] [MAIN ] FBIN: Loaded 51068 existing records. [2025-10-20T18:42:13.261409] [WARN ] [MAIN ] F: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:13.299284] [WARN ] [MAIN ] FAF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:13.362213] [WARN ] [MAIN ] FBIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:13.324864] [INFO ] [MAIN ] F: Wiping existing primary data before full refresh. [2025-10-20T18:42:13.377782] [INFO ] [MAIN ] FAF: Wiping existing primary data before full refresh. [2025-10-20T18:42:13.393407] [INFO ] [MAIN ] FBIN: Wiping existing primary data before full refresh. [2025-10-20T18:42:13.393574] [INFO ] [LOCAL ] FAF: Starting deletion of primary data. [2025-10-20T18:42:13.398793] [INFO ] [LOCAL ] FBIN: Starting deletion of primary data. [2025-10-20T18:42:13.415743] [INFO ] [LOCAL ] F: Starting deletion of primary data. [2025-10-20T18:42:13.452219] [DEBUG] [LOCAL ] FAF: Deleted primary data file. [2025-10-20T18:42:13.452256] [DEBUG] [LOCAL ] FBIN: Deleted primary data file. [2025-10-20T18:42:13.455331] [INFO ] [MAIN ] FAF: Starting full primary refresh. [2025-10-20T18:42:13.458256] [INFO ] [REMOTE ] FBK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:13.469219] [INFO ] [MAIN ] FBIN: Starting full primary refresh. [2025-10-20T18:42:13.463562] [DEBUG] [LOCAL ] F: Deleted primary data file. [2025-10-20T18:42:13.469603] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-01. [2025-10-20T18:42:13.469705] [INFO ] [MAIN ] F: Starting full primary refresh. [2025-10-20T18:42:14.413074] [INFO ] [REMOTE ] FAF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:14.440322] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-01. [2025-10-20T18:42:14.548907] [INFO ] [REMOTE ] F: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:14.572408] [INFO ] [REMOTE ] F: Fetching primary data for 2025-01. [2025-10-20T18:42:14.577901] [INFO ] [REMOTE ] FBIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:14.578968] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-01. [2025-10-20T18:42:14.777230] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T18:42:14.781372] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-02. [2025-10-20T18:42:15.503982] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T18:42:15.508180] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-03. [2025-10-20T18:42:16.106882] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T18:42:16.114832] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-02. [2025-10-20T18:42:16.209083] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:42:16.244495] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T18:42:16.251326] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-04. [2025-10-20T18:42:17.051365] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-02. [2025-10-20T18:42:17.175599] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-01 (2.60 seconds). [2025-10-20T18:42:17.426456] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T18:42:17.612354] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T18:42:17.851427] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-03. [2025-10-20T18:42:18.202041] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T18:42:18.651432] [INFO ] [REMOTE ] F: Fetching primary data for 2025-02. [2025-10-20T18:42:19.230537] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:42:19.451387] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-04. [2025-10-20T18:42:20.251434] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-03. [2025-10-20T18:42:20.982003] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T18:42:21.046629] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-02 (2.40 seconds). [2025-10-20T18:42:21.056333] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-05. [2025-10-20T18:42:21.413353] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T18:42:21.721253] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T18:42:21.851380] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-05. [2025-10-20T18:42:22.651391] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-04. [2025-10-20T18:42:23.252284] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T18:42:23.451369] [INFO ] [REMOTE ] F: Fetching primary data for 2025-03. [2025-10-20T18:42:24.205739] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T18:42:24.251385] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-05. [2025-10-20T18:42:25.051360] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-06. [2025-10-20T18:42:25.644444] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-10-20T18:42:25.851387] [INFO ] [REMOTE ] F: Fetching primary data for 2025-04. [2025-10-20T18:42:26.059980] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T18:42:26.628492] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T18:42:26.651345] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-07. [2025-10-20T18:42:27.451387] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-06. [2025-10-20T18:42:28.057072] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:42:28.251379] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-08. [2025-10-20T18:42:28.348188] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-04 (2.50 seconds). [2025-10-20T18:42:28.995473] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T18:42:29.051383] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-07. [2025-10-20T18:42:29.408945] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T18:42:29.851385] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-06. [2025-10-20T18:42:30.651342] [INFO ] [REMOTE ] F: Fetching primary data for 2025-05. [2025-10-20T18:42:30.741394] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T18:42:30.913843] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:42:31.451437] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-08. [2025-10-20T18:42:32.251385] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-09. [2025-10-20T18:42:33.047826] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T18:42:33.057882] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-07. [2025-10-20T18:42:33.229688] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-05 (2.58 seconds). [2025-10-20T18:42:33.654825] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T18:42:33.851383] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-09. [2025-10-20T18:42:34.083444] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T18:42:34.651427] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-08. [2025-10-20T18:42:35.451369] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-10. [2025-10-20T18:42:35.511307] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:42:35.690384] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:42:36.251379] [INFO ] [REMOTE ] F: Fetching primary data for 2025-06. [2025-10-20T18:42:36.777079] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T18:42:36.783399] [INFO ] [REMOTE ] FAF: Full fetch complete (58180 records). [2025-10-20T18:42:36.783605] [DEBUG] [VALIDATOR ] FAF: Validating primary data. [2025-10-20T18:42:37.057399] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-09. [2025-10-20T18:42:37.213257] [DEBUG] [VALIDATOR ] FAF: All expected trading days are present. [2025-10-20T18:42:37.213376] [DEBUG] [VALIDATOR ] FAF: Primary data validation passed. [2025-10-20T18:42:37.214783] [INFO ] [MAIN ] FAF: Fetched primary data is valid. [2025-10-20T18:42:37.214851] [DEBUG] [LOCAL ] FAF: Saving primary data. [2025-10-20T18:42:37.216673] [DEBUG] [LOCAL ] FAF: Trimmed 58180 records to 41567 (keeping 200 days of data) [2025-10-20T18:42:37.489767] [DEBUG] [LOCAL ] FAF: Primary data saved (41567 records). [2025-10-20T18:42:37.489955] [INFO ] [MAIN ] FAF: Full history saved. [2025-10-20T18:42:37.490055] [INFO ] [MAIN ] Update Progress: 801/2367 symbols completed. FAF took 26.0s. Estimated remaining: 3h 20m 30s. Rate: 7.8 symbols/min [2025-10-20T18:42:37.672342] [INFO ] [MAIN ] FBP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:37.723015] [DEBUG] [LOCAL ] FBP: Loaded 46466 existing records from primary storage. [2025-10-20T18:42:37.723458] [INFO ] [MAIN ] FBP: Loaded 46466 existing records. [2025-10-20T18:42:37.856461] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-10. [2025-10-20T18:42:37.920069] [WARN ] [MAIN ] FBP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:37.920381] [INFO ] [MAIN ] FBP: Wiping existing primary data before full refresh. [2025-10-20T18:42:37.920436] [INFO ] [LOCAL ] FBP: Starting deletion of primary data. [2025-10-20T18:42:37.920706] [DEBUG] [LOCAL ] FBP: Deleted primary data file. [2025-10-20T18:42:37.920765] [INFO ] [MAIN ] FBP: Starting full primary refresh. [2025-10-20T18:42:38.310282] [INFO ] [REMOTE ] FBP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:38.326797] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T18:42:38.418323] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T18:42:38.651392] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-10. [2025-10-20T18:42:39.451402] [INFO ] [REMOTE ] F: Fetching primary data for 2025-07. [2025-10-20T18:42:39.727319] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:42:39.735303] [INFO ] [REMOTE ] FBIN: Full fetch complete (73015 records). [2025-10-20T18:42:39.735444] [DEBUG] [VALIDATOR ] FBIN: Validating primary data. [2025-10-20T18:42:40.146110] [DEBUG] [VALIDATOR ] FBIN: All expected trading days are present. [2025-10-20T18:42:40.153034] [DEBUG] [VALIDATOR ] FBIN: Primary data validation passed. [2025-10-20T18:42:40.166374] [INFO ] [MAIN ] FBIN: Fetched primary data is valid. [2025-10-20T18:42:40.166532] [DEBUG] [LOCAL ] FBIN: Saving primary data. [2025-10-20T18:42:40.168528] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T18:42:40.175728] [INFO ] [REMOTE ] FBK: Full fetch complete (35423 records). [2025-10-20T18:42:40.175928] [DEBUG] [LOCAL ] FBIN: Trimmed 73015 records to 50414 (keeping 200 days of data) [2025-10-20T18:42:40.176001] [DEBUG] [VALIDATOR ] FBK: Validating primary data. [2025-10-20T18:42:40.266674] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-01. [2025-10-20T18:42:40.680937] [DEBUG] [VALIDATOR ] FBK: All expected trading days are present. [2025-10-20T18:42:40.681026] [DEBUG] [VALIDATOR ] FBK: Primary data validation passed. [2025-10-20T18:42:40.682917] [INFO ] [MAIN ] FBK: Fetched primary data is valid. [2025-10-20T18:42:40.683235] [DEBUG] [LOCAL ] FBK: Saving primary data. [2025-10-20T18:42:40.688875] [DEBUG] [LOCAL ] FBK: Trimmed 35423 records to 27519 (keeping 200 days of data) [2025-10-20T18:42:41.031885] [DEBUG] [LOCAL ] FBK: Primary data saved (27519 records). [2025-10-20T18:42:41.035971] [INFO ] [MAIN ] FBK: Full history saved. [2025-10-20T18:42:41.036334] [INFO ] [MAIN ] Update Progress: 802/2367 symbols completed. FBK took 29.6s. Estimated remaining: 3h 20m 14s. Rate: 7.8 symbols/min [2025-10-20T18:42:41.037254] [DEBUG] [LOCAL ] FBIN: Primary data saved (50414 records). [2025-10-20T18:42:41.037365] [INFO ] [MAIN ] FBIN: Full history saved. [2025-10-20T18:42:41.070061] [INFO ] [MAIN ] Update Progress: 803/2367 symbols completed. FBIN took 29.5s. Estimated remaining: 3h 19m 51s. Rate: 7.8 symbols/min [2025-10-20T18:42:41.369313] [INFO ] [MAIN ] FBRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:41.386073] [INFO ] [MAIN ] FC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:41.402825] [DEBUG] [LOCAL ] FC: Loaded 15052 existing records from primary storage. [2025-10-20T18:42:41.403091] [INFO ] [MAIN ] FC: Loaded 15052 existing records. [2025-10-20T18:42:41.542603] [DEBUG] [LOCAL ] FBRT: Loaded 35936 existing records from primary storage. [2025-10-20T18:42:41.566454] [INFO ] [MAIN ] FBRT: Loaded 35936 existing records. [2025-10-20T18:42:41.660910] [WARN ] [MAIN ] FC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:41.694828] [INFO ] [MAIN ] FC: Wiping existing primary data before full refresh. [2025-10-20T18:42:41.700114] [INFO ] [LOCAL ] FC: Starting deletion of primary data. [2025-10-20T18:42:41.708443] [DEBUG] [LOCAL ] FC: Deleted primary data file. [2025-10-20T18:42:41.708501] [INFO ] [MAIN ] FC: Starting full primary refresh. [2025-10-20T18:42:41.912317] [WARN ] [MAIN ] FBRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:41.912838] [INFO ] [MAIN ] FBRT: Wiping existing primary data before full refresh. [2025-10-20T18:42:41.918819] [INFO ] [LOCAL ] FBRT: Starting deletion of primary data. [2025-10-20T18:42:41.939786] [DEBUG] [LOCAL ] FBRT: Deleted primary data file. [2025-10-20T18:42:41.952814] [INFO ] [MAIN ] FBRT: Starting full primary refresh. [2025-10-20T18:42:42.351915] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-01 (2.09 seconds). [2025-10-20T18:42:42.361595] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-02. [2025-10-20T18:42:42.340217] [INFO ] [REMOTE ] FC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:42.366457] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-01. [2025-10-20T18:42:42.587280] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-07 (3.14 seconds). [2025-10-20T18:42:42.645075] [INFO ] [REMOTE ] FBRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:42.651326] [INFO ] [REMOTE ] F: Fetching primary data for 2025-08. [2025-10-20T18:42:43.316244] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T18:42:43.451345] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-02. [2025-10-20T18:42:43.862687] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T18:42:44.013540] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T18:42:44.251386] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-03. [2025-10-20T18:42:44.799639] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-08 (2.15 seconds). [2025-10-20T18:42:45.051389] [INFO ] [REMOTE ] F: Fetching primary data for 2025-09. [2025-10-20T18:42:45.825394] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T18:42:45.851339] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-01. [2025-10-20T18:42:46.651416] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-03. [2025-10-20T18:42:47.092445] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:42:47.458966] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-04. [2025-10-20T18:42:47.464850] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-09 (2.41 seconds). [2025-10-20T18:42:47.870486] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T18:42:48.251385] [INFO ] [REMOTE ] F: Fetching primary data for 2025-10. [2025-10-20T18:42:49.051408] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-02. [2025-10-20T18:42:49.089038] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:42:49.119860] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T18:42:49.127564] [INFO ] [REMOTE ] F: Full fetch complete (78000 records). [2025-10-20T18:42:49.127714] [DEBUG] [VALIDATOR ] F: Validating primary data. [2025-10-20T18:42:49.519635] [DEBUG] [VALIDATOR ] F: All expected trading days are present. [2025-10-20T18:42:49.519750] [DEBUG] [VALIDATOR ] F: Primary data validation passed. [2025-10-20T18:42:49.520692] [INFO ] [MAIN ] F: Fetched primary data is valid. [2025-10-20T18:42:49.520768] [DEBUG] [LOCAL ] F: Saving primary data. [2025-10-20T18:42:49.522232] [DEBUG] [LOCAL ] F: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-10-20T18:42:49.848499] [DEBUG] [LOCAL ] F: Primary data saved (53820 records). [2025-10-20T18:42:49.848668] [INFO ] [MAIN ] F: Full history saved. [2025-10-20T18:42:49.848764] [INFO ] [MAIN ] Update Progress: 804/2367 symbols completed. F took 38.4s. Estimated remaining: 3h 19m 46s. Rate: 7.8 symbols/min [2025-10-20T18:42:49.860011] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-04. [2025-10-20T18:42:50.030317] [INFO ] [MAIN ] FCF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:50.070553] [DEBUG] [LOCAL ] FCF: Loaded 39969 existing records from primary storage. [2025-10-20T18:42:50.070883] [INFO ] [MAIN ] FCF: Loaded 39969 existing records. [2025-10-20T18:42:50.250093] [WARN ] [MAIN ] FCF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:50.250394] [INFO ] [MAIN ] FCF: Wiping existing primary data before full refresh. [2025-10-20T18:42:50.250463] [INFO ] [LOCAL ] FCF: Starting deletion of primary data. [2025-10-20T18:42:50.250853] [DEBUG] [LOCAL ] FCF: Deleted primary data file. [2025-10-20T18:42:50.250954] [INFO ] [MAIN ] FCF: Starting full primary refresh. [2025-10-20T18:42:50.413478] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T18:42:50.634405] [INFO ] [REMOTE ] FCF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:50.651378] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-03. [2025-10-20T18:42:50.985800] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T18:42:51.451392] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-01. [2025-10-20T18:42:51.864874] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T18:42:52.251357] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-05. [2025-10-20T18:42:52.813141] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T18:42:52.892487] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-10-20T18:42:53.051382] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-06. [2025-10-20T18:42:53.851349] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-05. [2025-10-20T18:42:53.993703] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:42:54.651373] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-07. [2025-10-20T18:42:55.452810] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T18:42:55.451452] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-02. [2025-10-20T18:42:55.724397] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:42:56.251404] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-08. [2025-10-20T18:42:56.741680] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T18:42:57.051417] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-06. [2025-10-20T18:42:57.302636] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T18:42:57.851378] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-04. [2025-10-20T18:42:58.521356] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T18:42:58.651365] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-03. [2025-10-20T18:42:59.375284] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T18:42:59.451370] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-05. [2025-10-20T18:43:00.077295] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T18:43:00.251371] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-04. [2025-10-20T18:43:00.800501] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T18:43:01.051422] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-06. [2025-10-20T18:43:01.680752] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:43:01.851401] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-09. [2025-10-20T18:43:02.344846] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:43:02.401409] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-10-20T18:43:02.651400] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-10. [2025-10-20T18:43:03.451364] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-05. [2025-10-20T18:43:03.546331] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T18:43:03.548637] [INFO ] [REMOTE ] FC: Full fetch complete (21857 records). [2025-10-20T18:43:03.548731] [DEBUG] [VALIDATOR ] FC: Validating primary data. [2025-10-20T18:43:03.925771] [DEBUG] [VALIDATOR ] FC: All expected trading days are present. [2025-10-20T18:43:03.925847] [DEBUG] [VALIDATOR ] FC: Primary data validation passed. [2025-10-20T18:43:03.926277] [INFO ] [MAIN ] FC: Fetched primary data is valid. [2025-10-20T18:43:03.926315] [DEBUG] [LOCAL ] FC: Saving primary data. [2025-10-20T18:43:03.926897] [DEBUG] [LOCAL ] FC: Trimmed 21857 records to 14647 (keeping 200 days of data) [2025-10-20T18:43:04.011396] [DEBUG] [LOCAL ] FC: Primary data saved (14647 records). [2025-10-20T18:43:04.011535] [INFO ] [MAIN ] FC: Full history saved. [2025-10-20T18:43:04.011614] [INFO ] [MAIN ] Update Progress: 805/2367 symbols completed. FC took 22.9s. Estimated remaining: 3h 19m 51s. Rate: 7.8 symbols/min [2025-10-20T18:43:04.193313] [INFO ] [MAIN ] FCN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:04.221882] [DEBUG] [LOCAL ] FCN: Loaded 26402 existing records from primary storage. [2025-10-20T18:43:04.222156] [INFO ] [MAIN ] FCN: Loaded 26402 existing records. [2025-10-20T18:43:04.256479] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-07. [2025-10-20T18:43:04.408997] [WARN ] [MAIN ] FCN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:04.409253] [INFO ] [MAIN ] FCN: Wiping existing primary data before full refresh. [2025-10-20T18:43:04.409311] [INFO ] [LOCAL ] FCN: Starting deletion of primary data. [2025-10-20T18:43:04.409790] [DEBUG] [LOCAL ] FCN: Deleted primary data file. [2025-10-20T18:43:04.409855] [INFO ] [MAIN ] FCN: Starting full primary refresh. [2025-10-20T18:43:04.776681] [INFO ] [REMOTE ] FCN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:05.047134] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T18:43:05.054675] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-01. [2025-10-20T18:43:05.760249] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T18:43:05.778791] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T18:43:05.851366] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-02. [2025-10-20T18:43:06.651354] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-07. [2025-10-20T18:43:06.868720] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:43:07.451364] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-06. [2025-10-20T18:43:08.251346] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-08. [2025-10-20T18:43:08.448290] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:43:08.879190] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T18:43:09.051396] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-07. [2025-10-20T18:43:09.613184] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T18:43:09.851370] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-09. [2025-10-20T18:43:10.487197] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:43:10.651361] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-03. [2025-10-20T18:43:11.056566] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T18:43:11.451397] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-10. [2025-10-20T18:43:11.961746] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T18:43:12.251392] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-04. [2025-10-20T18:43:13.012662] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T18:43:13.019467] [INFO ] [REMOTE ] FBRT: Full fetch complete (49226 records). [2025-10-20T18:43:13.019705] [DEBUG] [VALIDATOR ] FBRT: Validating primary data. [2025-10-20T18:43:13.020896] [FATAL] [VALIDATOR ] FBRT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:43:13.020992] [FATAL] [MAIN ] FBRT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:43:13.021036] [WARN ] [MAIN ] FBRT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:43:13.021078] [INFO ] [LOCAL ] FBRT: Starting deletion of primary data. [2025-10-20T18:43:13.021164] [DEBUG] [LOCAL ] FBRT: No primary data file found to delete. [2025-10-20T18:43:13.021220] [INFO ] [LOCAL ] FBRT: Starting deletion of derivative data. [2025-10-20T18:43:13.021438] [DEBUG] [LOCAL ] FBRT: Deleted derivative data file. [2025-10-20T18:43:13.021582] [INFO ] [MAIN ] Update Progress: 806/2367 symbols completed. FBRT took 32.0s. Estimated remaining: 3h 19m 46s. Rate: 7.8 symbols/min [2025-10-20T18:43:13.056696] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-08. [2025-10-20T18:43:13.217242] [INFO ] [MAIN ] FCPT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:13.263174] [DEBUG] [LOCAL ] FCPT: Loaded 37885 existing records from primary storage. [2025-10-20T18:43:13.263617] [INFO ] [MAIN ] FCPT: Loaded 37885 existing records. [2025-10-20T18:43:13.462034] [WARN ] [MAIN ] FCPT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:13.462359] [INFO ] [MAIN ] FCPT: Wiping existing primary data before full refresh. [2025-10-20T18:43:13.462413] [INFO ] [LOCAL ] FCPT: Starting deletion of primary data. [2025-10-20T18:43:13.462709] [DEBUG] [LOCAL ] FCPT: Deleted primary data file. [2025-10-20T18:43:13.462771] [INFO ] [MAIN ] FCPT: Starting full primary refresh. [2025-10-20T18:43:13.859733] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-08. [2025-10-20T18:43:13.864311] [INFO ] [REMOTE ] FCPT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:13.868864] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:43:14.633199] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T18:43:14.651367] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-01. [2025-10-20T18:43:15.451394] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-05. [2025-10-20T18:43:15.506221] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T18:43:16.102815] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:43:16.251420] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-02. [2025-10-20T18:43:16.450169] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T18:43:17.051426] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-09. [2025-10-20T18:43:17.768388] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:43:17.851382] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-03. [2025-10-20T18:43:18.296928] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:43:18.651376] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-09. [2025-10-20T18:43:19.401472] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T18:43:19.451383] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-04. [2025-10-20T18:43:20.063918] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T18:43:20.251470] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-10. [2025-10-20T18:43:20.679291] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T18:43:21.051406] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-10. [2025-10-20T18:43:21.714975] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:43:21.720588] [INFO ] [REMOTE ] FCF: Full fetch complete (57955 records). [2025-10-20T18:43:21.720713] [DEBUG] [VALIDATOR ] FCF: Validating primary data. [2025-10-20T18:43:21.721481] [FATAL] [VALIDATOR ] FCF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:43:21.721567] [FATAL] [MAIN ] FCF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:43:21.721610] [WARN ] [MAIN ] FCF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:43:21.721674] [INFO ] [LOCAL ] FCF: Starting deletion of primary data. [2025-10-20T18:43:21.721752] [DEBUG] [LOCAL ] FCF: No primary data file found to delete. [2025-10-20T18:43:21.721789] [INFO ] [LOCAL ] FCF: Starting deletion of derivative data. [2025-10-20T18:43:21.721961] [DEBUG] [LOCAL ] FCF: Deleted derivative data file. [2025-10-20T18:43:21.722047] [INFO ] [MAIN ] Update Progress: 807/2367 symbols completed. FCF took 31.9s. Estimated remaining: 3h 19m 40s. Rate: 7.8 symbols/min [2025-10-20T18:43:21.858424] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-05. [2025-10-20T18:43:21.890238] [INFO ] [MAIN ] FCRX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:21.890425] [DEBUG] [LOCAL ] FCRX: No existing data file found in primary storage. [2025-10-20T18:43:21.890472] [WARN ] [MAIN ] FCRX: No existing data found, performing full refresh instead. [2025-10-20T18:43:21.890574] [INFO ] [MAIN ] FCRX: Wiping existing primary data before full refresh. [2025-10-20T18:43:21.890614] [INFO ] [LOCAL ] FCRX: Starting deletion of primary data. [2025-10-20T18:43:21.890690] [DEBUG] [LOCAL ] FCRX: No primary data file found to delete. [2025-10-20T18:43:21.890741] [INFO ] [MAIN ] FCRX: Starting full primary refresh. [2025-10-20T18:43:22.245513] [INFO ] [REMOTE ] FCRX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:22.651342] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-06. [2025-10-20T18:43:22.797564] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T18:43:22.804611] [INFO ] [REMOTE ] FBP: Full fetch complete (66777 records). [2025-10-20T18:43:22.804754] [DEBUG] [VALIDATOR ] FBP: Validating primary data. [2025-10-20T18:43:23.187372] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:43:23.231187] [DEBUG] [VALIDATOR ] FBP: All expected trading days are present. [2025-10-20T18:43:23.231590] [DEBUG] [VALIDATOR ] FBP: Primary data validation passed. [2025-10-20T18:43:23.232432] [INFO ] [MAIN ] FBP: Fetched primary data is valid. [2025-10-20T18:43:23.232491] [DEBUG] [LOCAL ] FBP: Saving primary data. [2025-10-20T18:43:23.233626] [DEBUG] [LOCAL ] FBP: Trimmed 66777 records to 45752 (keeping 200 days of data) [2025-10-20T18:43:23.454545] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-06. [2025-10-20T18:43:23.505457] [DEBUG] [LOCAL ] FBP: Primary data saved (45752 records). [2025-10-20T18:43:23.505571] [INFO ] [MAIN ] FBP: Full history saved. [2025-10-20T18:43:23.505664] [INFO ] [MAIN ] Update Progress: 808/2367 symbols completed. FBP took 46.0s. Estimated remaining: 3h 19m 21s. Rate: 7.8 symbols/min [2025-10-20T18:43:23.677821] [INFO ] [MAIN ] FCT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:23.692476] [DEBUG] [LOCAL ] FCT: Loaded 14906 existing records from primary storage. [2025-10-20T18:43:23.692696] [INFO ] [MAIN ] FCT: Loaded 14906 existing records. [2025-10-20T18:43:23.873665] [WARN ] [MAIN ] FCT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:23.887805] [INFO ] [MAIN ] FCT: Wiping existing primary data before full refresh. [2025-10-20T18:43:23.888810] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T18:43:23.888338] [INFO ] [LOCAL ] FCT: Starting deletion of primary data. [2025-10-20T18:43:23.894847] [DEBUG] [LOCAL ] FCT: Deleted primary data file. [2025-10-20T18:43:23.894905] [INFO ] [MAIN ] FCT: Starting full primary refresh. [2025-10-20T18:43:24.239926] [INFO ] [REMOTE ] FCT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:24.251335] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-01. [2025-10-20T18:43:24.878784] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:43:25.051343] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-07. [2025-10-20T18:43:25.361088] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:43:25.851364] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-02. [2025-10-20T18:43:26.308389] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T18:43:26.630321] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T18:43:26.651332] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-03. [2025-10-20T18:43:27.451456] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-07. [2025-10-20T18:43:27.561194] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T18:43:28.251358] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-04. [2025-10-20T18:43:28.731784] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:43:29.051388] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-08. [2025-10-20T18:43:29.162429] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T18:43:29.851429] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-08. [2025-10-20T18:43:30.188338] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T18:43:30.651360] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-01. [2025-10-20T18:43:31.225355] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T18:43:31.246462] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T18:43:31.451373] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-05. [2025-10-20T18:43:32.174468] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T18:43:32.251392] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-09. [2025-10-20T18:43:33.051373] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-02. [2025-10-20T18:43:33.301778] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T18:43:33.464678] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T18:43:33.851394] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-06. [2025-10-20T18:43:34.651383] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-03. [2025-10-20T18:43:34.817247] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T18:43:34.978158] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-10-20T18:43:35.451377] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-07. [2025-10-20T18:43:36.251344] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-10. [2025-10-20T18:43:36.615724] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T18:43:37.051364] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-04. [2025-10-20T18:43:37.397346] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T18:43:37.597460] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T18:43:37.603739] [INFO ] [REMOTE ] FCN: Full fetch complete (36076 records). [2025-10-20T18:43:37.603927] [DEBUG] [VALIDATOR ] FCN: Validating primary data. [2025-10-20T18:43:37.857352] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-08. [2025-10-20T18:43:37.988946] [DEBUG] [VALIDATOR ] FCN: All expected trading days are present. [2025-10-20T18:43:37.989027] [DEBUG] [VALIDATOR ] FCN: Primary data validation passed. [2025-10-20T18:43:37.989712] [INFO ] [MAIN ] FCN: Fetched primary data is valid. [2025-10-20T18:43:37.989774] [DEBUG] [LOCAL ] FCN: Saving primary data. [2025-10-20T18:43:37.990675] [DEBUG] [LOCAL ] FCN: Trimmed 36076 records to 25974 (keeping 200 days of data) [2025-10-20T18:43:38.154630] [DEBUG] [LOCAL ] FCN: Primary data saved (25974 records). [2025-10-20T18:43:38.154809] [INFO ] [MAIN ] FCN: Full history saved. [2025-10-20T18:43:38.154909] [INFO ] [MAIN ] Update Progress: 809/2367 symbols completed. FCN took 34.1s. Estimated remaining: 3h 19m 27s. Rate: 7.8 symbols/min [2025-10-20T18:43:38.330737] [INFO ] [MAIN ] FCX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:38.378965] [DEBUG] [LOCAL ] FCX: Loaded 54481 existing records from primary storage. [2025-10-20T18:43:38.379297] [INFO ] [MAIN ] FCX: Loaded 54481 existing records. [2025-10-20T18:43:38.558396] [WARN ] [MAIN ] FCX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:38.558829] [INFO ] [MAIN ] FCX: Wiping existing primary data before full refresh. [2025-10-20T18:43:38.558939] [INFO ] [LOCAL ] FCX: Starting deletion of primary data. [2025-10-20T18:43:38.559264] [DEBUG] [LOCAL ] FCX: Deleted primary data file. [2025-10-20T18:43:38.559319] [INFO ] [MAIN ] FCX: Starting full primary refresh. [2025-10-20T18:43:38.656035] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-05. [2025-10-20T18:43:38.862171] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T18:43:38.932976] [INFO ] [REMOTE ] FCX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:38.935741] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-10-20T18:43:39.451368] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-09. [2025-10-20T18:43:40.251353] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-06. [2025-10-20T18:43:40.492978] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T18:43:40.769418] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-10-20T18:43:41.051366] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-07. [2025-10-20T18:43:41.509190] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T18:43:41.851352] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-08. [2025-10-20T18:43:42.189559] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:43:42.651387] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-09. [2025-10-20T18:43:43.451373] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-09. [2025-10-20T18:43:43.875710] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-10-20T18:43:44.039573] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T18:43:44.251375] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-10. [2025-10-20T18:43:44.621279] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-10-20T18:43:44.622103] [INFO ] [REMOTE ] FCRX: Full fetch complete (1642 records). [2025-10-20T18:43:44.622189] [DEBUG] [VALIDATOR ] FCRX: Validating primary data. [2025-10-20T18:43:44.799629] [WARN ] [VALIDATOR ] FCRX: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-10-20T18:43:44.799860] [FATAL] [MAIN ] FCRX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:43:44.799929] [WARN ] [MAIN ] FCRX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:43:44.799999] [INFO ] [LOCAL ] FCRX: Starting deletion of primary data. [2025-10-20T18:43:44.800060] [DEBUG] [LOCAL ] FCRX: No primary data file found to delete. [2025-10-20T18:43:44.800096] [INFO ] [LOCAL ] FCRX: Starting deletion of derivative data. [2025-10-20T18:43:44.800133] [DEBUG] [LOCAL ] FCRX: No derivative data file found to delete. [2025-10-20T18:43:44.800185] [INFO ] [MAIN ] Update Progress: 810/2367 symbols completed. FCRX took 23.1s. Estimated remaining: 3h 19m 17s. Rate: 7.8 symbols/min [2025-10-20T18:43:44.965010] [INFO ] [MAIN ] FDP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:44.995749] [DEBUG] [LOCAL ] FDP: Loaded 30199 existing records from primary storage. [2025-10-20T18:43:44.996007] [INFO ] [MAIN ] FDP: Loaded 30199 existing records. [2025-10-20T18:43:45.056502] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-01. [2025-10-20T18:43:45.173542] [WARN ] [MAIN ] FDP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:45.173783] [INFO ] [MAIN ] FDP: Wiping existing primary data before full refresh. [2025-10-20T18:43:45.173831] [INFO ] [LOCAL ] FDP: Starting deletion of primary data. [2025-10-20T18:43:45.174130] [DEBUG] [LOCAL ] FDP: Deleted primary data file. [2025-10-20T18:43:45.174193] [INFO ] [MAIN ] FDP: Starting full primary refresh. [2025-10-20T18:43:45.545141] [INFO ] [REMOTE ] FDP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:45.851377] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-10. [2025-10-20T18:43:46.651356] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-01. [2025-10-20T18:43:47.271327] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-01 (2.21 seconds). [2025-10-20T18:43:47.312287] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:43:47.318232] [INFO ] [REMOTE ] FCPT: Full fetch complete (53882 records). [2025-10-20T18:43:47.318367] [DEBUG] [VALIDATOR ] FCPT: Validating primary data. [2025-10-20T18:43:47.457415] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-10. [2025-10-20T18:43:47.703378] [DEBUG] [VALIDATOR ] FCPT: All expected trading days are present. [2025-10-20T18:43:47.703473] [DEBUG] [VALIDATOR ] FCPT: Primary data validation passed. [2025-10-20T18:43:47.704219] [INFO ] [MAIN ] FCPT: Fetched primary data is valid. [2025-10-20T18:43:47.704283] [DEBUG] [LOCAL ] FCPT: Saving primary data. [2025-10-20T18:43:47.705483] [DEBUG] [LOCAL ] FCPT: Trimmed 53882 records to 37210 (keeping 200 days of data) [2025-10-20T18:43:47.890964] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:43:47.948300] [DEBUG] [LOCAL ] FCPT: Primary data saved (37210 records). [2025-10-20T18:43:47.948440] [INFO ] [MAIN ] FCPT: Full history saved. [2025-10-20T18:43:47.948925] [INFO ] [MAIN ] Update Progress: 811/2367 symbols completed. FCPT took 34.9s. Estimated remaining: 3h 19m 1s. Rate: 7.8 symbols/min [2025-10-20T18:43:48.115560] [INFO ] [MAIN ] FDS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:48.143006] [DEBUG] [LOCAL ] FDS: Loaded 28183 existing records from primary storage. [2025-10-20T18:43:48.143256] [INFO ] [MAIN ] FDS: Loaded 28183 existing records. [2025-10-20T18:43:48.256538] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-02. [2025-10-20T18:43:48.335419] [WARN ] [MAIN ] FDS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:48.335668] [INFO ] [MAIN ] FDS: Wiping existing primary data before full refresh. [2025-10-20T18:43:48.335729] [INFO ] [LOCAL ] FDS: Starting deletion of primary data. [2025-10-20T18:43:48.335953] [DEBUG] [LOCAL ] FDS: Deleted primary data file. [2025-10-20T18:43:48.336009] [INFO ] [MAIN ] FDS: Starting full primary refresh. [2025-10-20T18:43:48.713239] [INFO ] [REMOTE ] FDS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:48.785548] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T18:43:48.788425] [INFO ] [REMOTE ] FCT: Full fetch complete (21124 records). [2025-10-20T18:43:48.788503] [DEBUG] [VALIDATOR ] FCT: Validating primary data. [2025-10-20T18:43:49.056595] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-01. [2025-10-20T18:43:49.164210] [DEBUG] [VALIDATOR ] FCT: All expected trading days are present. [2025-10-20T18:43:49.164287] [DEBUG] [VALIDATOR ] FCT: Primary data validation passed. [2025-10-20T18:43:49.164774] [INFO ] [MAIN ] FCT: Fetched primary data is valid. [2025-10-20T18:43:49.164835] [DEBUG] [LOCAL ] FCT: Saving primary data. [2025-10-20T18:43:49.165499] [DEBUG] [LOCAL ] FCT: Trimmed 21124 records to 14667 (keeping 200 days of data) [2025-10-20T18:43:49.248795] [DEBUG] [LOCAL ] FCT: Primary data saved (14667 records). [2025-10-20T18:43:49.248933] [INFO ] [MAIN ] FCT: Full history saved. [2025-10-20T18:43:49.249009] [INFO ] [MAIN ] Update Progress: 812/2367 symbols completed. FCT took 25.7s. Estimated remaining: 3h 18m 41s. Rate: 7.8 symbols/min [2025-10-20T18:43:49.432758] [INFO ] [MAIN ] FDX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:49.481061] [DEBUG] [LOCAL ] FDX: Loaded 52099 existing records from primary storage. [2025-10-20T18:43:49.481379] [INFO ] [MAIN ] FDX: Loaded 52099 existing records. [2025-10-20T18:43:49.657552] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T18:43:49.685048] [WARN ] [MAIN ] FDX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:49.685241] [INFO ] [MAIN ] FDX: Wiping existing primary data before full refresh. [2025-10-20T18:43:49.685289] [INFO ] [LOCAL ] FDX: Starting deletion of primary data. [2025-10-20T18:43:49.685484] [DEBUG] [LOCAL ] FDX: Deleted primary data file. [2025-10-20T18:43:49.685528] [INFO ] [MAIN ] FDX: Starting full primary refresh. [2025-10-20T18:43:49.855923] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-03. [2025-10-20T18:43:50.040272] [INFO ] [REMOTE ] FDX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:50.363554] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T18:43:50.651375] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-02. [2025-10-20T18:43:51.451434] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-02. [2025-10-20T18:43:51.481927] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T18:43:52.251427] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-04. [2025-10-20T18:43:52.553126] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T18:43:52.711685] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T18:43:53.051362] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-03. [2025-10-20T18:43:53.160291] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T18:43:53.851426] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-01. [2025-10-20T18:43:54.651496] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-05. [2025-10-20T18:43:55.281173] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-03 (2.23 seconds). [2025-10-20T18:43:55.451370] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-04. [2025-10-20T18:43:55.567808] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T18:43:55.964397] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T18:43:56.251433] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-06. [2025-10-20T18:43:57.051427] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-02. [2025-10-20T18:43:57.573879] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T18:43:57.730152] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-04 (2.28 seconds). [2025-10-20T18:43:57.851375] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-05. [2025-10-20T18:43:58.651463] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-03. [2025-10-20T18:43:58.804536] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:43:59.451399] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-07. [2025-10-20T18:43:59.878731] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T18:43:59.972912] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:44:00.251416] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-06. [2025-10-20T18:44:00.514156] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T18:44:01.051389] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-03. [2025-10-20T18:44:01.851364] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-08. [2025-10-20T18:44:02.082531] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T18:44:02.651375] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-07. [2025-10-20T18:44:03.317168] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-03 (2.27 seconds). [2025-10-20T18:44:03.441457] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:44:03.451330] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-04. [2025-10-20T18:44:04.251370] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-09. [2025-10-20T18:44:05.051375] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-04. [2025-10-20T18:44:05.166034] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-07 (2.51 seconds). [2025-10-20T18:44:05.240843] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T18:44:05.503607] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:44:05.851362] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-10. [2025-10-20T18:44:06.171172] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T18:44:06.651363] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-05. [2025-10-20T18:44:07.121451] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T18:44:07.125854] [INFO ] [REMOTE ] FDP: Full fetch complete (42759 records). [2025-10-20T18:44:07.125999] [DEBUG] [VALIDATOR ] FDP: Validating primary data. [2025-10-20T18:44:07.456550] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-05. [2025-10-20T18:44:07.521217] [DEBUG] [VALIDATOR ] FDP: All expected trading days are present. [2025-10-20T18:44:07.521423] [DEBUG] [VALIDATOR ] FDP: Primary data validation passed. [2025-10-20T18:44:07.522168] [INFO ] [MAIN ] FDP: Fetched primary data is valid. [2025-10-20T18:44:07.522232] [DEBUG] [LOCAL ] FDP: Saving primary data. [2025-10-20T18:44:07.523451] [DEBUG] [LOCAL ] FDP: Trimmed 42759 records to 29728 (keeping 200 days of data) [2025-10-20T18:44:07.707429] [DEBUG] [LOCAL ] FDP: Primary data saved (29728 records). [2025-10-20T18:44:07.707707] [INFO ] [MAIN ] FDP: Full history saved. [2025-10-20T18:44:07.707851] [INFO ] [MAIN ] Update Progress: 813/2367 symbols completed. FDP took 22.9s. Estimated remaining: 3h 18m 54s. Rate: 7.8 symbols/min [2025-10-20T18:44:07.836085] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:44:07.912999] [INFO ] [MAIN ] FE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:07.963409] [DEBUG] [LOCAL ] FE: Loaded 54212 existing records from primary storage. [2025-10-20T18:44:07.963772] [INFO ] [MAIN ] FE: Loaded 54212 existing records. [2025-10-20T18:44:08.146660] [WARN ] [MAIN ] FE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:08.146898] [INFO ] [MAIN ] FE: Wiping existing primary data before full refresh. [2025-10-20T18:44:08.146946] [INFO ] [LOCAL ] FE: Starting deletion of primary data. [2025-10-20T18:44:08.147160] [DEBUG] [LOCAL ] FE: Deleted primary data file. [2025-10-20T18:44:08.147218] [INFO ] [MAIN ] FE: Starting full primary refresh. [2025-10-20T18:44:08.256733] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-06. [2025-10-20T18:44:08.513759] [INFO ] [REMOTE ] FE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:09.051438] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-01. [2025-10-20T18:44:09.569333] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:44:09.606065] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-10-20T18:44:09.851457] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-07. [2025-10-20T18:44:10.651369] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-06. [2025-10-20T18:44:10.752690] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:44:10.781189] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T18:44:11.451384] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-02. [2025-10-20T18:44:12.251412] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-08. [2025-10-20T18:44:12.588126] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-10-20T18:44:12.674557] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T18:44:13.051450] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-03. [2025-10-20T18:44:13.449266] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T18:44:13.851375] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-09. [2025-10-20T18:44:14.651452] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-08. [2025-10-20T18:44:14.777050] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T18:44:15.336757] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T18:44:15.451455] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-04. [2025-10-20T18:44:16.251428] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-10. [2025-10-20T18:44:16.795054] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T18:44:17.051421] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-07. [2025-10-20T18:44:17.254793] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:44:17.851494] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-05. [2025-10-20T18:44:18.075120] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T18:44:18.081221] [INFO ] [REMOTE ] FDS: Full fetch complete (39049 records). [2025-10-20T18:44:18.081344] [DEBUG] [VALIDATOR ] FDS: Validating primary data. [2025-10-20T18:44:18.459999] [DEBUG] [VALIDATOR ] FDS: All expected trading days are present. [2025-10-20T18:44:18.460089] [DEBUG] [VALIDATOR ] FDS: Primary data validation passed. [2025-10-20T18:44:18.460796] [INFO ] [MAIN ] FDS: Fetched primary data is valid. [2025-10-20T18:44:18.460857] [DEBUG] [LOCAL ] FDS: Saving primary data. [2025-10-20T18:44:18.461711] [DEBUG] [LOCAL ] FDS: Trimmed 39049 records to 27869 (keeping 200 days of data) [2025-10-20T18:44:18.632215] [DEBUG] [LOCAL ] FDS: Primary data saved (27869 records). [2025-10-20T18:44:18.632353] [INFO ] [MAIN ] FDS: Full history saved. [2025-10-20T18:44:18.632433] [INFO ] [MAIN ] Update Progress: 814/2367 symbols completed. FDS took 30.7s. Estimated remaining: 3h 18m 52s. Rate: 7.8 symbols/min [2025-10-20T18:44:18.662680] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-09. [2025-10-20T18:44:18.805273] [INFO ] [MAIN ] FENG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:18.805435] [DEBUG] [LOCAL ] FENG: No existing data file found in primary storage. [2025-10-20T18:44:18.805472] [WARN ] [MAIN ] FENG: No existing data found, performing full refresh instead. [2025-10-20T18:44:18.805580] [INFO ] [MAIN ] FENG: Wiping existing primary data before full refresh. [2025-10-20T18:44:18.805621] [INFO ] [LOCAL ] FENG: Starting deletion of primary data. [2025-10-20T18:44:18.805698] [DEBUG] [LOCAL ] FENG: No primary data file found to delete. [2025-10-20T18:44:18.805744] [INFO ] [MAIN ] FENG: Starting full primary refresh. [2025-10-20T18:44:19.171290] [INFO ] [REMOTE ] FENG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:19.202115] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-07 (2.15 seconds). [2025-10-20T18:44:19.451381] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-01. [2025-10-20T18:44:19.528712] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:44:20.063357] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T18:44:20.251404] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-06. [2025-10-20T18:44:20.866078] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-10-20T18:44:21.051380] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-10. [2025-10-20T18:44:21.851418] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-08. [2025-10-20T18:44:21.934909] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T18:44:21.953365] [INFO ] [REMOTE ] FCX: Full fetch complete (77878 records). [2025-10-20T18:44:21.964535] [DEBUG] [VALIDATOR ] FCX: Validating primary data. [2025-10-20T18:44:22.014545] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T18:44:22.380089] [DEBUG] [VALIDATOR ] FCX: All expected trading days are present. [2025-10-20T18:44:22.380179] [DEBUG] [VALIDATOR ] FCX: Primary data validation passed. [2025-10-20T18:44:22.381168] [INFO ] [MAIN ] FCX: Fetched primary data is valid. [2025-10-20T18:44:22.381236] [DEBUG] [LOCAL ] FCX: Saving primary data. [2025-10-20T18:44:22.382982] [DEBUG] [LOCAL ] FCX: Trimmed 77878 records to 53701 (keeping 200 days of data) [2025-10-20T18:44:22.651425] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-07. [2025-10-20T18:44:22.756463] [DEBUG] [LOCAL ] FCX: Primary data saved (53701 records). [2025-10-20T18:44:22.756625] [INFO ] [MAIN ] FCX: Full history saved. [2025-10-20T18:44:22.756748] [INFO ] [MAIN ] Update Progress: 815/2367 symbols completed. FCX took 44.6s. Estimated remaining: 3h 18m 38s. Rate: 7.8 symbols/min [2025-10-20T18:44:23.001665] [INFO ] [MAIN ] FERG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:23.067785] [DEBUG] [LOCAL ] FERG: Loaded 49492 existing records from primary storage. [2025-10-20T18:44:23.068235] [INFO ] [MAIN ] FERG: Loaded 49492 existing records. [2025-10-20T18:44:23.334638] [WARN ] [MAIN ] FERG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:23.334933] [INFO ] [MAIN ] FERG: Wiping existing primary data before full refresh. [2025-10-20T18:44:23.334983] [INFO ] [LOCAL ] FERG: Starting deletion of primary data. [2025-10-20T18:44:23.335468] [DEBUG] [LOCAL ] FERG: Deleted primary data file. [2025-10-20T18:44:23.335507] [INFO ] [MAIN ] FERG: Starting full primary refresh. [2025-10-20T18:44:23.455400] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-02. [2025-10-20T18:44:23.815213] [INFO ] [REMOTE ] FERG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:23.850494] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T18:44:23.918494] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T18:44:24.251368] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-01. [2025-10-20T18:44:24.766556] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-10-20T18:44:25.051367] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-03. [2025-10-20T18:44:25.355748] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-03 (0.30 seconds). [2025-10-20T18:44:25.798119] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T18:44:25.851346] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-04. [2025-10-20T18:44:26.339780] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T18:44:26.651389] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-02. [2025-10-20T18:44:27.451419] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-08. [2025-10-20T18:44:28.156399] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T18:44:28.251365] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-03. [2025-10-20T18:44:29.051423] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-05. [2025-10-20T18:44:29.168899] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:44:29.485960] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T18:44:29.851368] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-09. [2025-10-20T18:44:30.081014] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T18:44:30.651355] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-04. [2025-10-20T18:44:31.451389] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-06. [2025-10-20T18:44:31.664725] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T18:44:31.870912] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T18:44:32.251389] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-10. [2025-10-20T18:44:32.408245] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:44:33.051369] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-05. [2025-10-20T18:44:33.180045] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T18:44:33.188134] [INFO ] [REMOTE ] FE: Full fetch complete (77475 records). [2025-10-20T18:44:33.188270] [DEBUG] [VALIDATOR ] FE: Validating primary data. [2025-10-20T18:44:33.578658] [DEBUG] [VALIDATOR ] FE: All expected trading days are present. [2025-10-20T18:44:33.578751] [DEBUG] [VALIDATOR ] FE: Primary data validation passed. [2025-10-20T18:44:33.579768] [INFO ] [MAIN ] FE: Fetched primary data is valid. [2025-10-20T18:44:33.579831] [DEBUG] [LOCAL ] FE: Saving primary data. [2025-10-20T18:44:33.581293] [DEBUG] [LOCAL ] FE: Trimmed 77475 records to 53436 (keeping 200 days of data) [2025-10-20T18:44:33.858903] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-09. [2025-10-20T18:44:33.902008] [DEBUG] [LOCAL ] FE: Primary data saved (53436 records). [2025-10-20T18:44:33.902141] [INFO ] [MAIN ] FE: Full history saved. [2025-10-20T18:44:33.902913] [INFO ] [MAIN ] Update Progress: 816/2367 symbols completed. FE took 26.2s. Estimated remaining: 3h 18m 37s. Rate: 7.8 symbols/min [2025-10-20T18:44:34.077044] [INFO ] [MAIN ] FET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:34.092735] [DEBUG] [LOCAL ] FET: Loaded 13043 existing records from primary storage. [2025-10-20T18:44:34.092979] [INFO ] [MAIN ] FET: Loaded 13043 existing records. [2025-10-20T18:44:34.269868] [WARN ] [MAIN ] FET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:34.270074] [INFO ] [MAIN ] FET: Wiping existing primary data before full refresh. [2025-10-20T18:44:34.270123] [INFO ] [LOCAL ] FET: Starting deletion of primary data. [2025-10-20T18:44:34.270282] [DEBUG] [LOCAL ] FET: Deleted primary data file. [2025-10-20T18:44:34.270323] [INFO ] [MAIN ] FET: Starting full primary refresh. [2025-10-20T18:44:34.629055] [INFO ] [REMOTE ] FET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:34.651338] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-01. [2025-10-20T18:44:34.838443] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T18:44:35.321097] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T18:44:35.451362] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-06. [2025-10-20T18:44:35.684815] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:44:36.251407] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-07. [2025-10-20T18:44:36.631232] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-10-20T18:44:37.056519] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-08. [2025-10-20T18:44:37.072763] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:44:37.374952] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-10-20T18:44:37.851405] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-09. [2025-10-20T18:44:38.314860] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T18:44:38.651373] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-10. [2025-10-20T18:44:39.109251] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T18:44:39.110076] [INFO ] [REMOTE ] FENG: Full fetch complete (2278 records). [2025-10-20T18:44:39.110143] [DEBUG] [VALIDATOR ] FENG: Validating primary data. [2025-10-20T18:44:39.285985] [WARN ] [VALIDATOR ] FENG: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T18:44:39.286195] [FATAL] [MAIN ] FENG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:44:39.286265] [WARN ] [MAIN ] FENG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:44:39.286336] [INFO ] [LOCAL ] FENG: Starting deletion of primary data. [2025-10-20T18:44:39.286399] [DEBUG] [LOCAL ] FENG: No primary data file found to delete. [2025-10-20T18:44:39.286434] [INFO ] [LOCAL ] FENG: Starting deletion of derivative data. [2025-10-20T18:44:39.286474] [DEBUG] [LOCAL ] FENG: No derivative data file found to delete. [2025-10-20T18:44:39.286526] [INFO ] [MAIN ] Update Progress: 817/2367 symbols completed. FENG took 20.7s. Estimated remaining: 3h 18m 25s. Rate: 7.8 symbols/min [2025-10-20T18:44:39.455598] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-02. [2025-10-20T18:44:39.457298] [INFO ] [MAIN ] FF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:39.481985] [DEBUG] [LOCAL ] FF: Loaded 24686 existing records from primary storage. [2025-10-20T18:44:39.482224] [INFO ] [MAIN ] FF: Loaded 24686 existing records. [2025-10-20T18:44:39.656364] [WARN ] [MAIN ] FF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:39.656574] [INFO ] [MAIN ] FF: Wiping existing primary data before full refresh. [2025-10-20T18:44:39.656620] [INFO ] [LOCAL ] FF: Starting deletion of primary data. [2025-10-20T18:44:39.656857] [DEBUG] [LOCAL ] FF: Deleted primary data file. [2025-10-20T18:44:39.656913] [INFO ] [MAIN ] FF: Starting full primary refresh. [2025-10-20T18:44:40.032736] [INFO ] [REMOTE ] FF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:40.200009] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T18:44:40.251408] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-01. [2025-10-20T18:44:41.051383] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-07. [2025-10-20T18:44:41.616155] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T18:44:41.851439] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-02. [2025-10-20T18:44:42.651417] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-03. [2025-10-20T18:44:42.864690] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:44:43.071413] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T18:44:43.451386] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-10. [2025-10-20T18:44:43.492196] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T18:44:44.251412] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-04. [2025-10-20T18:44:45.051414] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-08. [2025-10-20T18:44:45.166988] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T18:44:45.841256] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-10 (2.39 seconds). [2025-10-20T18:44:45.851525] [INFO ] [REMOTE ] FDX: Full fetch complete (74329 records). [2025-10-20T18:44:45.851353] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-05. [2025-10-20T18:44:45.852392] [DEBUG] [VALIDATOR ] FDX: Validating primary data. [2025-10-20T18:44:46.246365] [DEBUG] [VALIDATOR ] FDX: All expected trading days are present. [2025-10-20T18:44:46.246453] [DEBUG] [VALIDATOR ] FDX: Primary data validation passed. [2025-10-20T18:44:46.247341] [INFO ] [MAIN ] FDX: Fetched primary data is valid. [2025-10-20T18:44:46.247405] [DEBUG] [LOCAL ] FDX: Saving primary data. [2025-10-20T18:44:46.248892] [DEBUG] [LOCAL ] FDX: Trimmed 74329 records to 51326 (keeping 200 days of data) [2025-10-20T18:44:46.564036] [DEBUG] [LOCAL ] FDX: Primary data saved (51326 records). [2025-10-20T18:44:46.564157] [INFO ] [MAIN ] FDX: Full history saved. [2025-10-20T18:44:46.564274] [INFO ] [MAIN ] Update Progress: 818/2367 symbols completed. FDX took 57.3s. Estimated remaining: 3h 18m 16s. Rate: 7.8 symbols/min [2025-10-20T18:44:46.654690] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-03. [2025-10-20T18:44:46.749471] [INFO ] [MAIN ] FFA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:46.759032] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T18:44:46.791882] [DEBUG] [LOCAL ] FFA: Loaded 5984 existing records from primary storage. [2025-10-20T18:44:46.793627] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:44:46.793779] [INFO ] [MAIN ] FFA: Loaded 5984 existing records. [2025-10-20T18:44:46.975924] [WARN ] [MAIN ] FFA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:46.976181] [INFO ] [MAIN ] FFA: Wiping existing primary data before full refresh. [2025-10-20T18:44:46.976237] [INFO ] [LOCAL ] FFA: Starting deletion of primary data. [2025-10-20T18:44:46.976378] [DEBUG] [LOCAL ] FFA: Deleted primary data file. [2025-10-20T18:44:46.976415] [INFO ] [MAIN ] FFA: Starting full primary refresh. [2025-10-20T18:44:47.323894] [INFO ] [REMOTE ] FFA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:47.451417] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-09. [2025-10-20T18:44:47.975578] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:44:48.251395] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-06. [2025-10-20T18:44:49.049012] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T18:44:49.051791] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-04. [2025-10-20T18:44:49.139284] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:44:49.851377] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-07. [2025-10-20T18:44:50.297375] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T18:44:50.651401] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-10. [2025-10-20T18:44:50.698637] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T18:44:51.451372] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-01. [2025-10-20T18:44:52.022677] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T18:44:52.251380] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-05. [2025-10-20T18:44:52.387437] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T18:44:52.395193] [INFO ] [REMOTE ] FERG: Full fetch complete (71210 records). [2025-10-20T18:44:52.395388] [DEBUG] [VALIDATOR ] FERG: Validating primary data. [2025-10-20T18:44:52.793497] [DEBUG] [VALIDATOR ] FERG: All expected trading days are present. [2025-10-20T18:44:52.793585] [DEBUG] [VALIDATOR ] FERG: Primary data validation passed. [2025-10-20T18:44:52.794438] [INFO ] [MAIN ] FERG: Fetched primary data is valid. [2025-10-20T18:44:52.794502] [DEBUG] [LOCAL ] FERG: Saving primary data. [2025-10-20T18:44:52.795875] [DEBUG] [LOCAL ] FERG: Trimmed 71210 records to 48769 (keeping 200 days of data) [2025-10-20T18:44:53.067880] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-02. [2025-10-20T18:44:53.100529] [DEBUG] [LOCAL ] FERG: Primary data saved (48769 records). [2025-10-20T18:44:53.100676] [INFO ] [MAIN ] FERG: Full history saved. [2025-10-20T18:44:53.100795] [INFO ] [MAIN ] Update Progress: 819/2367 symbols completed. FERG took 30.3s. Estimated remaining: 3h 18m 6s. Rate: 7.8 symbols/min [2025-10-20T18:44:53.283445] [INFO ] [MAIN ] FFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:53.301547] [DEBUG] [LOCAL ] FFC: Loaded 16896 existing records from primary storage. [2025-10-20T18:44:53.301821] [INFO ] [MAIN ] FFC: Loaded 16896 existing records. [2025-10-20T18:44:53.486767] [WARN ] [MAIN ] FFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:53.487068] [INFO ] [MAIN ] FFC: Wiping existing primary data before full refresh. [2025-10-20T18:44:53.487148] [INFO ] [LOCAL ] FFC: Starting deletion of primary data. [2025-10-20T18:44:53.487475] [DEBUG] [LOCAL ] FFC: Deleted primary data file. [2025-10-20T18:44:53.487538] [INFO ] [MAIN ] FFC: Starting full primary refresh. [2025-10-20T18:44:53.623212] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T18:44:53.708950] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T18:44:53.857066] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-03. [2025-10-20T18:44:53.897554] [INFO ] [REMOTE ] FFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:54.580911] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T18:44:54.651373] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-04. [2025-10-20T18:44:55.444520] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T18:44:55.451331] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-08. [2025-10-20T18:44:56.251370] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-05. [2025-10-20T18:44:56.494736] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:44:56.880516] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T18:44:57.051384] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-06. [2025-10-20T18:44:57.851395] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-01. [2025-10-20T18:44:58.086675] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:44:58.632443] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T18:44:58.651374] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-09. [2025-10-20T18:44:59.451352] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-02. [2025-10-20T18:44:59.750242] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T18:45:00.148902] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T18:45:00.251361] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-07. [2025-10-20T18:45:01.051355] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-03. [2025-10-20T18:45:01.376069] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:45:01.851375] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-08. [2025-10-20T18:45:02.186590] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:45:02.651392] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-10. [2025-10-20T18:45:03.001816] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T18:45:03.451359] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-04. [2025-10-20T18:45:03.703072] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T18:45:03.705569] [INFO ] [REMOTE ] FET: Full fetch complete (18434 records). [2025-10-20T18:45:03.705638] [DEBUG] [VALIDATOR ] FET: Validating primary data. [2025-10-20T18:45:04.075859] [DEBUG] [VALIDATOR ] FET: All expected trading days are present. [2025-10-20T18:45:04.076284] [DEBUG] [VALIDATOR ] FET: Primary data validation passed. [2025-10-20T18:45:04.076713] [INFO ] [MAIN ] FET: Fetched primary data is valid. [2025-10-20T18:45:04.076777] [DEBUG] [LOCAL ] FET: Saving primary data. [2025-10-20T18:45:04.077339] [DEBUG] [LOCAL ] FET: Trimmed 18434 records to 12965 (keeping 200 days of data) [2025-10-20T18:45:04.153221] [DEBUG] [LOCAL ] FET: Primary data saved (12965 records). [2025-10-20T18:45:04.153362] [INFO ] [MAIN ] FET: Full history saved. [2025-10-20T18:45:04.153451] [INFO ] [MAIN ] Update Progress: 820/2367 symbols completed. FET took 30.3s. Estimated remaining: 3h 18m 5s. Rate: 7.8 symbols/min [2025-10-20T18:45:04.256602] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-09. [2025-10-20T18:45:04.331124] [INFO ] [MAIN ] FFWM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:04.369951] [DEBUG] [LOCAL ] FFWM: Loaded 36515 existing records from primary storage. [2025-10-20T18:45:04.370211] [INFO ] [MAIN ] FFWM: Loaded 36515 existing records. [2025-10-20T18:45:04.560252] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T18:45:04.563879] [WARN ] [MAIN ] FFWM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:04.567574] [INFO ] [MAIN ] FFWM: Wiping existing primary data before full refresh. [2025-10-20T18:45:04.568020] [INFO ] [LOCAL ] FFWM: Starting deletion of primary data. [2025-10-20T18:45:04.568620] [DEBUG] [LOCAL ] FFWM: Deleted primary data file. [2025-10-20T18:45:04.568715] [INFO ] [MAIN ] FFWM: Starting full primary refresh. [2025-10-20T18:45:04.919736] [INFO ] [REMOTE ] FFWM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:05.051342] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-01. [2025-10-20T18:45:05.669387] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T18:45:05.851398] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-05. [2025-10-20T18:45:06.633336] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T18:45:06.651364] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-02. [2025-10-20T18:45:06.964552] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T18:45:07.451421] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-10. [2025-10-20T18:45:07.975959] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:45:08.251358] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-06. [2025-10-20T18:45:08.655074] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:45:08.658662] [INFO ] [REMOTE ] FF: Full fetch complete (36688 records). [2025-10-20T18:45:08.658744] [DEBUG] [VALIDATOR ] FF: Validating primary data. [2025-10-20T18:45:09.034902] [DEBUG] [VALIDATOR ] FF: All expected trading days are present. [2025-10-20T18:45:09.034998] [DEBUG] [VALIDATOR ] FF: Primary data validation passed. [2025-10-20T18:45:09.035755] [INFO ] [MAIN ] FF: Fetched primary data is valid. [2025-10-20T18:45:09.035818] [DEBUG] [LOCAL ] FF: Saving primary data. [2025-10-20T18:45:09.036618] [DEBUG] [LOCAL ] FF: Trimmed 36688 records to 24052 (keeping 200 days of data) [2025-10-20T18:45:09.082922] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-03. [2025-10-20T18:45:09.143804] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T18:45:09.181532] [DEBUG] [LOCAL ] FF: Primary data saved (24052 records). [2025-10-20T18:45:09.181673] [INFO ] [MAIN ] FF: Full history saved. [2025-10-20T18:45:09.181767] [INFO ] [MAIN ] Update Progress: 821/2367 symbols completed. FF took 29.9s. Estimated remaining: 3h 17m 52s. Rate: 7.8 symbols/min [2025-10-20T18:45:09.353599] [INFO ] [MAIN ] FG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:09.374477] [DEBUG] [LOCAL ] FG: Loaded 22484 existing records from primary storage. [2025-10-20T18:45:09.374723] [INFO ] [MAIN ] FG: Loaded 22484 existing records. [2025-10-20T18:45:09.550207] [WARN ] [MAIN ] FG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:09.550436] [INFO ] [MAIN ] FG: Wiping existing primary data before full refresh. [2025-10-20T18:45:09.550482] [INFO ] [LOCAL ] FG: Starting deletion of primary data. [2025-10-20T18:45:09.550697] [DEBUG] [LOCAL ] FG: Deleted primary data file. [2025-10-20T18:45:09.550762] [INFO ] [MAIN ] FG: Starting full primary refresh. [2025-10-20T18:45:09.863783] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-07. [2025-10-20T18:45:09.908456] [INFO ] [REMOTE ] FG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:10.500673] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T18:45:10.499869] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T18:45:10.651359] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-04. [2025-10-20T18:45:11.451393] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-08. [2025-10-20T18:45:11.973419] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T18:45:12.157846] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T18:45:12.251366] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-01. [2025-10-20T18:45:12.836177] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T18:45:13.051385] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-09. [2025-10-20T18:45:13.764320] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T18:45:13.851356] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-05. [2025-10-20T18:45:14.651398] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-10. [2025-10-20T18:45:15.312024] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T18:45:15.330782] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T18:45:15.333099] [INFO ] [REMOTE ] FFA: Full fetch complete (8628 records). [2025-10-20T18:45:15.333195] [DEBUG] [VALIDATOR ] FFA: Validating primary data. [2025-10-20T18:45:15.333703] [FATAL] [VALIDATOR ] FFA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:45:15.333803] [FATAL] [MAIN ] FFA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:45:15.333860] [WARN ] [MAIN ] FFA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:45:15.333939] [INFO ] [LOCAL ] FFA: Starting deletion of primary data. [2025-10-20T18:45:15.334036] [DEBUG] [LOCAL ] FFA: No primary data file found to delete. [2025-10-20T18:45:15.334086] [INFO ] [LOCAL ] FFA: Starting deletion of derivative data. [2025-10-20T18:45:15.334292] [DEBUG] [LOCAL ] FFA: Deleted derivative data file. [2025-10-20T18:45:15.334505] [INFO ] [MAIN ] Update Progress: 822/2367 symbols completed. FFA took 28.8s. Estimated remaining: 3h 17m 42s. Rate: 7.8 symbols/min [2025-10-20T18:45:15.456515] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-02. [2025-10-20T18:45:15.549220] [INFO ] [MAIN ] FGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:15.556627] [DEBUG] [LOCAL ] FGN: Loaded 3750 existing records from primary storage. [2025-10-20T18:45:15.556945] [INFO ] [MAIN ] FGN: Loaded 3750 existing records. [2025-10-20T18:45:15.796914] [WARN ] [MAIN ] FGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:15.797144] [INFO ] [MAIN ] FGN: Wiping existing primary data before full refresh. [2025-10-20T18:45:15.797194] [INFO ] [LOCAL ] FGN: Starting deletion of primary data. [2025-10-20T18:45:15.797379] [DEBUG] [LOCAL ] FGN: Deleted primary data file. [2025-10-20T18:45:15.797423] [INFO ] [MAIN ] FGN: Starting full primary refresh. [2025-10-20T18:45:16.256551] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-06. [2025-10-20T18:45:16.267240] [INFO ] [REMOTE ] FGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:16.323709] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T18:45:17.051369] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-03. [2025-10-20T18:45:17.254032] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T18:45:17.851399] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-07. [2025-10-20T18:45:18.178134] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T18:45:18.651412] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-06. [2025-10-20T18:45:18.913496] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T18:45:19.451452] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-01. [2025-10-20T18:45:20.095843] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T18:45:20.122631] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T18:45:20.251396] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-07. [2025-10-20T18:45:21.051457] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-04. [2025-10-20T18:45:21.588893] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T18:45:21.851423] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-02. [2025-10-20T18:45:22.333711] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T18:45:22.355869] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T18:45:22.651399] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-03. [2025-10-20T18:45:23.246427] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T18:45:23.451429] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-08. [2025-10-20T18:45:24.251422] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-05. [2025-10-20T18:45:24.927377] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:45:25.051420] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-09. [2025-10-20T18:45:25.421199] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:45:25.851377] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-08. [2025-10-20T18:45:26.246591] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:45:26.651361] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-04. [2025-10-20T18:45:26.803723] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T18:45:27.252441] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T18:45:27.451383] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-06. [2025-10-20T18:45:28.251355] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-09. [2025-10-20T18:45:28.395489] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:45:29.051403] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-05. [2025-10-20T18:45:29.512161] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T18:45:29.542175] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T18:45:29.851438] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-10. [2025-10-20T18:45:30.651382] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-10. [2025-10-20T18:45:30.780993] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T18:45:30.783510] [INFO ] [REMOTE ] FFC: Full fetch complete (24406 records). [2025-10-20T18:45:30.783582] [DEBUG] [VALIDATOR ] FFC: Validating primary data. [2025-10-20T18:45:31.153279] [DEBUG] [VALIDATOR ] FFC: All expected trading days are present. [2025-10-20T18:45:31.153354] [DEBUG] [VALIDATOR ] FFC: Primary data validation passed. [2025-10-20T18:45:31.153858] [INFO ] [MAIN ] FFC: Fetched primary data is valid. [2025-10-20T18:45:31.153918] [DEBUG] [LOCAL ] FFC: Saving primary data. [2025-10-20T18:45:31.154521] [DEBUG] [LOCAL ] FFC: Trimmed 24406 records to 16622 (keeping 200 days of data) [2025-10-20T18:45:31.251993] [DEBUG] [LOCAL ] FFC: Primary data saved (16622 records). [2025-10-20T18:45:31.252136] [INFO ] [MAIN ] FFC: Full history saved. [2025-10-20T18:45:31.252212] [INFO ] [MAIN ] Update Progress: 823/2367 symbols completed. FFC took 38.2s. Estimated remaining: 3h 17m 50s. Rate: 7.8 symbols/min [2025-10-20T18:45:31.419849] [INFO ] [MAIN ] FGSN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:31.420018] [DEBUG] [LOCAL ] FGSN: No existing data file found in primary storage. [2025-10-20T18:45:31.420062] [WARN ] [MAIN ] FGSN: No existing data found, performing full refresh instead. [2025-10-20T18:45:31.420172] [INFO ] [MAIN ] FGSN: Wiping existing primary data before full refresh. [2025-10-20T18:45:31.420219] [INFO ] [LOCAL ] FGSN: Starting deletion of primary data. [2025-10-20T18:45:31.420258] [DEBUG] [LOCAL ] FGSN: No primary data file found to delete. [2025-10-20T18:45:31.420290] [INFO ] [MAIN ] FGSN: Starting full primary refresh. [2025-10-20T18:45:31.456553] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-07. [2025-10-20T18:45:31.781099] [INFO ] [REMOTE ] FGSN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:32.226021] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T18:45:32.231572] [INFO ] [REMOTE ] FFWM: Full fetch complete (51973 records). [2025-10-20T18:45:32.231728] [DEBUG] [VALIDATOR ] FFWM: Validating primary data. [2025-10-20T18:45:32.263145] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-01. [2025-10-20T18:45:32.616534] [DEBUG] [VALIDATOR ] FFWM: All expected trading days are present. [2025-10-20T18:45:32.616771] [DEBUG] [VALIDATOR ] FFWM: Primary data validation passed. [2025-10-20T18:45:32.617556] [INFO ] [MAIN ] FFWM: Fetched primary data is valid. [2025-10-20T18:45:32.617636] [DEBUG] [LOCAL ] FFWM: Saving primary data. [2025-10-20T18:45:32.618719] [DEBUG] [LOCAL ] FFWM: Trimmed 51973 records to 35833 (keeping 200 days of data) [2025-10-20T18:45:32.736222] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:45:32.829452] [DEBUG] [LOCAL ] FFWM: Primary data saved (35833 records). [2025-10-20T18:45:32.829587] [INFO ] [MAIN ] FFWM: Full history saved. [2025-10-20T18:45:32.829688] [INFO ] [MAIN ] Update Progress: 824/2367 symbols completed. FFWM took 28.7s. Estimated remaining: 3h 17m 30s. Rate: 7.8 symbols/min [2025-10-20T18:45:33.000539] [INFO ] [MAIN ] FHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:33.038704] [DEBUG] [LOCAL ] FHI: Loaded 40432 existing records from primary storage. [2025-10-20T18:45:33.039010] [INFO ] [MAIN ] FHI: Loaded 40432 existing records. [2025-10-20T18:45:33.060685] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-06. [2025-10-20T18:45:33.229822] [WARN ] [MAIN ] FHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:33.230808] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T18:45:33.233378] [INFO ] [MAIN ] FHI: Wiping existing primary data before full refresh. [2025-10-20T18:45:33.234098] [INFO ] [LOCAL ] FHI: Starting deletion of primary data. [2025-10-20T18:45:33.234419] [DEBUG] [LOCAL ] FHI: Deleted primary data file. [2025-10-20T18:45:33.234477] [INFO ] [MAIN ] FHI: Starting full primary refresh. [2025-10-20T18:45:33.578838] [INFO ] [REMOTE ] FHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:33.851366] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-02. [2025-10-20T18:45:34.003631] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:45:34.378217] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:45:34.651389] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-03. [2025-10-20T18:45:35.336892] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T18:45:35.451360] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-04. [2025-10-20T18:45:36.201739] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T18:45:36.251342] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-05. [2025-10-20T18:45:36.757879] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T18:45:37.051440] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-06. [2025-10-20T18:45:37.654830] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T18:45:37.851412] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-07. [2025-10-20T18:45:38.331453] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T18:45:38.651407] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-01. [2025-10-20T18:45:39.451415] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-07. [2025-10-20T18:45:40.019603] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T18:45:40.251358] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-08. [2025-10-20T18:45:40.298885] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T18:45:40.857185] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T18:45:41.051373] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-09. [2025-10-20T18:45:41.623845] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T18:45:41.851402] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-10. [2025-10-20T18:45:42.496437] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T18:45:42.497545] [INFO ] [REMOTE ] FGSN: Full fetch complete (8036 records). [2025-10-20T18:45:42.497611] [DEBUG] [VALIDATOR ] FGSN: Validating primary data. [2025-10-20T18:45:42.652006] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-08. [2025-10-20T18:45:42.677824] [WARN ] [VALIDATOR ] FGSN: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-10-20T18:45:42.679279] [FATAL] [MAIN ] FGSN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:45:42.679354] [WARN ] [MAIN ] FGSN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:45:42.679417] [INFO ] [LOCAL ] FGSN: Starting deletion of primary data. [2025-10-20T18:45:42.679473] [DEBUG] [LOCAL ] FGSN: No primary data file found to delete. [2025-10-20T18:45:42.679508] [INFO ] [LOCAL ] FGSN: Starting deletion of derivative data. [2025-10-20T18:45:42.679547] [DEBUG] [LOCAL ] FGSN: No derivative data file found to delete. [2025-10-20T18:45:42.679604] [INFO ] [MAIN ] Update Progress: 825/2367 symbols completed. FGSN took 11.4s. Estimated remaining: 3h 17m 27s. Rate: 7.8 symbols/min [2025-10-20T18:45:42.850419] [INFO ] [MAIN ] FHN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:42.900914] [DEBUG] [LOCAL ] FHN: Loaded 54172 existing records from primary storage. [2025-10-20T18:45:42.901280] [INFO ] [MAIN ] FHN: Loaded 54172 existing records. [2025-10-20T18:45:43.078335] [WARN ] [MAIN ] FHN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:43.078550] [INFO ] [MAIN ] FHN: Wiping existing primary data before full refresh. [2025-10-20T18:45:43.078595] [INFO ] [LOCAL ] FHN: Starting deletion of primary data. [2025-10-20T18:45:43.078806] [DEBUG] [LOCAL ] FHN: Deleted primary data file. [2025-10-20T18:45:43.078861] [INFO ] [MAIN ] FHN: Starting full primary refresh. [2025-10-20T18:45:43.456566] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-02. [2025-10-20T18:45:43.462243] [INFO ] [REMOTE ] FHN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:43.860493] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T18:45:44.251384] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-08. [2025-10-20T18:45:44.475718] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:45:44.712143] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T18:45:45.051390] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-03. [2025-10-20T18:45:45.851386] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-09. [2025-10-20T18:45:46.512469] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T18:45:46.651353] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-04. [2025-10-20T18:45:46.865526] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T18:45:47.451394] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-01. [2025-10-20T18:45:48.251356] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-10. [2025-10-20T18:45:48.308633] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T18:45:49.051365] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-05. [2025-10-20T18:45:49.238600] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:45:49.449904] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:45:49.453450] [INFO ] [REMOTE ] FG: Full fetch complete (28853 records). [2025-10-20T18:45:49.453539] [DEBUG] [VALIDATOR ] FG: Validating primary data. [2025-10-20T18:45:49.827604] [DEBUG] [VALIDATOR ] FG: All expected trading days are present. [2025-10-20T18:45:49.827709] [DEBUG] [VALIDATOR ] FG: Primary data validation passed. [2025-10-20T18:45:49.828299] [INFO ] [MAIN ] FG: Fetched primary data is valid. [2025-10-20T18:45:49.828339] [DEBUG] [LOCAL ] FG: Saving primary data. [2025-10-20T18:45:49.829086] [DEBUG] [LOCAL ] FG: Trimmed 28853 records to 21737 (keeping 200 days of data) [2025-10-20T18:45:49.879877] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-09. [2025-10-20T18:45:49.959818] [DEBUG] [LOCAL ] FG: Primary data saved (21737 records). [2025-10-20T18:45:49.959946] [INFO ] [MAIN ] FG: Full history saved. [2025-10-20T18:45:49.960023] [INFO ] [MAIN ] Update Progress: 826/2367 symbols completed. FG took 40.8s. Estimated remaining: 3h 17m 18s. Rate: 7.8 symbols/min [2025-10-20T18:45:50.133795] [INFO ] [MAIN ] FI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:50.186369] [DEBUG] [LOCAL ] FI: Loaded 54357 existing records from primary storage. [2025-10-20T18:45:50.186817] [INFO ] [MAIN ] FI: Loaded 54357 existing records. [2025-10-20T18:45:50.381512] [WARN ] [MAIN ] FI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:50.382042] [INFO ] [MAIN ] FI: Wiping existing primary data before full refresh. [2025-10-20T18:45:50.382152] [INFO ] [LOCAL ] FI: Starting deletion of primary data. [2025-10-20T18:45:50.382344] [DEBUG] [LOCAL ] FI: Deleted primary data file. [2025-10-20T18:45:50.382388] [INFO ] [MAIN ] FI: Starting full primary refresh. [2025-10-20T18:45:50.473151] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T18:45:50.658785] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-02. [2025-10-20T18:45:50.775083] [INFO ] [REMOTE ] FI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:50.802702] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T18:45:51.451373] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-01. [2025-10-20T18:45:52.251351] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-06. [2025-10-20T18:45:52.305833] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:45:52.910059] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T18:45:53.051328] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-02. [2025-10-20T18:45:53.729967] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:45:53.851553] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-07. [2025-10-20T18:45:54.651360] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-10. [2025-10-20T18:45:54.846533] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T18:45:55.344912] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:45:55.442675] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T18:45:55.443553] [INFO ] [REMOTE ] FGN: Full fetch complete (5552 records). [2025-10-20T18:45:55.443617] [DEBUG] [VALIDATOR ] FGN: Validating primary data. [2025-10-20T18:45:55.456507] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-08. [2025-10-20T18:45:55.794821] [DEBUG] [VALIDATOR ] FGN: All expected trading days are present. [2025-10-20T18:45:55.794898] [DEBUG] [VALIDATOR ] FGN: Primary data validation passed. [2025-10-20T18:45:55.795050] [INFO ] [MAIN ] FGN: Fetched primary data is valid. [2025-10-20T18:45:55.795088] [DEBUG] [LOCAL ] FGN: Saving primary data. [2025-10-20T18:45:55.795447] [DEBUG] [LOCAL ] FGN: Trimmed 5552 records to 3636 (keeping 200 days of data) [2025-10-20T18:45:55.817332] [DEBUG] [LOCAL ] FGN: Primary data saved (3636 records). [2025-10-20T18:45:55.817440] [INFO ] [MAIN ] FGN: Full history saved. [2025-10-20T18:45:55.817512] [INFO ] [MAIN ] Update Progress: 827/2367 symbols completed. FGN took 40.5s. Estimated remaining: 3h 17m 7s. Rate: 7.8 symbols/min [2025-10-20T18:45:55.982149] [INFO ] [MAIN ] FICO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:56.004424] [DEBUG] [LOCAL ] FICO: Loaded 22042 existing records from primary storage. [2025-10-20T18:45:56.004663] [INFO ] [MAIN ] FICO: Loaded 22042 existing records. [2025-10-20T18:45:56.179706] [WARN ] [MAIN ] FICO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:56.179908] [INFO ] [MAIN ] FICO: Wiping existing primary data before full refresh. [2025-10-20T18:45:56.179953] [INFO ] [LOCAL ] FICO: Starting deletion of primary data. [2025-10-20T18:45:56.180111] [DEBUG] [LOCAL ] FICO: Deleted primary data file. [2025-10-20T18:45:56.180147] [INFO ] [MAIN ] FICO: Starting full primary refresh. [2025-10-20T18:45:56.256435] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-03. [2025-10-20T18:45:56.528000] [INFO ] [REMOTE ] FICO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:57.022165] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:45:57.051333] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-09. [2025-10-20T18:45:57.851400] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-01. [2025-10-20T18:45:58.249099] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T18:45:58.513831] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T18:45:58.651401] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-10. [2025-10-20T18:45:58.939445] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T18:45:59.451366] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-02. [2025-10-20T18:46:00.069842] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:46:00.075256] [INFO ] [REMOTE ] FHI: Full fetch complete (57678 records). [2025-10-20T18:46:00.075403] [DEBUG] [VALIDATOR ] FHI: Validating primary data. [2025-10-20T18:46:00.261959] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-03. [2025-10-20T18:46:00.475467] [DEBUG] [VALIDATOR ] FHI: All expected trading days are present. [2025-10-20T18:46:00.475679] [DEBUG] [VALIDATOR ] FHI: Primary data validation passed. [2025-10-20T18:46:00.476433] [INFO ] [MAIN ] FHI: Fetched primary data is valid. [2025-10-20T18:46:00.476477] [DEBUG] [LOCAL ] FHI: Saving primary data. [2025-10-20T18:46:00.477736] [DEBUG] [LOCAL ] FHI: Trimmed 57678 records to 39793 (keeping 200 days of data) [2025-10-20T18:46:00.591372] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:46:00.728400] [DEBUG] [LOCAL ] FHI: Primary data saved (39793 records). [2025-10-20T18:46:00.728540] [INFO ] [MAIN ] FHI: Full history saved. [2025-10-20T18:46:00.728621] [INFO ] [MAIN ] Update Progress: 828/2367 symbols completed. FHI took 27.9s. Estimated remaining: 3h 16m 54s. Rate: 7.8 symbols/min [2025-10-20T18:46:00.894832] [INFO ] [MAIN ] FIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:00.894977] [DEBUG] [LOCAL ] FIG: No existing data file found in primary storage. [2025-10-20T18:46:00.895014] [WARN ] [MAIN ] FIG: No existing data found, performing full refresh instead. [2025-10-20T18:46:00.895125] [INFO ] [MAIN ] FIG: Wiping existing primary data before full refresh. [2025-10-20T18:46:00.895171] [INFO ] [LOCAL ] FIG: Starting deletion of primary data. [2025-10-20T18:46:00.895248] [DEBUG] [LOCAL ] FIG: No primary data file found to delete. [2025-10-20T18:46:00.895294] [INFO ] [MAIN ] FIG: Starting full primary refresh. [2025-10-20T18:46:01.068991] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-04. [2025-10-20T18:46:01.254933] [INFO ] [REMOTE ] FIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:01.851360] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-01. [2025-10-20T18:46:02.106779] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-01 (0.26 seconds). [2025-10-20T18:46:02.541796] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-03 (2.28 seconds). [2025-10-20T18:46:02.651379] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-03. [2025-10-20T18:46:03.080486] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T18:46:03.451419] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-02. [2025-10-20T18:46:03.771902] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T18:46:03.787907] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:46:04.251421] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-04. [2025-10-20T18:46:05.051403] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-05. [2025-10-20T18:46:05.422809] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T18:46:05.851389] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-03. [2025-10-20T18:46:06.098343] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-03 (0.25 seconds). [2025-10-20T18:46:06.651351] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-05. [2025-10-20T18:46:06.790560] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:46:07.451417] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-04. [2025-10-20T18:46:07.920296] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:46:08.251399] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-04. [2025-10-20T18:46:08.560539] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-10-20T18:46:09.051435] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-06. [2025-10-20T18:46:09.485488] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T18:46:09.851403] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-06. [2025-10-20T18:46:10.218685] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T18:46:10.651416] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-05. [2025-10-20T18:46:11.353227] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T18:46:11.451410] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-07. [2025-10-20T18:46:12.251376] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-07. [2025-10-20T18:46:12.601063] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T18:46:12.863772] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:46:13.051366] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-08. [2025-10-20T18:46:13.699890] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T18:46:13.851396] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-05. [2025-10-20T18:46:14.187342] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-05 (0.34 seconds). [2025-10-20T18:46:14.651368] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-06. [2025-10-20T18:46:14.818222] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:46:15.451391] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-09. [2025-10-20T18:46:16.251372] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-08. [2025-10-20T18:46:16.573232] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T18:46:17.051429] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-06. [2025-10-20T18:46:17.192068] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T18:46:17.254512] [DEBUG] [REMOTE ] FIG: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:46:17.254908] [WARN ] [REMOTE ] FIG: No data returned for month 2025-06 [2025-10-20T18:46:17.255004] [FATAL] [REMOTE ] FIG: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:46:17.255083] [FATAL] [MAIN ] FIG: Failed to fetch primary data. [2025-10-20T18:46:17.255163] [WARN ] [MAIN ] FIG: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:46:17.255239] [INFO ] [LOCAL ] FIG: Starting deletion of primary data. [2025-10-20T18:46:17.255305] [DEBUG] [LOCAL ] FIG: No primary data file found to delete. [2025-10-20T18:46:17.255341] [INFO ] [LOCAL ] FIG: Starting deletion of derivative data. [2025-10-20T18:46:17.255384] [DEBUG] [LOCAL ] FIG: No derivative data file found to delete. [2025-10-20T18:46:17.255431] [INFO ] [MAIN ] Update Progress: 829/2367 symbols completed. FIG took 16.5s. Estimated remaining: 3h 17m 3s. Rate: 7.8 symbols/min [2025-10-20T18:46:17.437605] [INFO ] [MAIN ] FIGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:17.461253] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T18:46:17.509199] [DEBUG] [LOCAL ] FIGS: Loaded 49320 existing records from primary storage. [2025-10-20T18:46:17.509635] [INFO ] [MAIN ] FIGS: Loaded 49320 existing records. [2025-10-20T18:46:17.690382] [WARN ] [MAIN ] FIGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:17.690597] [INFO ] [MAIN ] FIGS: Wiping existing primary data before full refresh. [2025-10-20T18:46:17.690666] [INFO ] [LOCAL ] FIGS: Starting deletion of primary data. [2025-10-20T18:46:17.691096] [DEBUG] [LOCAL ] FIGS: Deleted primary data file. [2025-10-20T18:46:17.691153] [INFO ] [MAIN ] FIGS: Starting full primary refresh. [2025-10-20T18:46:17.852045] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-10. [2025-10-20T18:46:18.052955] [INFO ] [REMOTE ] FIGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:18.651377] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-07. [2025-10-20T18:46:19.451416] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-01. [2025-10-20T18:46:19.713165] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T18:46:19.721214] [INFO ] [REMOTE ] FHN: Full fetch complete (77474 records). [2025-10-20T18:46:19.721364] [DEBUG] [VALIDATOR ] FHN: Validating primary data. [2025-10-20T18:46:20.111593] [DEBUG] [VALIDATOR ] FHN: All expected trading days are present. [2025-10-20T18:46:20.111701] [DEBUG] [VALIDATOR ] FHN: Primary data validation passed. [2025-10-20T18:46:20.112603] [INFO ] [MAIN ] FHN: Fetched primary data is valid. [2025-10-20T18:46:20.112665] [DEBUG] [LOCAL ] FHN: Saving primary data. [2025-10-20T18:46:20.114370] [DEBUG] [LOCAL ] FHN: Trimmed 77474 records to 53394 (keeping 200 days of data) [2025-10-20T18:46:20.289461] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-09. [2025-10-20T18:46:20.445593] [DEBUG] [LOCAL ] FHN: Primary data saved (53394 records). [2025-10-20T18:46:20.445738] [INFO ] [MAIN ] FHN: Full history saved. [2025-10-20T18:46:20.445821] [INFO ] [MAIN ] Update Progress: 830/2367 symbols completed. FHN took 37.8s. Estimated remaining: 3h 16m 47s. Rate: 7.8 symbols/min [2025-10-20T18:46:20.619814] [INFO ] [MAIN ] FIHL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:20.674254] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T18:46:20.700341] [DEBUG] [LOCAL ] FIHL: Loaded 39982 existing records from primary storage. [2025-10-20T18:46:20.700675] [INFO ] [MAIN ] FIHL: Loaded 39982 existing records. [2025-10-20T18:46:20.880004] [WARN ] [MAIN ] FIHL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:20.880245] [INFO ] [MAIN ] FIHL: Wiping existing primary data before full refresh. [2025-10-20T18:46:20.880299] [INFO ] [LOCAL ] FIHL: Starting deletion of primary data. [2025-10-20T18:46:20.880483] [DEBUG] [LOCAL ] FIHL: Deleted primary data file. [2025-10-20T18:46:20.880537] [INFO ] [MAIN ] FIHL: Starting full primary refresh. [2025-10-20T18:46:21.063802] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-08. [2025-10-20T18:46:21.291409] [INFO ] [REMOTE ] FIHL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:21.285430] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:46:21.314515] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T18:46:21.851403] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-01. [2025-10-20T18:46:22.651404] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-02. [2025-10-20T18:46:22.996676] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T18:46:23.331074] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:46:23.451373] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-09. [2025-10-20T18:46:24.139574] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:46:24.251362] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-03. [2025-10-20T18:46:25.051424] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-02. [2025-10-20T18:46:25.554892] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-10-20T18:46:25.627830] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:46:25.851417] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-10. [2025-10-20T18:46:26.344346] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T18:46:26.651686] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-04. [2025-10-20T18:46:27.308863] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:46:27.313416] [INFO ] [REMOTE ] FICO: Full fetch complete (29935 records). [2025-10-20T18:46:27.313567] [DEBUG] [VALIDATOR ] FICO: Validating primary data. [2025-10-20T18:46:27.458168] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-03. [2025-10-20T18:46:27.702188] [DEBUG] [VALIDATOR ] FICO: All expected trading days are present. [2025-10-20T18:46:27.702265] [DEBUG] [VALIDATOR ] FICO: Primary data validation passed. [2025-10-20T18:46:27.702832] [INFO ] [MAIN ] FICO: Fetched primary data is valid. [2025-10-20T18:46:27.702893] [DEBUG] [LOCAL ] FICO: Saving primary data. [2025-10-20T18:46:27.703780] [DEBUG] [LOCAL ] FICO: Trimmed 29935 records to 21862 (keeping 200 days of data) [2025-10-20T18:46:27.836777] [DEBUG] [LOCAL ] FICO: Primary data saved (21862 records). [2025-10-20T18:46:27.836901] [INFO ] [MAIN ] FICO: Full history saved. [2025-10-20T18:46:27.836972] [INFO ] [MAIN ] Update Progress: 831/2367 symbols completed. FICO took 32.0s. Estimated remaining: 3h 16m 39s. Rate: 7.8 symbols/min [2025-10-20T18:46:28.002596] [INFO ] [MAIN ] FINS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:28.012145] [DEBUG] [LOCAL ] FINS: Loaded 8404 existing records from primary storage. [2025-10-20T18:46:28.012347] [INFO ] [MAIN ] FINS: Loaded 8404 existing records. [2025-10-20T18:46:28.190276] [WARN ] [MAIN ] FINS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:28.190453] [INFO ] [MAIN ] FINS: Wiping existing primary data before full refresh. [2025-10-20T18:46:28.190498] [INFO ] [LOCAL ] FINS: Starting deletion of primary data. [2025-10-20T18:46:28.190675] [DEBUG] [LOCAL ] FINS: Deleted primary data file. [2025-10-20T18:46:28.190730] [INFO ] [MAIN ] FINS: Starting full primary refresh. [2025-10-20T18:46:28.252973] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-10. [2025-10-20T18:46:28.546771] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T18:46:28.586942] [INFO ] [REMOTE ] FINS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:29.051379] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-01. [2025-10-20T18:46:29.158318] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:46:29.192490] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T18:46:29.200710] [INFO ] [REMOTE ] FI: Full fetch complete (77477 records). [2025-10-20T18:46:29.200900] [DEBUG] [VALIDATOR ] FI: Validating primary data. [2025-10-20T18:46:29.586485] [DEBUG] [VALIDATOR ] FI: All expected trading days are present. [2025-10-20T18:46:29.586579] [DEBUG] [VALIDATOR ] FI: Primary data validation passed. [2025-10-20T18:46:29.587516] [INFO ] [MAIN ] FI: Fetched primary data is valid. [2025-10-20T18:46:29.587588] [DEBUG] [LOCAL ] FI: Saving primary data. [2025-10-20T18:46:29.589091] [DEBUG] [LOCAL ] FI: Trimmed 77477 records to 53585 (keeping 200 days of data) [2025-10-20T18:46:29.804712] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T18:46:29.877750] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-05. [2025-10-20T18:46:29.931264] [DEBUG] [LOCAL ] FI: Primary data saved (53585 records). [2025-10-20T18:46:29.931432] [INFO ] [MAIN ] FI: Full history saved. [2025-10-20T18:46:29.931990] [INFO ] [MAIN ] Update Progress: 832/2367 symbols completed. FI took 40.0s. Estimated remaining: 3h 16m 21s. Rate: 7.8 symbols/min [2025-10-20T18:46:30.113278] [INFO ] [MAIN ] FINV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:30.165152] [DEBUG] [LOCAL ] FINV: Loaded 45943 existing records from primary storage. [2025-10-20T18:46:30.165469] [INFO ] [MAIN ] FINV: Loaded 45943 existing records. [2025-10-20T18:46:30.340460] [WARN ] [MAIN ] FINV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:30.340703] [INFO ] [MAIN ] FINV: Wiping existing primary data before full refresh. [2025-10-20T18:46:30.340762] [INFO ] [LOCAL ] FINV: Starting deletion of primary data. [2025-10-20T18:46:30.340955] [DEBUG] [LOCAL ] FINV: Deleted primary data file. [2025-10-20T18:46:30.341009] [INFO ] [MAIN ] FINV: Starting full primary refresh. [2025-10-20T18:46:30.652734] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-04. [2025-10-20T18:46:30.693168] [INFO ] [REMOTE ] FINV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:31.382508] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T18:46:31.451412] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-06. [2025-10-20T18:46:31.836321] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T18:46:32.251413] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-05. [2025-10-20T18:46:33.051369] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-01. [2025-10-20T18:46:33.092418] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:46:33.521337] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:46:33.851378] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-07. [2025-10-20T18:46:34.462027] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T18:46:34.651363] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-02. [2025-10-20T18:46:34.962118] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T18:46:35.451403] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-08. [2025-10-20T18:46:35.998672] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T18:46:36.251387] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-02. [2025-10-20T18:46:36.934865] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T18:46:37.051366] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-03. [2025-10-20T18:46:37.119242] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T18:46:37.851379] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-06. [2025-10-20T18:46:38.645634] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:46:38.654708] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-03. [2025-10-20T18:46:39.259052] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:46:39.272331] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:46:39.451358] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-07. [2025-10-20T18:46:40.251378] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-04. [2025-10-20T18:46:40.694155] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T18:46:41.051399] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-09. [2025-10-20T18:46:41.066797] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T18:46:41.851362] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-05. [2025-10-20T18:46:42.559483] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:46:42.643478] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T18:46:42.651319] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-06. [2025-10-20T18:46:43.410669] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T18:46:43.451408] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-08. [2025-10-20T18:46:44.251437] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-04. [2025-10-20T18:46:44.921184] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T18:46:45.051346] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-09. [2025-10-20T18:46:45.824182] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T18:46:45.851365] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-05. [2025-10-20T18:46:46.264734] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T18:46:46.651423] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-10. [2025-10-20T18:46:47.338172] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T18:46:47.451346] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-06. [2025-10-20T18:46:47.911300] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:46:47.917972] [INFO ] [REMOTE ] FIHL: Full fetch complete (56277 records). [2025-10-20T18:46:47.918119] [DEBUG] [VALIDATOR ] FIHL: Validating primary data. [2025-10-20T18:46:48.260148] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-10. [2025-10-20T18:46:48.379741] [DEBUG] [VALIDATOR ] FIHL: All expected trading days are present. [2025-10-20T18:46:48.379871] [DEBUG] [VALIDATOR ] FIHL: Primary data validation passed. [2025-10-20T18:46:48.381234] [INFO ] [MAIN ] FIHL: Fetched primary data is valid. [2025-10-20T18:46:48.381304] [DEBUG] [LOCAL ] FIHL: Saving primary data. [2025-10-20T18:46:48.383235] [DEBUG] [LOCAL ] FIHL: Trimmed 56277 records to 39199 (keeping 200 days of data) [2025-10-20T18:46:48.642279] [DEBUG] [LOCAL ] FIHL: Primary data saved (39199 records). [2025-10-20T18:46:48.642698] [INFO ] [MAIN ] FIHL: Full history saved. [2025-10-20T18:46:48.642823] [INFO ] [MAIN ] Update Progress: 833/2367 symbols completed. FIHL took 28.2s. Estimated remaining: 3h 16m 34s. Rate: 7.8 symbols/min [2025-10-20T18:46:48.839363] [INFO ] [MAIN ] FIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:48.897215] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T18:46:48.928602] [DEBUG] [LOCAL ] FIS: Loaded 54114 existing records from primary storage. [2025-10-20T18:46:48.929151] [INFO ] [MAIN ] FIS: Loaded 54114 existing records. [2025-10-20T18:46:49.052947] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-07. [2025-10-20T18:46:49.140797] [WARN ] [MAIN ] FIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:49.141113] [INFO ] [MAIN ] FIS: Wiping existing primary data before full refresh. [2025-10-20T18:46:49.141164] [INFO ] [LOCAL ] FIS: Starting deletion of primary data. [2025-10-20T18:46:49.141829] [DEBUG] [LOCAL ] FIS: Deleted primary data file. [2025-10-20T18:46:49.141889] [INFO ] [MAIN ] FIS: Starting full primary refresh. [2025-10-20T18:46:49.508612] [INFO ] [REMOTE ] FIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:49.851398] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-07. [2025-10-20T18:46:50.042808] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T18:46:50.051136] [INFO ] [REMOTE ] FIGS: Full fetch complete (70868 records). [2025-10-20T18:46:50.051425] [DEBUG] [VALIDATOR ] FIGS: Validating primary data. [2025-10-20T18:46:50.433354] [DEBUG] [VALIDATOR ] FIGS: All expected trading days are present. [2025-10-20T18:46:50.433438] [DEBUG] [VALIDATOR ] FIGS: Primary data validation passed. [2025-10-20T18:46:50.434351] [INFO ] [MAIN ] FIGS: Fetched primary data is valid. [2025-10-20T18:46:50.434413] [DEBUG] [LOCAL ] FIGS: Saving primary data. [2025-10-20T18:46:50.435770] [DEBUG] [LOCAL ] FIGS: Trimmed 70868 records to 48620 (keeping 200 days of data) [2025-10-20T18:46:50.652076] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-01. [2025-10-20T18:46:50.656057] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:46:50.719421] [DEBUG] [LOCAL ] FIGS: Primary data saved (48620 records). [2025-10-20T18:46:50.719745] [INFO ] [MAIN ] FIGS: Full history saved. [2025-10-20T18:46:50.720611] [INFO ] [MAIN ] Update Progress: 834/2367 symbols completed. FIGS took 33.5s. Estimated remaining: 3h 16m 16s. Rate: 7.8 symbols/min [2025-10-20T18:46:50.891977] [INFO ] [MAIN ] FIX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:50.923089] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:46:50.959419] [DEBUG] [LOCAL ] FIX: Loaded 29256 existing records from primary storage. [2025-10-20T18:46:50.959730] [INFO ] [MAIN ] FIX: Loaded 29256 existing records. [2025-10-20T18:46:51.132730] [WARN ] [MAIN ] FIX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:51.132978] [INFO ] [MAIN ] FIX: Wiping existing primary data before full refresh. [2025-10-20T18:46:51.133026] [INFO ] [LOCAL ] FIX: Starting deletion of primary data. [2025-10-20T18:46:51.133216] [DEBUG] [LOCAL ] FIX: Deleted primary data file. [2025-10-20T18:46:51.133253] [INFO ] [MAIN ] FIX: Starting full primary refresh. [2025-10-20T18:46:51.456536] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-08. [2025-10-20T18:46:51.486676] [INFO ] [REMOTE ] FIX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:52.148964] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T18:46:52.217772] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:46:52.251341] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-02. [2025-10-20T18:46:53.051369] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-09. [2025-10-20T18:46:53.748919] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-10-20T18:46:53.851346] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-01. [2025-10-20T18:46:54.036573] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T18:46:54.651366] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-03. [2025-10-20T18:46:55.281810] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:46:55.451376] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-08. [2025-10-20T18:46:56.083958] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T18:46:56.251364] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-04. [2025-10-20T18:46:56.995622] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:46:57.051358] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-02. [2025-10-20T18:46:57.752913] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T18:46:57.851373] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-05. [2025-10-20T18:46:58.636370] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T18:46:58.651329] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-03. [2025-10-20T18:46:59.181341] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:46:59.451370] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-06. [2025-10-20T18:47:00.196531] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T18:47:00.251332] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-10. [2025-10-20T18:47:01.028274] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T18:47:01.035017] [INFO ] [REMOTE ] FINS: Full fetch complete (11593 records). [2025-10-20T18:47:01.035205] [DEBUG] [VALIDATOR ] FINS: Validating primary data. [2025-10-20T18:47:01.056497] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-04. [2025-10-20T18:47:01.551602] [DEBUG] [VALIDATOR ] FINS: All expected trading days are present. [2025-10-20T18:47:01.551788] [DEBUG] [VALIDATOR ] FINS: Primary data validation passed. [2025-10-20T18:47:01.552478] [INFO ] [MAIN ] FINS: Fetched primary data is valid. [2025-10-20T18:47:01.552611] [DEBUG] [LOCAL ] FINS: Saving primary data. [2025-10-20T18:47:01.553910] [DEBUG] [LOCAL ] FINS: Trimmed 11593 records to 8283 (keeping 200 days of data) [2025-10-20T18:47:01.657397] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-06 (2.21 seconds). [2025-10-20T18:47:01.697221] [DEBUG] [LOCAL ] FINS: Primary data saved (8283 records). [2025-10-20T18:47:01.697413] [INFO ] [MAIN ] FINS: Full history saved. [2025-10-20T18:47:01.697527] [INFO ] [MAIN ] Update Progress: 835/2367 symbols completed. FINS took 33.9s. Estimated remaining: 3h 16m 14s. Rate: 7.8 symbols/min [2025-10-20T18:47:01.852310] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-07. [2025-10-20T18:47:01.943919] [INFO ] [MAIN ] FIXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:01.944177] [DEBUG] [LOCAL ] FIXT: No existing data file found in primary storage. [2025-10-20T18:47:01.944226] [WARN ] [MAIN ] FIXT: No existing data found, performing full refresh instead. [2025-10-20T18:47:01.944431] [INFO ] [MAIN ] FIXT: Wiping existing primary data before full refresh. [2025-10-20T18:47:01.944487] [INFO ] [LOCAL ] FIXT: Starting deletion of primary data. [2025-10-20T18:47:01.944532] [DEBUG] [LOCAL ] FIXT: No primary data file found to delete. [2025-10-20T18:47:01.944564] [INFO ] [MAIN ] FIXT: Starting full primary refresh. [2025-10-20T18:47:02.383149] [INFO ] [REMOTE ] FIXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:02.651362] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-01. [2025-10-20T18:47:02.715941] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T18:47:02.847408] [DEBUG] [REMOTE ] FIXT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:47:02.847801] [WARN ] [REMOTE ] FIXT: No data returned for month 2025-01 [2025-10-20T18:47:02.847903] [FATAL] [REMOTE ] FIXT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:47:02.847954] [FATAL] [MAIN ] FIXT: Failed to fetch primary data. [2025-10-20T18:47:02.848014] [WARN ] [MAIN ] FIXT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:47:02.848085] [INFO ] [LOCAL ] FIXT: Starting deletion of primary data. [2025-10-20T18:47:02.848146] [DEBUG] [LOCAL ] FIXT: No primary data file found to delete. [2025-10-20T18:47:02.848181] [INFO ] [LOCAL ] FIXT: Starting deletion of derivative data. [2025-10-20T18:47:02.848219] [DEBUG] [LOCAL ] FIXT: No derivative data file found to delete. [2025-10-20T18:47:02.848264] [INFO ] [MAIN ] Update Progress: 836/2367 symbols completed. FIXT took 1.2s. Estimated remaining: 3h 15m 54s. Rate: 7.8 symbols/min [2025-10-20T18:47:03.015006] [INFO ] [MAIN ] FLC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:03.022939] [DEBUG] [LOCAL ] FLC: Loaded 5454 existing records from primary storage. [2025-10-20T18:47:03.023148] [INFO ] [MAIN ] FLC: Loaded 5454 existing records. [2025-10-20T18:47:03.210513] [WARN ] [MAIN ] FLC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:03.210750] [INFO ] [MAIN ] FLC: Wiping existing primary data before full refresh. [2025-10-20T18:47:03.210802] [INFO ] [LOCAL ] FLC: Starting deletion of primary data. [2025-10-20T18:47:03.211211] [DEBUG] [LOCAL ] FLC: Deleted primary data file. [2025-10-20T18:47:03.211267] [INFO ] [MAIN ] FLC: Starting full primary refresh. [2025-10-20T18:47:03.461096] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-09. [2025-10-20T18:47:03.578137] [INFO ] [REMOTE ] FLC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:03.734543] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T18:47:04.251443] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-05. [2025-10-20T18:47:04.974548] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:47:05.051337] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-10. [2025-10-20T18:47:05.564039] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T18:47:05.851433] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-06. [2025-10-20T18:47:06.651406] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-08. [2025-10-20T18:47:06.761715] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T18:47:06.768773] [INFO ] [REMOTE ] FINV: Full fetch complete (64815 records). [2025-10-20T18:47:06.768885] [DEBUG] [VALIDATOR ] FINV: Validating primary data. [2025-10-20T18:47:07.165287] [DEBUG] [VALIDATOR ] FINV: All expected trading days are present. [2025-10-20T18:47:07.165455] [DEBUG] [VALIDATOR ] FINV: Primary data validation passed. [2025-10-20T18:47:07.166343] [INFO ] [MAIN ] FINV: Fetched primary data is valid. [2025-10-20T18:47:07.166424] [DEBUG] [LOCAL ] FINV: Saving primary data. [2025-10-20T18:47:07.182797] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T18:47:07.188988] [DEBUG] [LOCAL ] FINV: Trimmed 64815 records to 45271 (keeping 200 days of data) [2025-10-20T18:47:07.435754] [DEBUG] [LOCAL ] FINV: Primary data saved (45271 records). [2025-10-20T18:47:07.435879] [INFO ] [MAIN ] FINV: Full history saved. [2025-10-20T18:47:07.436422] [INFO ] [MAIN ] Update Progress: 837/2367 symbols completed. FINV took 37.5s. Estimated remaining: 3h 15m 41s. Rate: 7.8 symbols/min [2025-10-20T18:47:07.451376] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-07. [2025-10-20T18:47:08.251439] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-01. [2025-10-20T18:47:08.351030] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T18:47:08.811942] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T18:47:08.870062] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T18:47:09.051427] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-09. [2025-10-20T18:47:09.851391] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-02. [2025-10-20T18:47:10.378912] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:47:10.663424] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-03. [2025-10-20T18:47:10.693987] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T18:47:11.325373] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T18:47:11.451415] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-10. [2025-10-20T18:47:12.251458] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-04. [2025-10-20T18:47:12.304088] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T18:47:12.312348] [INFO ] [REMOTE ] FIS: Full fetch complete (77386 records). [2025-10-20T18:47:12.312517] [DEBUG] [VALIDATOR ] FIS: Validating primary data. [2025-10-20T18:47:12.716073] [DEBUG] [VALIDATOR ] FIS: All expected trading days are present. [2025-10-20T18:47:12.716160] [DEBUG] [VALIDATOR ] FIS: Primary data validation passed. [2025-10-20T18:47:12.717055] [INFO ] [MAIN ] FIS: Fetched primary data is valid. [2025-10-20T18:47:12.717117] [DEBUG] [LOCAL ] FIS: Saving primary data. [2025-10-20T18:47:12.718550] [DEBUG] [LOCAL ] FIS: Trimmed 77386 records to 53327 (keeping 200 days of data) [2025-10-20T18:47:12.967208] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T18:47:13.054423] [DEBUG] [LOCAL ] FIS: Primary data saved (53327 records). [2025-10-20T18:47:13.051811] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-05. [2025-10-20T18:47:13.054571] [INFO ] [MAIN ] FIS: Full history saved. [2025-10-20T18:47:13.054708] [INFO ] [MAIN ] Update Progress: 838/2367 symbols completed. FIS took 24.4s. Estimated remaining: 3h 15m 30s. Rate: 7.8 symbols/min [2025-10-20T18:47:13.664757] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T18:47:13.851408] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-06. [2025-10-20T18:47:14.415680] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T18:47:14.651382] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-07. [2025-10-20T18:47:15.239243] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T18:47:15.451401] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-08. [2025-10-20T18:47:16.251470] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-08. [2025-10-20T18:47:16.735950] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T18:47:16.987639] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T18:47:17.051362] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-09. [2025-10-20T18:47:17.851459] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-09. [2025-10-20T18:47:18.377417] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T18:47:18.466147] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T18:47:18.651430] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-10. [2025-10-20T18:47:19.229998] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T18:47:19.231444] [INFO ] [REMOTE ] FLC: Full fetch complete (7769 records). [2025-10-20T18:47:19.231534] [DEBUG] [VALIDATOR ] FLC: Validating primary data. [2025-10-20T18:47:19.458378] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-10. [2025-10-20T18:47:19.694455] [DEBUG] [VALIDATOR ] FLC: All expected trading days are present. [2025-10-20T18:47:19.694565] [DEBUG] [VALIDATOR ] FLC: Primary data validation passed. [2025-10-20T18:47:19.695044] [INFO ] [MAIN ] FLC: Fetched primary data is valid. [2025-10-20T18:47:19.695103] [DEBUG] [LOCAL ] FLC: Saving primary data. [2025-10-20T18:47:19.695621] [DEBUG] [LOCAL ] FLC: Trimmed 7769 records to 5417 (keeping 200 days of data) [2025-10-20T18:47:19.738686] [DEBUG] [LOCAL ] FLC: Primary data saved (5417 records). [2025-10-20T18:47:19.738830] [INFO ] [MAIN ] FLC: Full history saved. [2025-10-20T18:47:19.738910] [INFO ] [MAIN ] Update Progress: 839/2367 symbols completed. FLC took 16.9s. Estimated remaining: 3h 15m 20s. Rate: 7.8 symbols/min [2025-10-20T18:47:20.718745] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:47:20.725264] [INFO ] [REMOTE ] FIX: Full fetch complete (45598 records). [2025-10-20T18:47:20.725459] [DEBUG] [VALIDATOR ] FIX: Validating primary data. [2025-10-20T18:47:21.128590] [DEBUG] [VALIDATOR ] FIX: All expected trading days are present. [2025-10-20T18:47:21.128690] [DEBUG] [VALIDATOR ] FIX: Primary data validation passed. [2025-10-20T18:47:21.129409] [INFO ] [MAIN ] FIX: Fetched primary data is valid. [2025-10-20T18:47:21.129471] [DEBUG] [LOCAL ] FIX: Saving primary data. [2025-10-20T18:47:21.130414] [DEBUG] [LOCAL ] FIX: Trimmed 45598 records to 28617 (keeping 200 days of data) [2025-10-20T18:47:21.305693] [DEBUG] [LOCAL ] FIX: Primary data saved (28617 records). [2025-10-20T18:47:21.305845] [INFO ] [MAIN ] FIX: Full history saved. [2025-10-20T18:47:21.305924] [INFO ] [MAIN ] Update Progress: 840/2367 symbols completed. FIX took 30.6s. Estimated remaining: 3h 15m 1s. Rate: 7.8 symbols/min [2025-10-20T18:47:21.957187] [INFO ] [MAIN ] FLOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:21.959078] [INFO ] [MAIN ] FLG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:22.057965] [DEBUG] [LOCAL ] FLOC: No existing data file found in primary storage. [2025-10-20T18:47:22.083295] [WARN ] [MAIN ] FLOC: No existing data found, performing full refresh instead. [2025-10-20T18:47:22.057824] [INFO ] [MAIN ] FLO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:22.087160] [INFO ] [MAIN ] FLNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:22.088008] [INFO ] [MAIN ] FLOC: Wiping existing primary data before full refresh. [2025-10-20T18:47:22.088750] [INFO ] [LOCAL ] FLOC: Starting deletion of primary data. [2025-10-20T18:47:22.091793] [DEBUG] [LOCAL ] FLOC: No primary data file found to delete. [2025-10-20T18:47:22.091855] [INFO ] [MAIN ] FLOC: Starting full primary refresh. [2025-10-20T18:47:22.395045] [DEBUG] [LOCAL ] FLG: Loaded 54107 existing records from primary storage. [2025-10-20T18:47:22.403288] [DEBUG] [LOCAL ] FLNG: Loaded 34189 existing records from primary storage. [2025-10-20T18:47:22.417585] [INFO ] [MAIN ] FLG: Loaded 54107 existing records. [2025-10-20T18:47:22.439236] [INFO ] [MAIN ] FLNG: Loaded 34189 existing records. [2025-10-20T18:47:22.601902] [DEBUG] [LOCAL ] FLO: Loaded 52831 existing records from primary storage. [2025-10-20T18:47:22.685968] [INFO ] [MAIN ] FLO: Loaded 52831 existing records. [2025-10-20T18:47:22.782323] [INFO ] [REMOTE ] FLOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:22.862617] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-01. [2025-10-20T18:47:22.945181] [WARN ] [MAIN ] FLNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:22.969373] [INFO ] [MAIN ] FLNG: Wiping existing primary data before full refresh. [2025-10-20T18:47:22.997889] [INFO ] [LOCAL ] FLNG: Starting deletion of primary data. [2025-10-20T18:47:23.003330] [DEBUG] [LOCAL ] FLNG: Deleted primary data file. [2025-10-20T18:47:23.027176] [INFO ] [MAIN ] FLNG: Starting full primary refresh. [2025-10-20T18:47:23.047781] [WARN ] [MAIN ] FLG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:23.115084] [INFO ] [MAIN ] FLG: Wiping existing primary data before full refresh. [2025-10-20T18:47:23.130527] [INFO ] [LOCAL ] FLG: Starting deletion of primary data. [2025-10-20T18:47:23.130428] [WARN ] [MAIN ] FLO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:23.142865] [DEBUG] [LOCAL ] FLG: Deleted primary data file. [2025-10-20T18:47:23.154757] [INFO ] [MAIN ] FLG: Starting full primary refresh. [2025-10-20T18:47:23.183530] [INFO ] [MAIN ] FLO: Wiping existing primary data before full refresh. [2025-10-20T18:47:23.183694] [INFO ] [LOCAL ] FLO: Starting deletion of primary data. [2025-10-20T18:47:23.271486] [DEBUG] [LOCAL ] FLO: Deleted primary data file. [2025-10-20T18:47:23.297921] [INFO ] [MAIN ] FLO: Starting full primary refresh. [2025-10-20T18:47:24.227123] [INFO ] [REMOTE ] FLNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:24.268042] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-01. [2025-10-20T18:47:24.298026] [INFO ] [REMOTE ] FLG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:24.336013] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-01. [2025-10-20T18:47:24.344609] [INFO ] [REMOTE ] FLO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:24.336902] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T18:47:24.345618] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-01. [2025-10-20T18:47:24.351286] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-02. [2025-10-20T18:47:25.647119] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T18:47:25.653439] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-03. [2025-10-20T18:47:25.713741] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:47:25.721197] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-02. [2025-10-20T18:47:25.830064] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:47:25.851346] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-02. [2025-10-20T18:47:26.022956] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:47:26.651490] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-02. [2025-10-20T18:47:26.770771] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:47:27.253419] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:47:27.451391] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-03. [2025-10-20T18:47:27.553008] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T18:47:27.998219] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T18:47:28.251395] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-03. [2025-10-20T18:47:28.901279] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T18:47:29.051395] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-03. [2025-10-20T18:47:29.439086] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T18:47:29.851376] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-04. [2025-10-20T18:47:30.651350] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-04. [2025-10-20T18:47:30.830131] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T18:47:31.118264] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T18:47:31.451353] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-04. [2025-10-20T18:47:32.025174] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T18:47:32.251386] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-05. [2025-10-20T18:47:33.001405] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T18:47:33.051405] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-04. [2025-10-20T18:47:33.526065] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:47:33.851373] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-05. [2025-10-20T18:47:34.651371] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-05. [2025-10-20T18:47:34.690728] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T18:47:35.173006] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T18:47:35.451399] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-06. [2025-10-20T18:47:36.269816] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-05. [2025-10-20T18:47:36.272730] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:47:36.676890] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T18:47:37.051360] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-06. [2025-10-20T18:47:37.565075] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T18:47:37.851378] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-06. [2025-10-20T18:47:38.651349] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-06. [2025-10-20T18:47:38.866315] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T18:47:39.038348] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T18:47:39.451371] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-07. [2025-10-20T18:47:40.006289] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T18:47:40.251356] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-07. [2025-10-20T18:47:40.577753] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T18:47:41.051368] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-07. [2025-10-20T18:47:41.851454] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-08. [2025-10-20T18:47:42.117996] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:47:42.167841] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:47:42.656456] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-08. [2025-10-20T18:47:42.668236] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T18:47:43.451383] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-08. [2025-10-20T18:47:44.251350] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-07. [2025-10-20T18:47:44.459205] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T18:47:44.976579] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T18:47:45.051356] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-09. [2025-10-20T18:47:45.851366] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-09. [2025-10-20T18:47:46.032000] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T18:47:46.160676] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T18:47:46.605745] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T18:47:46.651333] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-10. [2025-10-20T18:47:47.451375] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-10. [2025-10-20T18:47:47.918480] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T18:47:47.922930] [INFO ] [REMOTE ] FLOC: Full fetch complete (40525 records). [2025-10-20T18:47:47.923058] [DEBUG] [VALIDATOR ] FLOC: Validating primary data. [2025-10-20T18:47:48.126701] [WARN ] [VALIDATOR ] FLOC: Validation failed - insufficient historical data. Have 191 unique days, need 200 days [2025-10-20T18:47:48.127156] [FATAL] [MAIN ] FLOC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:47:48.127225] [WARN ] [MAIN ] FLOC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:47:48.127294] [INFO ] [LOCAL ] FLOC: Starting deletion of primary data. [2025-10-20T18:47:48.127356] [DEBUG] [LOCAL ] FLOC: No primary data file found to delete. [2025-10-20T18:47:48.127393] [INFO ] [LOCAL ] FLOC: Starting deletion of derivative data. [2025-10-20T18:47:48.127430] [DEBUG] [LOCAL ] FLOC: No derivative data file found to delete. [2025-10-20T18:47:48.127486] [INFO ] [MAIN ] Update Progress: 841/2367 symbols completed. FLOC took 26.8s. Estimated remaining: 3h 15m 28s. Rate: 7.8 symbols/min [2025-10-20T18:47:48.266601] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-08. [2025-10-20T18:47:48.299681] [INFO ] [MAIN ] FLR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:48.348797] [DEBUG] [LOCAL ] FLR: Loaded 54019 existing records from primary storage. [2025-10-20T18:47:48.349126] [INFO ] [MAIN ] FLR: Loaded 54019 existing records. [2025-10-20T18:47:48.524790] [WARN ] [MAIN ] FLR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:48.525043] [INFO ] [MAIN ] FLR: Wiping existing primary data before full refresh. [2025-10-20T18:47:48.525114] [INFO ] [LOCAL ] FLR: Starting deletion of primary data. [2025-10-20T18:47:48.525828] [DEBUG] [LOCAL ] FLR: Deleted primary data file. [2025-10-20T18:47:48.525885] [INFO ] [MAIN ] FLR: Starting full primary refresh. [2025-10-20T18:47:48.850920] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T18:47:48.903706] [INFO ] [REMOTE ] FLNG: Full fetch complete (50059 records). [2025-10-20T18:47:48.910793] [DEBUG] [VALIDATOR ] FLNG: Validating primary data. [2025-10-20T18:47:48.914522] [INFO ] [REMOTE ] FLR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:49.056475] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-01. [2025-10-20T18:47:49.350940] [DEBUG] [VALIDATOR ] FLNG: All expected trading days are present. [2025-10-20T18:47:49.351018] [DEBUG] [VALIDATOR ] FLNG: Primary data validation passed. [2025-10-20T18:47:49.351813] [INFO ] [MAIN ] FLNG: Fetched primary data is valid. [2025-10-20T18:47:49.351877] [DEBUG] [LOCAL ] FLNG: Saving primary data. [2025-10-20T18:47:49.353022] [DEBUG] [LOCAL ] FLNG: Trimmed 50059 records to 33591 (keeping 200 days of data) [2025-10-20T18:47:49.565148] [DEBUG] [LOCAL ] FLNG: Primary data saved (33591 records). [2025-10-20T18:47:49.565284] [INFO ] [MAIN ] FLNG: Full history saved. [2025-10-20T18:47:49.565363] [INFO ] [MAIN ] Update Progress: 842/2367 symbols completed. FLNG took 28.3s. Estimated remaining: 3h 15m 9s. Rate: 7.8 symbols/min [2025-10-20T18:47:49.746859] [INFO ] [MAIN ] FLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:49.796868] [DEBUG] [LOCAL ] FLS: Loaded 51134 existing records from primary storage. [2025-10-20T18:47:49.797262] [INFO ] [MAIN ] FLS: Loaded 51134 existing records. [2025-10-20T18:47:49.857929] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-09. [2025-10-20T18:47:50.009866] [WARN ] [MAIN ] FLS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:50.010203] [INFO ] [MAIN ] FLS: Wiping existing primary data before full refresh. [2025-10-20T18:47:50.010300] [INFO ] [LOCAL ] FLS: Starting deletion of primary data. [2025-10-20T18:47:50.010754] [DEBUG] [LOCAL ] FLS: Deleted primary data file. [2025-10-20T18:47:50.010845] [INFO ] [MAIN ] FLS: Starting full primary refresh. [2025-10-20T18:47:50.281223] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T18:47:50.459675] [INFO ] [REMOTE ] FLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:50.651377] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-09. [2025-10-20T18:47:50.766408] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T18:47:51.451391] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-02. [2025-10-20T18:47:51.550510] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:47:52.251365] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-01. [2025-10-20T18:47:52.301186] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T18:47:52.772853] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:47:53.051362] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-10. [2025-10-20T18:47:53.539502] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T18:47:53.851432] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-10. [2025-10-20T18:47:54.651409] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-03. [2025-10-20T18:47:54.783037] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:47:54.791188] [INFO ] [REMOTE ] FLG: Full fetch complete (77425 records). [2025-10-20T18:47:54.791333] [DEBUG] [VALIDATOR ] FLG: Validating primary data. [2025-10-20T18:47:55.186825] [DEBUG] [VALIDATOR ] FLG: All expected trading days are present. [2025-10-20T18:47:55.187026] [DEBUG] [VALIDATOR ] FLG: Primary data validation passed. [2025-10-20T18:47:55.187993] [INFO ] [MAIN ] FLG: Fetched primary data is valid. [2025-10-20T18:47:55.188065] [DEBUG] [LOCAL ] FLG: Saving primary data. [2025-10-20T18:47:55.189518] [DEBUG] [LOCAL ] FLG: Trimmed 77425 records to 53322 (keeping 200 days of data) [2025-10-20T18:47:55.465711] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-02. [2025-10-20T18:47:55.517428] [DEBUG] [LOCAL ] FLG: Primary data saved (53322 records). [2025-10-20T18:47:55.517791] [INFO ] [MAIN ] FLG: Full history saved. [2025-10-20T18:47:55.518457] [INFO ] [MAIN ] Update Progress: 843/2367 symbols completed. FLG took 34.2s. Estimated remaining: 3h 14m 59s. Rate: 7.8 symbols/min [2025-10-20T18:47:55.713365] [INFO ] [MAIN ] FLUT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:55.718403] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:47:55.727776] [INFO ] [REMOTE ] FLO: Full fetch complete (75303 records). [2025-10-20T18:47:55.728095] [DEBUG] [VALIDATOR ] FLO: Validating primary data. [2025-10-20T18:47:56.063780] [DEBUG] [LOCAL ] FLUT: Loaded 51478 existing records from primary storage. [2025-10-20T18:47:56.085396] [INFO ] [MAIN ] FLUT: Loaded 51478 existing records. [2025-10-20T18:47:56.391534] [DEBUG] [VALIDATOR ] FLO: All expected trading days are present. [2025-10-20T18:47:56.401539] [DEBUG] [VALIDATOR ] FLO: Primary data validation passed. [2025-10-20T18:47:56.406831] [INFO ] [MAIN ] FLO: Fetched primary data is valid. [2025-10-20T18:47:56.406983] [DEBUG] [LOCAL ] FLO: Saving primary data. [2025-10-20T18:47:56.440340] [WARN ] [MAIN ] FLUT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:56.441237] [INFO ] [MAIN ] FLUT: Wiping existing primary data before full refresh. [2025-10-20T18:47:56.441373] [INFO ] [LOCAL ] FLUT: Starting deletion of primary data. [2025-10-20T18:47:56.442129] [DEBUG] [LOCAL ] FLO: Trimmed 75303 records to 52083 (keeping 200 days of data) [2025-10-20T18:47:56.442219] [DEBUG] [LOCAL ] FLUT: Deleted primary data file. [2025-10-20T18:47:56.442444] [INFO ] [MAIN ] FLUT: Starting full primary refresh. [2025-10-20T18:47:56.722743] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T18:47:56.950501] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-04. [2025-10-20T18:47:57.140926] [INFO ] [REMOTE ] FLUT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:57.149338] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-01. [2025-10-20T18:47:57.256918] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T18:47:57.263379] [DEBUG] [LOCAL ] FLO: Primary data saved (52083 records). [2025-10-20T18:47:57.272734] [INFO ] [MAIN ] FLO: Full history saved. [2025-10-20T18:47:57.273064] [INFO ] [MAIN ] Update Progress: 844/2367 symbols completed. FLO took 35.9s. Estimated remaining: 3h 14m 40s. Rate: 7.8 symbols/min [2025-10-20T18:47:57.461201] [INFO ] [MAIN ] FLXR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:57.485431] [DEBUG] [LOCAL ] FLXR: Loaded 21398 existing records from primary storage. [2025-10-20T18:47:57.485795] [INFO ] [MAIN ] FLXR: Loaded 21398 existing records. [2025-10-20T18:47:57.674316] [WARN ] [MAIN ] FLXR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:57.674631] [INFO ] [MAIN ] FLXR: Wiping existing primary data before full refresh. [2025-10-20T18:47:57.674717] [INFO ] [LOCAL ] FLXR: Starting deletion of primary data. [2025-10-20T18:47:57.674959] [DEBUG] [LOCAL ] FLXR: Deleted primary data file. [2025-10-20T18:47:57.674997] [INFO ] [MAIN ] FLXR: Starting full primary refresh. [2025-10-20T18:47:57.855309] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-03. [2025-10-20T18:47:58.046593] [INFO ] [REMOTE ] FLXR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:58.651374] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-01. [2025-10-20T18:47:58.779628] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:47:58.840874] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:47:59.236204] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T18:47:59.451361] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-05. [2025-10-20T18:47:59.739411] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T18:48:00.251470] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-04. [2025-10-20T18:48:01.051360] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-02. [2025-10-20T18:48:01.207369] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T18:48:01.738408] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:48:01.851393] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-06. [2025-10-20T18:48:02.651400] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-05. [2025-10-20T18:48:02.852253] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T18:48:02.984489] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T18:48:03.451434] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-02. [2025-10-20T18:48:04.215617] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T18:48:04.246851] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T18:48:04.256406] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-03. [2025-10-20T18:48:05.051396] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-07. [2025-10-20T18:48:05.193075] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T18:48:05.851414] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-03. [2025-10-20T18:48:06.651415] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-04. [2025-10-20T18:48:06.976013] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T18:48:07.451406] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-08. [2025-10-20T18:48:07.581774] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T18:48:07.738069] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:48:08.251403] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-04. [2025-10-20T18:48:09.051391] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-06. [2025-10-20T18:48:09.240272] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T18:48:09.851358] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-09. [2025-10-20T18:48:10.123867] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T18:48:10.651428] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-05. [2025-10-20T18:48:10.816597] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:48:11.240541] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T18:48:11.451411] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-10. [2025-10-20T18:48:11.696378] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:48:12.251448] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-05. [2025-10-20T18:48:13.051393] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-07. [2025-10-20T18:48:13.292629] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:48:13.300902] [INFO ] [REMOTE ] FLR: Full fetch complete (76837 records). [2025-10-20T18:48:13.301065] [DEBUG] [VALIDATOR ] FLR: Validating primary data. [2025-10-20T18:48:13.695989] [DEBUG] [VALIDATOR ] FLR: All expected trading days are present. [2025-10-20T18:48:13.696127] [DEBUG] [VALIDATOR ] FLR: Primary data validation passed. [2025-10-20T18:48:13.697079] [INFO ] [MAIN ] FLR: Fetched primary data is valid. [2025-10-20T18:48:13.697219] [DEBUG] [LOCAL ] FLR: Saving primary data. [2025-10-20T18:48:13.698762] [DEBUG] [LOCAL ] FLR: Trimmed 76837 records to 53258 (keeping 200 days of data) [2025-10-20T18:48:13.867320] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-06. [2025-10-20T18:48:14.027730] [DEBUG] [LOCAL ] FLR: Primary data saved (53258 records). [2025-10-20T18:48:14.027907] [INFO ] [MAIN ] FLR: Full history saved. [2025-10-20T18:48:14.028543] [INFO ] [MAIN ] Update Progress: 845/2367 symbols completed. FLR took 25.9s. Estimated remaining: 3h 14m 49s. Rate: 7.8 symbols/min [2025-10-20T18:48:14.208089] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T18:48:14.231542] [INFO ] [MAIN ] FMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:14.293272] [DEBUG] [LOCAL ] FMC: Loaded 52624 existing records from primary storage. [2025-10-20T18:48:14.293589] [INFO ] [MAIN ] FMC: Loaded 52624 existing records. [2025-10-20T18:48:14.473397] [WARN ] [MAIN ] FMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:14.473600] [INFO ] [MAIN ] FMC: Wiping existing primary data before full refresh. [2025-10-20T18:48:14.473671] [INFO ] [LOCAL ] FMC: Starting deletion of primary data. [2025-10-20T18:48:14.474247] [DEBUG] [LOCAL ] FMC: Deleted primary data file. [2025-10-20T18:48:14.474286] [INFO ] [MAIN ] FMC: Starting full primary refresh. [2025-10-20T18:48:14.654832] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-06. [2025-10-20T18:48:14.854424] [INFO ] [REMOTE ] FMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:14.887793] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T18:48:14.922553] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:48:15.451368] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-07. [2025-10-20T18:48:16.251437] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-01. [2025-10-20T18:48:16.436884] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T18:48:16.582961] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T18:48:17.051576] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-08. [2025-10-20T18:48:17.851391] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-07. [2025-10-20T18:48:17.991074] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T18:48:18.133376] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:48:18.651379] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-09. [2025-10-20T18:48:19.451343] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-02. [2025-10-20T18:48:19.549342] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T18:48:19.832485] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:48:20.251372] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-10. [2025-10-20T18:48:21.051374] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-08. [2025-10-20T18:48:21.237484] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T18:48:21.624512] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:48:21.628963] [INFO ] [REMOTE ] FLXR: Full fetch complete (27223 records). [2025-10-20T18:48:21.629070] [DEBUG] [VALIDATOR ] FLXR: Validating primary data. [2025-10-20T18:48:21.857706] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-03. [2025-10-20T18:48:22.066721] [DEBUG] [VALIDATOR ] FLXR: All expected trading days are present. [2025-10-20T18:48:22.066808] [DEBUG] [VALIDATOR ] FLXR: Primary data validation passed. [2025-10-20T18:48:22.067333] [INFO ] [MAIN ] FLXR: Fetched primary data is valid. [2025-10-20T18:48:22.067371] [DEBUG] [LOCAL ] FLXR: Saving primary data. [2025-10-20T18:48:22.068075] [DEBUG] [LOCAL ] FLXR: Trimmed 27223 records to 21242 (keeping 200 days of data) [2025-10-20T18:48:22.199949] [DEBUG] [LOCAL ] FLXR: Primary data saved (21242 records). [2025-10-20T18:48:22.200079] [INFO ] [MAIN ] FLXR: Full history saved. [2025-10-20T18:48:22.200154] [INFO ] [MAIN ] Update Progress: 846/2367 symbols completed. FLXR took 24.9s. Estimated remaining: 3h 14m 42s. Rate: 7.8 symbols/min [2025-10-20T18:48:22.373486] [INFO ] [MAIN ] FMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:22.381516] [DEBUG] [LOCAL ] FMN: Loaded 5571 existing records from primary storage. [2025-10-20T18:48:22.381867] [INFO ] [MAIN ] FMN: Loaded 5571 existing records. [2025-10-20T18:48:22.563719] [WARN ] [MAIN ] FMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:22.563967] [INFO ] [MAIN ] FMN: Wiping existing primary data before full refresh. [2025-10-20T18:48:22.564043] [INFO ] [LOCAL ] FMN: Starting deletion of primary data. [2025-10-20T18:48:22.564399] [DEBUG] [LOCAL ] FMN: Deleted primary data file. [2025-10-20T18:48:22.564522] [INFO ] [MAIN ] FMN: Starting full primary refresh. [2025-10-20T18:48:22.665113] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-08. [2025-10-20T18:48:22.946881] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T18:48:22.979612] [INFO ] [REMOTE ] FMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:23.451382] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-01. [2025-10-20T18:48:23.574222] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T18:48:24.251440] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-04. [2025-10-20T18:48:24.303134] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T18:48:24.418118] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T18:48:25.051384] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-09. [2025-10-20T18:48:25.856568] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-02. [2025-10-20T18:48:25.875807] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:48:26.607947] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:48:26.651386] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-03. [2025-10-20T18:48:26.742159] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:48:27.408888] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T18:48:27.451374] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-10. [2025-10-20T18:48:28.251417] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-04. [2025-10-20T18:48:28.955156] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T18:48:29.051365] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-09. [2025-10-20T18:48:29.317509] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:48:29.325217] [INFO ] [REMOTE ] FLS: Full fetch complete (72489 records). [2025-10-20T18:48:29.325361] [DEBUG] [VALIDATOR ] FLS: Validating primary data. [2025-10-20T18:48:29.709723] [DEBUG] [VALIDATOR ] FLS: All expected trading days are present. [2025-10-20T18:48:29.709809] [DEBUG] [VALIDATOR ] FLS: Primary data validation passed. [2025-10-20T18:48:29.710701] [INFO ] [MAIN ] FLS: Fetched primary data is valid. [2025-10-20T18:48:29.710764] [DEBUG] [LOCAL ] FLS: Saving primary data. [2025-10-20T18:48:29.712088] [DEBUG] [LOCAL ] FLS: Trimmed 72489 records to 50452 (keeping 200 days of data) [2025-10-20T18:48:29.882885] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-05. [2025-10-20T18:48:30.025014] [DEBUG] [LOCAL ] FLS: Primary data saved (50452 records). [2025-10-20T18:48:30.025149] [INFO ] [MAIN ] FLS: Full history saved. [2025-10-20T18:48:30.025238] [INFO ] [MAIN ] Update Progress: 847/2367 symbols completed. FLS took 40.5s. Estimated remaining: 3h 14m 35s. Rate: 7.8 symbols/min [2025-10-20T18:48:30.208465] [INFO ] [MAIN ] FMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:30.241243] [DEBUG] [LOCAL ] FMS: Loaded 30733 existing records from primary storage. [2025-10-20T18:48:30.241505] [INFO ] [MAIN ] FMS: Loaded 30733 existing records. [2025-10-20T18:48:30.418747] [WARN ] [MAIN ] FMS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:30.418965] [INFO ] [MAIN ] FMS: Wiping existing primary data before full refresh. [2025-10-20T18:48:30.419123] [INFO ] [LOCAL ] FMS: Starting deletion of primary data. [2025-10-20T18:48:30.419449] [DEBUG] [LOCAL ] FMS: Deleted primary data file. [2025-10-20T18:48:30.419509] [INFO ] [MAIN ] FMS: Starting full primary refresh. [2025-10-20T18:48:30.508138] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T18:48:30.656696] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-06. [2025-10-20T18:48:30.810816] [INFO ] [REMOTE ] FMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:30.812708] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T18:48:31.367903] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T18:48:31.451342] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-01. [2025-10-20T18:48:32.251349] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-07. [2025-10-20T18:48:32.648215] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T18:48:32.814266] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T18:48:33.051378] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-02. [2025-10-20T18:48:33.851350] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-05. [2025-10-20T18:48:34.194803] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:48:34.651361] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-03. [2025-10-20T18:48:35.451401] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-08. [2025-10-20T18:48:35.585425] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T18:48:35.806333] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T18:48:36.035683] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T18:48:36.251385] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-09. [2025-10-20T18:48:36.903837] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T18:48:37.051381] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-06. [2025-10-20T18:48:37.851403] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-10. [2025-10-20T18:48:38.564540] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T18:48:38.566240] [INFO ] [REMOTE ] FMN: Full fetch complete (10091 records). [2025-10-20T18:48:38.566308] [DEBUG] [VALIDATOR ] FMN: Validating primary data. [2025-10-20T18:48:38.654949] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-10. [2025-10-20T18:48:38.935162] [DEBUG] [VALIDATOR ] FMN: All expected trading days are present. [2025-10-20T18:48:38.945330] [DEBUG] [VALIDATOR ] FMN: Primary data validation passed. [2025-10-20T18:48:38.962886] [INFO ] [MAIN ] FMN: Fetched primary data is valid. [2025-10-20T18:48:38.962970] [DEBUG] [LOCAL ] FMN: Saving primary data. [2025-10-20T18:48:38.964997] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T18:48:38.965396] [DEBUG] [LOCAL ] FMN: Trimmed 10091 records to 5517 (keeping 200 days of data) [2025-10-20T18:48:39.008420] [DEBUG] [LOCAL ] FMN: Primary data saved (5517 records). [2025-10-20T18:48:39.008510] [INFO ] [MAIN ] FMN: Full history saved. [2025-10-20T18:48:39.008586] [INFO ] [MAIN ] Update Progress: 848/2367 symbols completed. FMN took 16.8s. Estimated remaining: 3h 14m 29s. Rate: 7.8 symbols/min [2025-10-20T18:48:39.196823] [INFO ] [MAIN ] FMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:39.234565] [DEBUG] [LOCAL ] FMX: Loaded 41172 existing records from primary storage. [2025-10-20T18:48:39.234868] [INFO ] [MAIN ] FMX: Loaded 41172 existing records. [2025-10-20T18:48:39.408700] [WARN ] [MAIN ] FMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:39.408902] [INFO ] [MAIN ] FMX: Wiping existing primary data before full refresh. [2025-10-20T18:48:39.408948] [INFO ] [LOCAL ] FMX: Starting deletion of primary data. [2025-10-20T18:48:39.409276] [DEBUG] [LOCAL ] FMX: Deleted primary data file. [2025-10-20T18:48:39.409332] [INFO ] [MAIN ] FMX: Starting full primary refresh. [2025-10-20T18:48:39.462497] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-07. [2025-10-20T18:48:39.759437] [INFO ] [REMOTE ] FMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:40.251385] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-01. [2025-10-20T18:48:40.620214] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T18:48:40.627990] [INFO ] [REMOTE ] FLUT: Full fetch complete (73367 records). [2025-10-20T18:48:40.628146] [DEBUG] [VALIDATOR ] FLUT: Validating primary data. [2025-10-20T18:48:41.033023] [DEBUG] [VALIDATOR ] FLUT: All expected trading days are present. [2025-10-20T18:48:41.033104] [DEBUG] [VALIDATOR ] FLUT: Primary data validation passed. [2025-10-20T18:48:41.034061] [INFO ] [MAIN ] FLUT: Fetched primary data is valid. [2025-10-20T18:48:41.034125] [DEBUG] [LOCAL ] FLUT: Saving primary data. [2025-10-20T18:48:41.035518] [DEBUG] [LOCAL ] FLUT: Trimmed 73367 records to 50697 (keeping 200 days of data) [2025-10-20T18:48:41.086773] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-04. [2025-10-20T18:48:41.367369] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T18:48:41.392201] [DEBUG] [LOCAL ] FLUT: Primary data saved (50697 records). [2025-10-20T18:48:41.392703] [INFO ] [MAIN ] FLUT: Full history saved. [2025-10-20T18:48:41.393340] [INFO ] [MAIN ] Update Progress: 849/2367 symbols completed. FLUT took 45.9s. Estimated remaining: 3h 14m 12s. Rate: 7.8 symbols/min [2025-10-20T18:48:41.563249] [INFO ] [MAIN ] FMY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:41.566940] [DEBUG] [LOCAL ] FMY: Loaded 1397 existing records from primary storage. [2025-10-20T18:48:41.567101] [INFO ] [MAIN ] FMY: Loaded 1397 existing records. [2025-10-20T18:48:41.750701] [WARN ] [MAIN ] FMY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:41.751085] [INFO ] [MAIN ] FMY: Wiping existing primary data before full refresh. [2025-10-20T18:48:41.751162] [INFO ] [LOCAL ] FMY: Starting deletion of primary data. [2025-10-20T18:48:41.751528] [DEBUG] [LOCAL ] FMY: Deleted primary data file. [2025-10-20T18:48:41.751590] [INFO ] [MAIN ] FMY: Starting full primary refresh. [2025-10-20T18:48:41.853918] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-08. [2025-10-20T18:48:41.861963] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T18:48:42.146616] [INFO ] [REMOTE ] FMY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:42.570204] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:48:42.651356] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-01. [2025-10-20T18:48:42.932534] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-01 (0.28 seconds). [2025-10-20T18:48:43.451364] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-05. [2025-10-20T18:48:43.617597] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:48:44.251400] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-02. [2025-10-20T18:48:44.799268] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T18:48:45.051360] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-06. [2025-10-20T18:48:45.584212] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:48:45.851397] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-03. [2025-10-20T18:48:46.336388] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:48:46.651370] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-02. [2025-10-20T18:48:46.933831] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-02 (0.28 seconds). [2025-10-20T18:48:47.334625] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T18:48:47.451342] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-04. [2025-10-20T18:48:48.251382] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-07. [2025-10-20T18:48:49.056579] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-03. [2025-10-20T18:48:49.068320] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:48:49.401806] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T18:48:49.569182] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T18:48:49.851400] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-04. [2025-10-20T18:48:50.387841] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T18:48:50.651384] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-05. [2025-10-20T18:48:50.954074] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-10-20T18:48:51.451376] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-05. [2025-10-20T18:48:52.251375] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-06. [2025-10-20T18:48:52.524716] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-10-20T18:48:52.938234] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T18:48:53.051344] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-09. [2025-10-20T18:48:53.851426] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-07. [2025-10-20T18:48:54.276617] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T18:48:54.654712] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-08. [2025-10-20T18:48:54.658084] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:48:55.017573] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T18:48:55.451394] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-06. [2025-10-20T18:48:56.251382] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-10. [2025-10-20T18:48:56.686872] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T18:48:57.051357] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-07. [2025-10-20T18:48:57.851345] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-08. [2025-10-20T18:48:58.252477] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T18:48:58.260326] [INFO ] [REMOTE ] FMC: Full fetch complete (75384 records). [2025-10-20T18:48:58.260463] [DEBUG] [VALIDATOR ] FMC: Validating primary data. [2025-10-20T18:48:58.652036] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-09. [2025-10-20T18:48:58.652325] [DEBUG] [VALIDATOR ] FMC: All expected trading days are present. [2025-10-20T18:48:58.653382] [DEBUG] [VALIDATOR ] FMC: Primary data validation passed. [2025-10-20T18:48:58.654492] [INFO ] [MAIN ] FMC: Fetched primary data is valid. [2025-10-20T18:48:58.654624] [DEBUG] [LOCAL ] FMC: Saving primary data. [2025-10-20T18:48:58.681840] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T18:48:58.691181] [DEBUG] [LOCAL ] FMC: Trimmed 75384 records to 51892 (keeping 200 days of data) [2025-10-20T18:48:59.020211] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-10-20T18:48:59.048557] [DEBUG] [LOCAL ] FMC: Primary data saved (51892 records). [2025-10-20T18:48:59.048739] [INFO ] [MAIN ] FMC: Full history saved. [2025-10-20T18:48:59.049686] [INFO ] [MAIN ] Update Progress: 850/2367 symbols completed. FMC took 45.0s. Estimated remaining: 3h 14m 22s. Rate: 7.8 symbols/min [2025-10-20T18:48:59.149392] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T18:48:59.219516] [INFO ] [MAIN ] FN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:59.258786] [DEBUG] [LOCAL ] FN: Loaded 38299 existing records from primary storage. [2025-10-20T18:48:59.259059] [INFO ] [MAIN ] FN: Loaded 38299 existing records. [2025-10-20T18:48:59.434663] [WARN ] [MAIN ] FN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:59.434937] [INFO ] [MAIN ] FN: Wiping existing primary data before full refresh. [2025-10-20T18:48:59.434984] [INFO ] [LOCAL ] FN: Starting deletion of primary data. [2025-10-20T18:48:59.435871] [DEBUG] [LOCAL ] FN: Deleted primary data file. [2025-10-20T18:48:59.435943] [INFO ] [MAIN ] FN: Starting full primary refresh. [2025-10-20T18:48:59.456562] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-09. [2025-10-20T18:48:59.843704] [INFO ] [REMOTE ] FN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:00.251393] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-01. [2025-10-20T18:49:01.051449] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-08. [2025-10-20T18:49:01.087625] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:49:01.851410] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-10. [2025-10-20T18:49:01.954535] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:49:02.651375] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-02. [2025-10-20T18:49:02.777137] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:49:03.451056] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T18:49:03.455463] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-10. [2025-10-20T18:49:03.457029] [INFO ] [REMOTE ] FMS: Full fetch complete (42998 records). [2025-10-20T18:49:03.457101] [DEBUG] [VALIDATOR ] FMS: Validating primary data. [2025-10-20T18:49:03.887161] [DEBUG] [VALIDATOR ] FMS: All expected trading days are present. [2025-10-20T18:49:03.887380] [DEBUG] [VALIDATOR ] FMS: Primary data validation passed. [2025-10-20T18:49:03.888343] [INFO ] [MAIN ] FMS: Fetched primary data is valid. [2025-10-20T18:49:03.888486] [DEBUG] [LOCAL ] FMS: Saving primary data. [2025-10-20T18:49:03.889926] [DEBUG] [LOCAL ] FMS: Trimmed 42998 records to 30419 (keeping 200 days of data) [2025-10-20T18:49:04.025916] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T18:49:04.031619] [INFO ] [REMOTE ] FMY: Full fetch complete (1955 records). [2025-10-20T18:49:04.045688] [DEBUG] [VALIDATOR ] FMY: Validating primary data. [2025-10-20T18:49:04.257577] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-09. [2025-10-20T18:49:04.316392] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:49:04.605435] [DEBUG] [LOCAL ] FMS: Primary data saved (30419 records). [2025-10-20T18:49:04.616041] [INFO ] [MAIN ] FMS: Full history saved. [2025-10-20T18:49:04.626565] [INFO ] [MAIN ] Update Progress: 851/2367 symbols completed. FMS took 34.6s. Estimated remaining: 3h 14m 11s. Rate: 7.8 symbols/min [2025-10-20T18:49:04.805328] [DEBUG] [VALIDATOR ] FMY: All expected trading days are present. [2025-10-20T18:49:04.816142] [DEBUG] [VALIDATOR ] FMY: Primary data validation passed. [2025-10-20T18:49:04.832003] [INFO ] [MAIN ] FMY: Fetched primary data is valid. [2025-10-20T18:49:04.868579] [DEBUG] [LOCAL ] FMY: Saving primary data. [2025-10-20T18:49:04.886308] [DEBUG] [LOCAL ] FMY: Trimmed 1955 records to 1381 (keeping 200 days of data) [2025-10-20T18:49:04.887917] [INFO ] [MAIN ] FNB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:04.899792] [DEBUG] [LOCAL ] FMY: Primary data saved (1381 records). [2025-10-20T18:49:04.899983] [INFO ] [MAIN ] FMY: Full history saved. [2025-10-20T18:49:04.900083] [INFO ] [MAIN ] Update Progress: 852/2367 symbols completed. FMY took 23.5s. Estimated remaining: 3h 13m 50s. Rate: 7.8 symbols/min [2025-10-20T18:49:05.079795] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-03. [2025-10-20T18:49:05.097917] [INFO ] [MAIN ] FND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:05.127865] [DEBUG] [LOCAL ] FNB: Loaded 53376 existing records from primary storage. [2025-10-20T18:49:05.128324] [INFO ] [MAIN ] FNB: Loaded 53376 existing records. [2025-10-20T18:49:05.343429] [WARN ] [MAIN ] FNB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:05.343768] [INFO ] [MAIN ] FNB: Wiping existing primary data before full refresh. [2025-10-20T18:49:05.343851] [INFO ] [LOCAL ] FNB: Starting deletion of primary data. [2025-10-20T18:49:05.344774] [DEBUG] [LOCAL ] FND: Loaded 52865 existing records from primary storage. [2025-10-20T18:49:05.345108] [INFO ] [MAIN ] FND: Loaded 52865 existing records. [2025-10-20T18:49:05.377802] [DEBUG] [LOCAL ] FNB: Deleted primary data file. [2025-10-20T18:49:05.377906] [INFO ] [MAIN ] FNB: Starting full primary refresh. [2025-10-20T18:49:05.698608] [WARN ] [MAIN ] FND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:05.713872] [INFO ] [MAIN ] FND: Wiping existing primary data before full refresh. [2025-10-20T18:49:05.721727] [INFO ] [LOCAL ] FND: Starting deletion of primary data. [2025-10-20T18:49:05.757048] [DEBUG] [LOCAL ] FND: Deleted primary data file. [2025-10-20T18:49:05.767851] [INFO ] [MAIN ] FND: Starting full primary refresh. [2025-10-20T18:49:06.048908] [INFO ] [REMOTE ] FNB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:06.060448] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-01. [2025-10-20T18:49:06.315076] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T18:49:06.370043] [INFO ] [REMOTE ] FND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:06.651374] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-01. [2025-10-20T18:49:06.842771] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T18:49:07.451388] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-04. [2025-10-20T18:49:07.486706] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:49:07.966837] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T18:49:08.251371] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-02. [2025-10-20T18:49:09.051351] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-02. [2025-10-20T18:49:09.115909] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T18:49:09.851382] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-05. [2025-10-20T18:49:09.913500] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:49:10.593195] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:49:10.651338] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-03. [2025-10-20T18:49:11.095592] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T18:49:11.451393] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-06. [2025-10-20T18:49:12.251365] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-10. [2025-10-20T18:49:12.432152] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T18:49:12.670082] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T18:49:13.051373] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-07. [2025-10-20T18:49:13.851411] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-04. [2025-10-20T18:49:13.913798] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T18:49:13.919479] [INFO ] [REMOTE ] FMX: Full fetch complete (57933 records). [2025-10-20T18:49:13.919614] [DEBUG] [VALIDATOR ] FMX: Validating primary data. [2025-10-20T18:49:14.315809] [DEBUG] [VALIDATOR ] FMX: All expected trading days are present. [2025-10-20T18:49:14.315921] [DEBUG] [VALIDATOR ] FMX: Primary data validation passed. [2025-10-20T18:49:14.316732] [INFO ] [MAIN ] FMX: Fetched primary data is valid. [2025-10-20T18:49:14.316864] [DEBUG] [LOCAL ] FMX: Saving primary data. [2025-10-20T18:49:14.318236] [DEBUG] [LOCAL ] FMX: Trimmed 57933 records to 40649 (keeping 200 days of data) [2025-10-20T18:49:14.481333] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:49:14.624128] [DEBUG] [LOCAL ] FMX: Primary data saved (40649 records). [2025-10-20T18:49:14.624249] [INFO ] [MAIN ] FMX: Full history saved. [2025-10-20T18:49:14.624324] [INFO ] [MAIN ] Update Progress: 853/2367 symbols completed. FMX took 35.6s. Estimated remaining: 3h 13m 46s. Rate: 7.8 symbols/min [2025-10-20T18:49:14.656473] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-08. [2025-10-20T18:49:14.794286] [INFO ] [MAIN ] FNF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:14.836121] [DEBUG] [LOCAL ] FNF: Loaded 49639 existing records from primary storage. [2025-10-20T18:49:14.836456] [INFO ] [MAIN ] FNF: Loaded 49639 existing records. [2025-10-20T18:49:15.014362] [WARN ] [MAIN ] FNF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:15.014564] [INFO ] [MAIN ] FNF: Wiping existing primary data before full refresh. [2025-10-20T18:49:15.014610] [INFO ] [LOCAL ] FNF: Starting deletion of primary data. [2025-10-20T18:49:15.015155] [DEBUG] [LOCAL ] FNF: Deleted primary data file. [2025-10-20T18:49:15.015209] [INFO ] [MAIN ] FNF: Starting full primary refresh. [2025-10-20T18:49:15.370317] [INFO ] [REMOTE ] FNF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:15.451364] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-03. [2025-10-20T18:49:15.682569] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:49:16.251432] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-05. [2025-10-20T18:49:16.405255] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T18:49:17.051413] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-09. [2025-10-20T18:49:17.089918] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T18:49:17.802370] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:49:17.851357] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-06. [2025-10-20T18:49:18.195344] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T18:49:18.651355] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-01. [2025-10-20T18:49:19.451444] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-04. [2025-10-20T18:49:19.564413] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T18:49:20.033210] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T18:49:20.251389] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-10. [2025-10-20T18:49:21.051356] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-02. [2025-10-20T18:49:21.155871] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:49:21.608330] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T18:49:21.613285] [INFO ] [REMOTE ] FN: Full fetch complete (56372 records). [2025-10-20T18:49:21.613403] [DEBUG] [VALIDATOR ] FN: Validating primary data. [2025-10-20T18:49:21.856637] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-05. [2025-10-20T18:49:22.012100] [DEBUG] [VALIDATOR ] FN: All expected trading days are present. [2025-10-20T18:49:22.012192] [DEBUG] [VALIDATOR ] FN: Primary data validation passed. [2025-10-20T18:49:22.012996] [INFO ] [MAIN ] FN: Fetched primary data is valid. [2025-10-20T18:49:22.013060] [DEBUG] [LOCAL ] FN: Saving primary data. [2025-10-20T18:49:22.014364] [DEBUG] [LOCAL ] FN: Trimmed 56372 records to 37620 (keeping 200 days of data) [2025-10-20T18:49:22.245935] [DEBUG] [LOCAL ] FN: Primary data saved (37620 records). [2025-10-20T18:49:22.246096] [INFO ] [MAIN ] FN: Full history saved. [2025-10-20T18:49:22.246279] [INFO ] [MAIN ] Update Progress: 854/2367 symbols completed. FN took 23.2s. Estimated remaining: 3h 13m 38s. Rate: 7.8 symbols/min [2025-10-20T18:49:22.394079] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T18:49:22.487475] [INFO ] [MAIN ] FNV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:22.535178] [DEBUG] [LOCAL ] FNV: Loaded 46890 existing records from primary storage. [2025-10-20T18:49:22.535554] [INFO ] [MAIN ] FNV: Loaded 46890 existing records. [2025-10-20T18:49:22.656552] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-03. [2025-10-20T18:49:22.715939] [WARN ] [MAIN ] FNV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:22.716126] [INFO ] [MAIN ] FNV: Wiping existing primary data before full refresh. [2025-10-20T18:49:22.716172] [INFO ] [LOCAL ] FNV: Starting deletion of primary data. [2025-10-20T18:49:22.716753] [DEBUG] [LOCAL ] FNV: Deleted primary data file. [2025-10-20T18:49:22.716808] [INFO ] [MAIN ] FNV: Starting full primary refresh. [2025-10-20T18:49:23.065828] [INFO ] [REMOTE ] FNV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:23.451417] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-01. [2025-10-20T18:49:23.750877] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:49:24.251492] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-06. [2025-10-20T18:49:24.485583] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T18:49:24.913044] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T18:49:25.051404] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-04. [2025-10-20T18:49:25.856539] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-02. [2025-10-20T18:49:25.878138] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T18:49:26.651365] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-07. [2025-10-20T18:49:26.810007] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:49:27.385921] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:49:27.451336] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-03. [2025-10-20T18:49:28.251412] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-05. [2025-10-20T18:49:28.439899] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T18:49:29.056501] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-07. [2025-10-20T18:49:29.072508] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:49:29.851432] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-04. [2025-10-20T18:49:29.950547] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:49:30.242966] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T18:49:30.651424] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-08. [2025-10-20T18:49:31.451363] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-08. [2025-10-20T18:49:31.647419] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:49:32.165084] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T18:49:32.251356] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-09. [2025-10-20T18:49:33.051420] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-06. [2025-10-20T18:49:33.230548] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:49:33.856518] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-09. [2025-10-20T18:49:33.874874] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:49:34.612123] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T18:49:34.651331] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-07. [2025-10-20T18:49:35.451407] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-05. [2025-10-20T18:49:35.580734] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:49:36.052024] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T18:49:36.251381] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-10. [2025-10-20T18:49:37.051360] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-08. [2025-10-20T18:49:37.128222] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:49:37.851400] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-06. [2025-10-20T18:49:38.101380] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:49:38.109048] [INFO ] [REMOTE ] FNB: Full fetch complete (76209 records). [2025-10-20T18:49:38.109197] [DEBUG] [VALIDATOR ] FNB: Validating primary data. [2025-10-20T18:49:38.521672] [DEBUG] [VALIDATOR ] FNB: All expected trading days are present. [2025-10-20T18:49:38.521816] [DEBUG] [VALIDATOR ] FNB: Primary data validation passed. [2025-10-20T18:49:38.522747] [INFO ] [MAIN ] FNB: Fetched primary data is valid. [2025-10-20T18:49:38.522826] [DEBUG] [LOCAL ] FNB: Saving primary data. [2025-10-20T18:49:38.524337] [DEBUG] [LOCAL ] FNB: Trimmed 76209 records to 52605 (keeping 200 days of data) [2025-10-20T18:49:38.701948] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-10. [2025-10-20T18:49:38.780113] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:49:38.897422] [DEBUG] [LOCAL ] FNB: Primary data saved (52605 records). [2025-10-20T18:49:38.897569] [INFO ] [MAIN ] FNB: Full history saved. [2025-10-20T18:49:38.897683] [INFO ] [MAIN ] Update Progress: 855/2367 symbols completed. FNB took 34.3s. Estimated remaining: 3h 13m 46s. Rate: 7.8 symbols/min [2025-10-20T18:49:39.084999] [INFO ] [MAIN ] FOA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:39.098590] [DEBUG] [LOCAL ] FOA: Loaded 13224 existing records from primary storage. [2025-10-20T18:49:39.098808] [INFO ] [MAIN ] FOA: Loaded 13224 existing records. [2025-10-20T18:49:39.288356] [WARN ] [MAIN ] FOA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:39.288783] [INFO ] [MAIN ] FOA: Wiping existing primary data before full refresh. [2025-10-20T18:49:39.288907] [INFO ] [LOCAL ] FOA: Starting deletion of primary data. [2025-10-20T18:49:39.289317] [DEBUG] [LOCAL ] FOA: Deleted primary data file. [2025-10-20T18:49:39.289389] [INFO ] [MAIN ] FOA: Starting full primary refresh. [2025-10-20T18:49:39.460094] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-09. [2025-10-20T18:49:39.461073] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:49:39.758668] [INFO ] [REMOTE ] FOA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:40.251416] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-07. [2025-10-20T18:49:40.526965] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:49:40.534483] [INFO ] [REMOTE ] FND: Full fetch complete (75042 records). [2025-10-20T18:49:40.534597] [DEBUG] [VALIDATOR ] FND: Validating primary data. [2025-10-20T18:49:40.927926] [DEBUG] [VALIDATOR ] FND: All expected trading days are present. [2025-10-20T18:49:40.928066] [DEBUG] [VALIDATOR ] FND: Primary data validation passed. [2025-10-20T18:49:40.929109] [INFO ] [MAIN ] FND: Fetched primary data is valid. [2025-10-20T18:49:40.929298] [DEBUG] [LOCAL ] FND: Saving primary data. [2025-10-20T18:49:40.930762] [DEBUG] [LOCAL ] FND: Trimmed 75042 records to 52091 (keeping 200 days of data) [2025-10-20T18:49:41.041377] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:49:41.052547] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-01. [2025-10-20T18:49:41.268990] [DEBUG] [LOCAL ] FND: Primary data saved (52091 records). [2025-10-20T18:49:41.269166] [INFO ] [MAIN ] FND: Full history saved. [2025-10-20T18:49:41.269682] [INFO ] [MAIN ] Update Progress: 856/2367 symbols completed. FND took 36.4s. Estimated remaining: 3h 13m 29s. Rate: 7.8 symbols/min [2025-10-20T18:49:41.436381] [INFO ] [MAIN ] FOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:41.448510] [DEBUG] [LOCAL ] FOF: Loaded 8707 existing records from primary storage. [2025-10-20T18:49:41.448755] [INFO ] [MAIN ] FOF: Loaded 8707 existing records. [2025-10-20T18:49:41.627366] [WARN ] [MAIN ] FOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:41.627794] [INFO ] [MAIN ] FOF: Wiping existing primary data before full refresh. [2025-10-20T18:49:41.627851] [INFO ] [LOCAL ] FOF: Starting deletion of primary data. [2025-10-20T18:49:41.628060] [DEBUG] [LOCAL ] FOF: Deleted primary data file. [2025-10-20T18:49:41.628116] [INFO ] [MAIN ] FOF: Starting full primary refresh. [2025-10-20T18:49:41.856830] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-10. [2025-10-20T18:49:41.991337] [INFO ] [REMOTE ] FOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:42.017367] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T18:49:42.088212] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:49:42.651369] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-02. [2025-10-20T18:49:43.451413] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-08. [2025-10-20T18:49:43.478834] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T18:49:43.485887] [INFO ] [REMOTE ] FNF: Full fetch complete (70031 records). [2025-10-20T18:49:43.485988] [DEBUG] [VALIDATOR ] FNF: Validating primary data. [2025-10-20T18:49:43.809946] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T18:49:43.901001] [DEBUG] [VALIDATOR ] FNF: All expected trading days are present. [2025-10-20T18:49:43.901082] [DEBUG] [VALIDATOR ] FNF: Primary data validation passed. [2025-10-20T18:49:43.901927] [INFO ] [MAIN ] FNF: Fetched primary data is valid. [2025-10-20T18:49:43.901989] [DEBUG] [LOCAL ] FNF: Saving primary data. [2025-10-20T18:49:43.903395] [DEBUG] [LOCAL ] FNF: Trimmed 70031 records to 48842 (keeping 200 days of data) [2025-10-20T18:49:44.193038] [DEBUG] [LOCAL ] FNF: Primary data saved (48842 records). [2025-10-20T18:49:44.193219] [INFO ] [MAIN ] FNF: Full history saved. [2025-10-20T18:49:44.193312] [INFO ] [MAIN ] Update Progress: 857/2367 symbols completed. FNF took 29.6s. Estimated remaining: 3h 13m 13s. Rate: 7.8 symbols/min [2025-10-20T18:49:44.257687] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-01. [2025-10-20T18:49:44.368357] [INFO ] [MAIN ] FOR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:44.396614] [DEBUG] [LOCAL ] FOR: Loaded 20025 existing records from primary storage. [2025-10-20T18:49:44.396901] [INFO ] [MAIN ] FOR: Loaded 20025 existing records. [2025-10-20T18:49:44.579162] [WARN ] [MAIN ] FOR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:44.579400] [INFO ] [MAIN ] FOR: Wiping existing primary data before full refresh. [2025-10-20T18:49:44.579447] [INFO ] [LOCAL ] FOR: Starting deletion of primary data. [2025-10-20T18:49:44.579620] [DEBUG] [LOCAL ] FOR: Deleted primary data file. [2025-10-20T18:49:44.579727] [INFO ] [MAIN ] FOR: Starting full primary refresh. [2025-10-20T18:49:44.952964] [INFO ] [REMOTE ] FOR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:45.045250] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T18:49:45.051318] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-02. [2025-10-20T18:49:45.089130] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T18:49:45.512847] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T18:49:45.851349] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-03. [2025-10-20T18:49:46.599999] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T18:49:46.651341] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-04. [2025-10-20T18:49:47.245915] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T18:49:47.451365] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-09. [2025-10-20T18:49:48.251396] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-05. [2025-10-20T18:49:48.714460] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T18:49:49.051339] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-03. [2025-10-20T18:49:49.101245] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T18:49:49.851345] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-06. [2025-10-20T18:49:50.005751] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T18:49:50.576349] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:49:50.651373] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-07. [2025-10-20T18:49:51.154802] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-10-20T18:49:51.451425] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-10. [2025-10-20T18:49:52.251415] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-04. [2025-10-20T18:49:53.056447] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-01. [2025-10-20T18:49:53.068711] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T18:49:53.076230] [INFO ] [REMOTE ] FNV: Full fetch complete (66861 records). [2025-10-20T18:49:53.076354] [DEBUG] [VALIDATOR ] FNV: Validating primary data. [2025-10-20T18:49:53.249071] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T18:49:53.480240] [DEBUG] [VALIDATOR ] FNV: All expected trading days are present. [2025-10-20T18:49:53.480322] [DEBUG] [VALIDATOR ] FNV: Primary data validation passed. [2025-10-20T18:49:53.481197] [INFO ] [MAIN ] FNV: Fetched primary data is valid. [2025-10-20T18:49:53.481260] [DEBUG] [LOCAL ] FNV: Saving primary data. [2025-10-20T18:49:53.482531] [DEBUG] [LOCAL ] FNV: Trimmed 66861 records to 46152 (keeping 200 days of data) [2025-10-20T18:49:53.762598] [DEBUG] [LOCAL ] FNV: Primary data saved (46152 records). [2025-10-20T18:49:53.762790] [INFO ] [MAIN ] FNV: Full history saved. [2025-10-20T18:49:53.762892] [INFO ] [MAIN ] Update Progress: 858/2367 symbols completed. FNV took 31.5s. Estimated remaining: 3h 13m 9s. Rate: 7.8 symbols/min [2025-10-20T18:49:53.859693] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-08. [2025-10-20T18:49:53.931569] [INFO ] [MAIN ] FOUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:53.978047] [DEBUG] [LOCAL ] FOUR: Loaded 50475 existing records from primary storage. [2025-10-20T18:49:53.978341] [INFO ] [MAIN ] FOUR: Loaded 50475 existing records. [2025-10-20T18:49:54.154959] [WARN ] [MAIN ] FOUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:54.155208] [INFO ] [MAIN ] FOUR: Wiping existing primary data before full refresh. [2025-10-20T18:49:54.155273] [INFO ] [LOCAL ] FOUR: Starting deletion of primary data. [2025-10-20T18:49:54.155858] [DEBUG] [LOCAL ] FOUR: Deleted primary data file. [2025-10-20T18:49:54.155923] [INFO ] [MAIN ] FOUR: Starting full primary refresh. [2025-10-20T18:49:54.325696] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T18:49:54.539071] [INFO ] [REMOTE ] FOUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:54.624271] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T18:49:54.651347] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-01. [2025-10-20T18:49:55.451377] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-09. [2025-10-20T18:49:56.251358] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-05. [2025-10-20T18:49:56.262426] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T18:49:56.331261] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:49:57.051376] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-02. [2025-10-20T18:49:57.146765] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T18:49:57.851388] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-06. [2025-10-20T18:49:58.635616] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T18:49:58.651352] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-02. [2025-10-20T18:49:58.847468] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T18:49:59.451371] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-10. [2025-10-20T18:49:59.675181] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:49:59.987161] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T18:49:59.989793] [INFO ] [REMOTE ] FOF: Full fetch complete (11895 records). [2025-10-20T18:49:59.989890] [DEBUG] [VALIDATOR ] FOF: Validating primary data. [2025-10-20T18:50:00.256559] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-07. [2025-10-20T18:50:00.371439] [DEBUG] [VALIDATOR ] FOF: All expected trading days are present. [2025-10-20T18:50:00.371515] [DEBUG] [VALIDATOR ] FOF: Primary data validation passed. [2025-10-20T18:50:00.371799] [INFO ] [MAIN ] FOF: Fetched primary data is valid. [2025-10-20T18:50:00.371858] [DEBUG] [LOCAL ] FOF: Saving primary data. [2025-10-20T18:50:00.372344] [DEBUG] [LOCAL ] FOF: Trimmed 11895 records to 8615 (keeping 200 days of data) [2025-10-20T18:50:00.431725] [DEBUG] [LOCAL ] FOF: Primary data saved (8615 records). [2025-10-20T18:50:00.431914] [INFO ] [MAIN ] FOF: Full history saved. [2025-10-20T18:50:00.432007] [INFO ] [MAIN ] Update Progress: 859/2367 symbols completed. FOF took 19.2s. Estimated remaining: 3h 12m 60s. Rate: 7.8 symbols/min [2025-10-20T18:50:00.640661] [INFO ] [MAIN ] FPF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:00.662393] [DEBUG] [LOCAL ] FPF: Loaded 19175 existing records from primary storage. [2025-10-20T18:50:00.662680] [INFO ] [MAIN ] FPF: Loaded 19175 existing records. [2025-10-20T18:50:00.867125] [WARN ] [MAIN ] FPF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:00.867342] [INFO ] [MAIN ] FPF: Wiping existing primary data before full refresh. [2025-10-20T18:50:00.867391] [INFO ] [LOCAL ] FPF: Starting deletion of primary data. [2025-10-20T18:50:00.867721] [DEBUG] [LOCAL ] FPF: Deleted primary data file. [2025-10-20T18:50:00.867778] [INFO ] [MAIN ] FPF: Starting full primary refresh. [2025-10-20T18:50:01.051771] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-03. [2025-10-20T18:50:01.254618] [INFO ] [REMOTE ] FPF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:01.375085] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:50:01.851360] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-01. [2025-10-20T18:50:02.651338] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-08. [2025-10-20T18:50:02.835267] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T18:50:03.246475] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T18:50:03.451448] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-03. [2025-10-20T18:50:03.629621] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:50:04.251408] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-04. [2025-10-20T18:50:04.684795] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T18:50:05.051395] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-04. [2025-10-20T18:50:05.851349] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-09. [2025-10-20T18:50:06.258441] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T18:50:06.293749] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T18:50:06.651411] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-05. [2025-10-20T18:50:06.859609] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T18:50:07.451355] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-10. [2025-10-20T18:50:07.824320] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:50:08.251367] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-05. [2025-10-20T18:50:08.374940] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T18:50:08.377846] [INFO ] [REMOTE ] FOA: Full fetch complete (21457 records). [2025-10-20T18:50:08.378016] [DEBUG] [VALIDATOR ] FOA: Validating primary data. [2025-10-20T18:50:08.756574] [DEBUG] [VALIDATOR ] FOA: All expected trading days are present. [2025-10-20T18:50:08.756685] [DEBUG] [VALIDATOR ] FOA: Primary data validation passed. [2025-10-20T18:50:08.757163] [INFO ] [MAIN ] FOA: Fetched primary data is valid. [2025-10-20T18:50:08.757206] [DEBUG] [LOCAL ] FOA: Saving primary data. [2025-10-20T18:50:08.757886] [DEBUG] [LOCAL ] FOA: Trimmed 21457 records to 13064 (keeping 200 days of data) [2025-10-20T18:50:08.833079] [DEBUG] [LOCAL ] FOA: Primary data saved (13064 records). [2025-10-20T18:50:08.833223] [INFO ] [MAIN ] FOA: Full history saved. [2025-10-20T18:50:08.833302] [INFO ] [MAIN ] Update Progress: 860/2367 symbols completed. FOA took 29.9s. Estimated remaining: 3h 12m 53s. Rate: 7.8 symbols/min [2025-10-20T18:50:09.002472] [INFO ] [MAIN ] FPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:09.022432] [DEBUG] [LOCAL ] FPH: Loaded 19603 existing records from primary storage. [2025-10-20T18:50:09.022760] [INFO ] [MAIN ] FPH: Loaded 19603 existing records. [2025-10-20T18:50:09.056577] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-02. [2025-10-20T18:50:09.205192] [WARN ] [MAIN ] FPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:09.205419] [INFO ] [MAIN ] FPH: Wiping existing primary data before full refresh. [2025-10-20T18:50:09.205469] [INFO ] [LOCAL ] FPH: Starting deletion of primary data. [2025-10-20T18:50:09.205757] [DEBUG] [LOCAL ] FPH: Deleted primary data file. [2025-10-20T18:50:09.205814] [INFO ] [MAIN ] FPH: Starting full primary refresh. [2025-10-20T18:50:09.561084] [INFO ] [REMOTE ] FPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:09.851364] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-01. [2025-10-20T18:50:10.025634] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T18:50:10.075839] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:50:10.651378] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-06. [2025-10-20T18:50:11.152970] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T18:50:11.451381] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-03. [2025-10-20T18:50:11.646051] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T18:50:12.251385] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-07. [2025-10-20T18:50:12.443412] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T18:50:13.051343] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-06. [2025-10-20T18:50:13.188853] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T18:50:13.851385] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-08. [2025-10-20T18:50:14.568606] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T18:50:14.651355] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-07. [2025-10-20T18:50:14.884037] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T18:50:15.451387] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-02. [2025-10-20T18:50:16.251345] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-04. [2025-10-20T18:50:16.292074] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:50:16.579124] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:50:17.051356] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-03. [2025-10-20T18:50:17.374294] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T18:50:17.780378] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T18:50:17.851389] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-04. [2025-10-20T18:50:18.651349] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-08. [2025-10-20T18:50:18.883295] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T18:50:19.451400] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-05. [2025-10-20T18:50:20.234506] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T18:50:20.251322] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-06. [2025-10-20T18:50:20.312880] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T18:50:21.051385] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-09. [2025-10-20T18:50:21.144126] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T18:50:21.851381] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-07. [2025-10-20T18:50:22.656487] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-05. [2025-10-20T18:50:22.675625] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:50:22.982406] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T18:50:23.204357] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T18:50:23.451416] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-06. [2025-10-20T18:50:24.251396] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-08. [2025-10-20T18:50:24.470540] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T18:50:25.051407] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-07. [2025-10-20T18:50:25.239522] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:50:25.851348] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-09. [2025-10-20T18:50:26.061904] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T18:50:26.651352] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-09. [2025-10-20T18:50:26.915753] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T18:50:27.451381] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-08. [2025-10-20T18:50:27.630976] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T18:50:28.251345] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-10. [2025-10-20T18:50:28.412001] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T18:50:29.051366] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-09. [2025-10-20T18:50:29.272993] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T18:50:29.276193] [INFO ] [REMOTE ] FPH: Full fetch complete (30906 records). [2025-10-20T18:50:29.276319] [DEBUG] [VALIDATOR ] FPH: Validating primary data. [2025-10-20T18:50:29.642558] [DEBUG] [VALIDATOR ] FPH: All expected trading days are present. [2025-10-20T18:50:29.642666] [DEBUG] [VALIDATOR ] FPH: Primary data validation passed. [2025-10-20T18:50:29.643245] [INFO ] [MAIN ] FPH: Fetched primary data is valid. [2025-10-20T18:50:29.643300] [DEBUG] [LOCAL ] FPH: Saving primary data. [2025-10-20T18:50:29.643983] [DEBUG] [LOCAL ] FPH: Trimmed 30906 records to 19328 (keeping 200 days of data) [2025-10-20T18:50:29.746993] [DEBUG] [LOCAL ] FPH: Primary data saved (19328 records). [2025-10-20T18:50:29.747118] [INFO ] [MAIN ] FPH: Full history saved. [2025-10-20T18:50:29.747193] [INFO ] [MAIN ] Update Progress: 861/2367 symbols completed. FPH took 20.9s. Estimated remaining: 3h 13m 9s. Rate: 7.8 symbols/min [2025-10-20T18:50:29.856502] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-10. [2025-10-20T18:50:29.919015] [INFO ] [MAIN ] FPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:29.950531] [DEBUG] [LOCAL ] FPI: Loaded 31833 existing records from primary storage. [2025-10-20T18:50:29.950844] [INFO ] [MAIN ] FPI: Loaded 31833 existing records. [2025-10-20T18:50:30.086363] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T18:50:30.150551] [WARN ] [MAIN ] FPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:30.150782] [INFO ] [MAIN ] FPI: Wiping existing primary data before full refresh. [2025-10-20T18:50:30.150832] [INFO ] [LOCAL ] FPI: Starting deletion of primary data. [2025-10-20T18:50:30.151016] [DEBUG] [LOCAL ] FPI: Deleted primary data file. [2025-10-20T18:50:30.151053] [INFO ] [MAIN ] FPI: Starting full primary refresh. [2025-10-20T18:50:30.502710] [INFO ] [REMOTE ] FPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:30.651351] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-01. [2025-10-20T18:50:31.451358] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-10. [2025-10-20T18:50:31.844017] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T18:50:32.132364] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-10 (2.28 seconds). [2025-10-20T18:50:32.140406] [INFO ] [REMOTE ] FOUR: Full fetch complete (72267 records). [2025-10-20T18:50:32.140592] [DEBUG] [VALIDATOR ] FOUR: Validating primary data. [2025-10-20T18:50:32.256740] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-10. [2025-10-20T18:50:32.587735] [DEBUG] [VALIDATOR ] FOUR: All expected trading days are present. [2025-10-20T18:50:32.587841] [DEBUG] [VALIDATOR ] FOUR: Primary data validation passed. [2025-10-20T18:50:32.588828] [INFO ] [MAIN ] FOUR: Fetched primary data is valid. [2025-10-20T18:50:32.588893] [DEBUG] [LOCAL ] FOUR: Saving primary data. [2025-10-20T18:50:32.590537] [DEBUG] [LOCAL ] FOUR: Trimmed 72267 records to 49761 (keeping 200 days of data) [2025-10-20T18:50:32.712891] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:50:32.787080] [INFO ] [REMOTE ] FOR: Full fetch complete (29773 records). [2025-10-20T18:50:32.787531] [DEBUG] [VALIDATOR ] FOR: Validating primary data. [2025-10-20T18:50:33.056080] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-02. [2025-10-20T18:50:33.285415] [DEBUG] [LOCAL ] FOUR: Primary data saved (49761 records). [2025-10-20T18:50:33.314459] [DEBUG] [VALIDATOR ] FOR: All expected trading days are present. [2025-10-20T18:50:33.312237] [INFO ] [MAIN ] FOUR: Full history saved. [2025-10-20T18:50:33.314745] [DEBUG] [VALIDATOR ] FOR: Primary data validation passed. [2025-10-20T18:50:33.315700] [INFO ] [MAIN ] FOR: Fetched primary data is valid. [2025-10-20T18:50:33.316567] [INFO ] [MAIN ] Update Progress: 862/2367 symbols completed. FOUR took 39.6s. Estimated remaining: 3h 12m 54s. Rate: 7.8 symbols/min [2025-10-20T18:50:33.316701] [DEBUG] [LOCAL ] FOR: Saving primary data. [2025-10-20T18:50:33.434089] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T18:50:33.434791] [DEBUG] [LOCAL ] FOR: Trimmed 29773 records to 19738 (keeping 200 days of data) [2025-10-20T18:50:33.478708] [INFO ] [REMOTE ] FPF: Full fetch complete (26543 records). [2025-10-20T18:50:33.541876] [INFO ] [MAIN ] FR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:33.542018] [DEBUG] [VALIDATOR ] FPF: Validating primary data. [2025-10-20T18:50:33.854720] [DEBUG] [LOCAL ] FOR: Primary data saved (19738 records). [2025-10-20T18:50:33.898453] [INFO ] [MAIN ] FOR: Full history saved. [2025-10-20T18:50:33.931077] [INFO ] [MAIN ] Update Progress: 863/2367 symbols completed. FOR took 49.7s. Estimated remaining: 3h 12m 34s. Rate: 7.8 symbols/min [2025-10-20T18:50:34.183891] [DEBUG] [LOCAL ] FR: Loaded 47301 existing records from primary storage. [2025-10-20T18:50:34.222035] [INFO ] [MAIN ] FR: Loaded 47301 existing records. [2025-10-20T18:50:34.227475] [DEBUG] [VALIDATOR ] FPF: All expected trading days are present. [2025-10-20T18:50:34.319190] [DEBUG] [VALIDATOR ] FPF: Primary data validation passed. [2025-10-20T18:50:34.347400] [INFO ] [MAIN ] FRA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:34.353747] [INFO ] [MAIN ] FPF: Fetched primary data is valid. [2025-10-20T18:50:34.390036] [DEBUG] [LOCAL ] FPF: Saving primary data. [2025-10-20T18:50:34.487872] [WARN ] [MAIN ] FR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:34.489142] [INFO ] [MAIN ] FR: Wiping existing primary data before full refresh. [2025-10-20T18:50:34.490890] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T18:50:34.496171] [INFO ] [LOCAL ] FR: Starting deletion of primary data. [2025-10-20T18:50:34.489287] [DEBUG] [LOCAL ] FPF: Trimmed 26543 records to 18844 (keeping 200 days of data) [2025-10-20T18:50:34.498242] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-03. [2025-10-20T18:50:34.497924] [DEBUG] [LOCAL ] FR: Deleted primary data file. [2025-10-20T18:50:34.505446] [INFO ] [MAIN ] FR: Starting full primary refresh. [2025-10-20T18:50:34.505378] [DEBUG] [LOCAL ] FRA: Loaded 14476 existing records from primary storage. [2025-10-20T18:50:34.560181] [INFO ] [MAIN ] FRA: Loaded 14476 existing records. [2025-10-20T18:50:34.849035] [DEBUG] [LOCAL ] FPF: Primary data saved (18844 records). [2025-10-20T18:50:34.859512] [INFO ] [MAIN ] FPF: Full history saved. [2025-10-20T18:50:34.859822] [INFO ] [MAIN ] Update Progress: 864/2367 symbols completed. FPF took 34.4s. Estimated remaining: 3h 12m 14s. Rate: 7.8 symbols/min [2025-10-20T18:50:35.039328] [WARN ] [MAIN ] FRA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:35.074310] [INFO ] [MAIN ] FRA: Wiping existing primary data before full refresh. [2025-10-20T18:50:35.080688] [INFO ] [LOCAL ] FRA: Starting deletion of primary data. [2025-10-20T18:50:35.123539] [DEBUG] [LOCAL ] FRA: Deleted primary data file. [2025-10-20T18:50:35.147972] [INFO ] [MAIN ] FRA: Starting full primary refresh. [2025-10-20T18:50:35.357428] [INFO ] [MAIN ] FRGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:35.489454] [INFO ] [REMOTE ] FR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:35.562344] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-01. [2025-10-20T18:50:35.656621] [DEBUG] [LOCAL ] FRGE: Loaded 16315 existing records from primary storage. [2025-10-20T18:50:35.676604] [INFO ] [MAIN ] FRGE: Loaded 16315 existing records. [2025-10-20T18:50:35.857131] [INFO ] [REMOTE ] FRA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:35.862964] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-01. [2025-10-20T18:50:35.951878] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T18:50:35.955633] [WARN ] [MAIN ] FRGE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:35.962430] [INFO ] [MAIN ] FRGE: Wiping existing primary data before full refresh. [2025-10-20T18:50:35.962793] [INFO ] [LOCAL ] FRGE: Starting deletion of primary data. [2025-10-20T18:50:35.963035] [DEBUG] [LOCAL ] FRGE: Deleted primary data file. [2025-10-20T18:50:35.963092] [INFO ] [MAIN ] FRGE: Starting full primary refresh. [2025-10-20T18:50:36.256475] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-04. [2025-10-20T18:50:36.306760] [INFO ] [REMOTE ] FRGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:37.031486] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:50:37.051328] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-02. [2025-10-20T18:50:37.133208] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:50:37.245993] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T18:50:37.851404] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-05. [2025-10-20T18:50:38.004214] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T18:50:38.651392] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-01. [2025-10-20T18:50:39.243227] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T18:50:39.451372] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-02. [2025-10-20T18:50:40.091038] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:50:40.251358] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-06. [2025-10-20T18:50:40.970583] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:50:41.051370] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-03. [2025-10-20T18:50:41.579375] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T18:50:41.851431] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-03. [2025-10-20T18:50:42.507408] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T18:50:42.651340] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-04. [2025-10-20T18:50:42.827517] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T18:50:43.451389] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-07. [2025-10-20T18:50:44.251391] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-04. [2025-10-20T18:50:44.369464] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T18:50:44.901976] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T18:50:45.051346] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-08. [2025-10-20T18:50:45.219868] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T18:50:45.851351] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-05. [2025-10-20T18:50:45.980993] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:50:46.651357] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-09. [2025-10-20T18:50:46.755390] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T18:50:47.451345] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-02. [2025-10-20T18:50:47.672724] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T18:50:48.251385] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-10. [2025-10-20T18:50:48.719182] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T18:50:49.051369] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-03. [2025-10-20T18:50:49.309673] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:50:49.315138] [INFO ] [REMOTE ] FPI: Full fetch complete (43901 records). [2025-10-20T18:50:49.315264] [DEBUG] [VALIDATOR ] FPI: Validating primary data. [2025-10-20T18:50:49.731508] [DEBUG] [VALIDATOR ] FPI: All expected trading days are present. [2025-10-20T18:50:49.731821] [DEBUG] [VALIDATOR ] FPI: Primary data validation passed. [2025-10-20T18:50:49.732753] [INFO ] [MAIN ] FPI: Fetched primary data is valid. [2025-10-20T18:50:49.732825] [DEBUG] [LOCAL ] FPI: Saving primary data. [2025-10-20T18:50:49.734150] [DEBUG] [LOCAL ] FPI: Trimmed 43901 records to 31464 (keeping 200 days of data) [2025-10-20T18:50:49.855062] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-05. [2025-10-20T18:50:49.961320] [DEBUG] [LOCAL ] FPI: Primary data saved (31464 records). [2025-10-20T18:50:49.961476] [INFO ] [MAIN ] FPI: Full history saved. [2025-10-20T18:50:49.961561] [INFO ] [MAIN ] Update Progress: 865/2367 symbols completed. FPI took 20.2s. Estimated remaining: 3h 12m 19s. Rate: 7.8 symbols/min [2025-10-20T18:50:50.177499] [INFO ] [MAIN ] FRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:50.236934] [DEBUG] [LOCAL ] FRO: Loaded 52460 existing records from primary storage. [2025-10-20T18:50:50.237525] [INFO ] [MAIN ] FRO: Loaded 52460 existing records. [2025-10-20T18:50:50.502132] [WARN ] [MAIN ] FRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:50.502513] [INFO ] [MAIN ] FRO: Wiping existing primary data before full refresh. [2025-10-20T18:50:50.502678] [INFO ] [LOCAL ] FRO: Starting deletion of primary data. [2025-10-20T18:50:50.512754] [DEBUG] [LOCAL ] FRO: Deleted primary data file. [2025-10-20T18:50:50.527692] [INFO ] [MAIN ] FRO: Starting full primary refresh. [2025-10-20T18:50:50.571873] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:50:50.656520] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-06. [2025-10-20T18:50:51.102065] [INFO ] [REMOTE ] FRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:51.451380] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-01. [2025-10-20T18:50:51.604880] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T18:50:51.700949] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:50:52.251408] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-04. [2025-10-20T18:50:53.051388] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-06. [2025-10-20T18:50:53.485049] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-10-20T18:50:53.512220] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T18:50:53.851376] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-05. [2025-10-20T18:50:54.421292] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:50:54.651364] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-07. [2025-10-20T18:50:54.890134] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:50:55.451374] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-07. [2025-10-20T18:50:56.251437] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-02. [2025-10-20T18:50:56.303431] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:50:56.418037] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T18:50:57.051424] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-08. [2025-10-20T18:50:57.851459] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-06. [2025-10-20T18:50:57.870516] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T18:50:58.117477] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T18:50:58.651431] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-08. [2025-10-20T18:50:58.719139] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T18:50:59.451382] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-07. [2025-10-20T18:51:00.116381] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:51:00.251393] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-09. [2025-10-20T18:51:00.340148] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T18:51:01.051426] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-03. [2025-10-20T18:51:01.692824] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T18:51:01.851417] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-09. [2025-10-20T18:51:02.651414] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-10. [2025-10-20T18:51:03.155448] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T18:51:03.313581] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T18:51:03.317090] [INFO ] [REMOTE ] FRA: Full fetch complete (23561 records). [2025-10-20T18:51:03.317320] [DEBUG] [VALIDATOR ] FRA: Validating primary data. [2025-10-20T18:51:03.437958] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:51:03.456637] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-08. [2025-10-20T18:51:03.719140] [DEBUG] [VALIDATOR ] FRA: All expected trading days are present. [2025-10-20T18:51:03.719221] [DEBUG] [VALIDATOR ] FRA: Primary data validation passed. [2025-10-20T18:51:03.719702] [INFO ] [MAIN ] FRA: Fetched primary data is valid. [2025-10-20T18:51:03.719761] [DEBUG] [LOCAL ] FRA: Saving primary data. [2025-10-20T18:51:03.720328] [DEBUG] [LOCAL ] FRA: Trimmed 23561 records to 14261 (keeping 200 days of data) [2025-10-20T18:51:03.809446] [DEBUG] [LOCAL ] FRA: Primary data saved (14261 records). [2025-10-20T18:51:03.809592] [INFO ] [MAIN ] FRA: Full history saved. [2025-10-20T18:51:03.809695] [INFO ] [MAIN ] Update Progress: 866/2367 symbols completed. FRA took 29.9s. Estimated remaining: 3h 12m 22s. Rate: 7.8 symbols/min [2025-10-20T18:51:03.978963] [INFO ] [MAIN ] FRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:04.016313] [DEBUG] [LOCAL ] FRT: Loaded 42290 existing records from primary storage. [2025-10-20T18:51:04.016615] [INFO ] [MAIN ] FRT: Loaded 42290 existing records. [2025-10-20T18:51:04.198667] [WARN ] [MAIN ] FRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:04.198936] [INFO ] [MAIN ] FRT: Wiping existing primary data before full refresh. [2025-10-20T18:51:04.198990] [INFO ] [LOCAL ] FRT: Starting deletion of primary data. [2025-10-20T18:51:04.199289] [DEBUG] [LOCAL ] FRT: Deleted primary data file. [2025-10-20T18:51:04.199347] [INFO ] [MAIN ] FRT: Starting full primary refresh. [2025-10-20T18:51:04.260669] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-04. [2025-10-20T18:51:04.522584] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T18:51:04.608469] [INFO ] [REMOTE ] FRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:05.051426] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-01. [2025-10-20T18:51:05.851452] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-10. [2025-10-20T18:51:06.193254] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T18:51:06.521004] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T18:51:06.651389] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-05. [2025-10-20T18:51:07.449312] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T18:51:07.455756] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-02. [2025-10-20T18:51:07.457506] [INFO ] [REMOTE ] FR: Full fetch complete (67109 records). [2025-10-20T18:51:07.457589] [DEBUG] [VALIDATOR ] FR: Validating primary data. [2025-10-20T18:51:07.858271] [DEBUG] [VALIDATOR ] FR: All expected trading days are present. [2025-10-20T18:51:07.858364] [DEBUG] [VALIDATOR ] FR: Primary data validation passed. [2025-10-20T18:51:07.859297] [INFO ] [MAIN ] FR: Fetched primary data is valid. [2025-10-20T18:51:07.859377] [DEBUG] [LOCAL ] FR: Saving primary data. [2025-10-20T18:51:07.860898] [DEBUG] [LOCAL ] FR: Trimmed 67109 records to 46684 (keeping 200 days of data) [2025-10-20T18:51:07.985566] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:51:08.190089] [DEBUG] [LOCAL ] FR: Primary data saved (46684 records). [2025-10-20T18:51:08.190236] [INFO ] [MAIN ] FR: Full history saved. [2025-10-20T18:51:08.190319] [INFO ] [MAIN ] Update Progress: 867/2367 symbols completed. FR took 34.9s. Estimated remaining: 3h 12m 9s. Rate: 7.8 symbols/min [2025-10-20T18:51:08.256503] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-09. [2025-10-20T18:51:08.365153] [INFO ] [MAIN ] FSCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:08.403072] [DEBUG] [LOCAL ] FSCO: Loaded 39152 existing records from primary storage. [2025-10-20T18:51:08.403407] [INFO ] [MAIN ] FSCO: Loaded 39152 existing records. [2025-10-20T18:51:08.587935] [WARN ] [MAIN ] FSCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:08.588152] [INFO ] [MAIN ] FSCO: Wiping existing primary data before full refresh. [2025-10-20T18:51:08.588200] [INFO ] [LOCAL ] FSCO: Starting deletion of primary data. [2025-10-20T18:51:08.588705] [DEBUG] [LOCAL ] FSCO: Deleted primary data file. [2025-10-20T18:51:08.588761] [INFO ] [MAIN ] FSCO: Starting full primary refresh. [2025-10-20T18:51:08.961811] [INFO ] [REMOTE ] FSCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:09.028394] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:51:09.051353] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-06. [2025-10-20T18:51:09.136382] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T18:51:09.851374] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-03. [2025-10-20T18:51:10.651415] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-10. [2025-10-20T18:51:10.718184] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:51:11.427599] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T18:51:11.451378] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-04. [2025-10-20T18:51:11.533789] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T18:51:11.536485] [INFO ] [REMOTE ] FRGE: Full fetch complete (30455 records). [2025-10-20T18:51:11.536598] [DEBUG] [VALIDATOR ] FRGE: Validating primary data. [2025-10-20T18:51:11.959552] [DEBUG] [VALIDATOR ] FRGE: All expected trading days are present. [2025-10-20T18:51:11.959662] [DEBUG] [VALIDATOR ] FRGE: Primary data validation passed. [2025-10-20T18:51:11.960272] [INFO ] [MAIN ] FRGE: Fetched primary data is valid. [2025-10-20T18:51:11.960315] [DEBUG] [LOCAL ] FRGE: Saving primary data. [2025-10-20T18:51:11.961249] [DEBUG] [LOCAL ] FRGE: Trimmed 30455 records to 15767 (keeping 200 days of data) [2025-10-20T18:51:12.066421] [DEBUG] [LOCAL ] FRGE: Primary data saved (15767 records). [2025-10-20T18:51:12.066586] [INFO ] [MAIN ] FRGE: Full history saved. [2025-10-20T18:51:12.066709] [INFO ] [MAIN ] Update Progress: 868/2367 symbols completed. FRGE took 37.2s. Estimated remaining: 3h 11m 55s. Rate: 7.8 symbols/min [2025-10-20T18:51:12.251731] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-07. [2025-10-20T18:51:12.269941] [INFO ] [MAIN ] FSK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:12.319014] [DEBUG] [LOCAL ] FSK: Loaded 50273 existing records from primary storage. [2025-10-20T18:51:12.319367] [INFO ] [MAIN ] FSK: Loaded 50273 existing records. [2025-10-20T18:51:12.495969] [WARN ] [MAIN ] FSK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:12.496187] [INFO ] [MAIN ] FSK: Wiping existing primary data before full refresh. [2025-10-20T18:51:12.496234] [INFO ] [LOCAL ] FSK: Starting deletion of primary data. [2025-10-20T18:51:12.496424] [DEBUG] [LOCAL ] FSK: Deleted primary data file. [2025-10-20T18:51:12.496461] [INFO ] [MAIN ] FSK: Starting full primary refresh. [2025-10-20T18:51:12.851005] [INFO ] [REMOTE ] FSK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:13.051403] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-01. [2025-10-20T18:51:13.099152] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:51:13.851378] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-05. [2025-10-20T18:51:14.023174] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T18:51:14.206674] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:51:14.651384] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-08. [2025-10-20T18:51:15.390262] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:51:15.451411] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-06. [2025-10-20T18:51:16.261376] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-02. [2025-10-20T18:51:16.267048] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T18:51:16.412162] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T18:51:17.051449] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-01. [2025-10-20T18:51:17.534206] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T18:51:17.851377] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-03. [2025-10-20T18:51:18.651397] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-07. [2025-10-20T18:51:18.723753] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T18:51:19.451445] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-02. [2025-10-20T18:51:19.601358] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:51:19.928283] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:51:20.251393] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-04. [2025-10-20T18:51:21.051404] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-08. [2025-10-20T18:51:21.180346] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T18:51:21.736882] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:51:21.851349] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-05. [2025-10-20T18:51:22.546772] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T18:51:22.651377] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-03. [2025-10-20T18:51:23.110036] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T18:51:23.451400] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-09. [2025-10-20T18:51:24.257537] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-09. [2025-10-20T18:51:24.272983] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:51:25.063855] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-06. [2025-10-20T18:51:25.066338] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:51:25.457732] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:51:25.851389] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-10. [2025-10-20T18:51:26.465495] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:51:26.651377] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-04. [2025-10-20T18:51:27.451437] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-10. [2025-10-20T18:51:27.713433] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T18:51:27.721058] [INFO ] [REMOTE ] FRO: Full fetch complete (75378 records). [2025-10-20T18:51:27.721186] [DEBUG] [VALIDATOR ] FRO: Validating primary data. [2025-10-20T18:51:28.116328] [DEBUG] [VALIDATOR ] FRO: All expected trading days are present. [2025-10-20T18:51:28.116494] [DEBUG] [VALIDATOR ] FRO: Primary data validation passed. [2025-10-20T18:51:28.117469] [INFO ] [MAIN ] FRO: Fetched primary data is valid. [2025-10-20T18:51:28.117575] [DEBUG] [LOCAL ] FRO: Saving primary data. [2025-10-20T18:51:28.119327] [DEBUG] [LOCAL ] FRO: Trimmed 75378 records to 51691 (keeping 200 days of data) [2025-10-20T18:51:28.242944] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:51:28.256973] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-07. [2025-10-20T18:51:28.463726] [DEBUG] [LOCAL ] FRO: Primary data saved (51691 records). [2025-10-20T18:51:28.463874] [INFO ] [MAIN ] FRO: Full history saved. [2025-10-20T18:51:28.463953] [INFO ] [MAIN ] Update Progress: 869/2367 symbols completed. FRO took 38.5s. Estimated remaining: 3h 12m 2s. Rate: 7.8 symbols/min [2025-10-20T18:51:28.634962] [INFO ] [MAIN ] FSLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:28.680989] [DEBUG] [LOCAL ] FSLY: Loaded 52088 existing records from primary storage. [2025-10-20T18:51:28.681385] [INFO ] [MAIN ] FSLY: Loaded 52088 existing records. [2025-10-20T18:51:28.870228] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:51:28.873792] [WARN ] [MAIN ] FSLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:28.879151] [INFO ] [MAIN ] FSLY: Wiping existing primary data before full refresh. [2025-10-20T18:51:28.879400] [INFO ] [LOCAL ] FSLY: Starting deletion of primary data. [2025-10-20T18:51:28.879740] [DEBUG] [LOCAL ] FSLY: Deleted primary data file. [2025-10-20T18:51:28.879806] [INFO ] [MAIN ] FSLY: Starting full primary refresh. [2025-10-20T18:51:28.879881] [INFO ] [REMOTE ] FRT: Full fetch complete (60504 records). [2025-10-20T18:51:28.941939] [DEBUG] [VALIDATOR ] FRT: Validating primary data. [2025-10-20T18:51:29.056456] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-05. [2025-10-20T18:51:29.593062] [INFO ] [REMOTE ] FSLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:29.669487] [DEBUG] [VALIDATOR ] FRT: All expected trading days are present. [2025-10-20T18:51:29.669732] [DEBUG] [VALIDATOR ] FRT: Primary data validation passed. [2025-10-20T18:51:29.670573] [INFO ] [MAIN ] FRT: Fetched primary data is valid. [2025-10-20T18:51:29.670692] [DEBUG] [LOCAL ] FRT: Saving primary data. [2025-10-20T18:51:29.695280] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:51:29.694190] [DEBUG] [LOCAL ] FRT: Trimmed 60504 records to 41648 (keeping 200 days of data) [2025-10-20T18:51:29.876592] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-01. [2025-10-20T18:51:29.955841] [DEBUG] [LOCAL ] FRT: Primary data saved (41648 records). [2025-10-20T18:51:29.955948] [INFO ] [MAIN ] FRT: Full history saved. [2025-10-20T18:51:29.956019] [INFO ] [MAIN ] Update Progress: 870/2367 symbols completed. FRT took 26.1s. Estimated remaining: 3h 11m 44s. Rate: 7.8 symbols/min [2025-10-20T18:51:30.123523] [INFO ] [MAIN ] FSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:30.173183] [DEBUG] [LOCAL ] FSM: Loaded 54081 existing records from primary storage. [2025-10-20T18:51:30.173631] [INFO ] [MAIN ] FSM: Loaded 54081 existing records. [2025-10-20T18:51:30.352970] [WARN ] [MAIN ] FSM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:30.353188] [INFO ] [MAIN ] FSM: Wiping existing primary data before full refresh. [2025-10-20T18:51:30.353242] [INFO ] [LOCAL ] FSM: Starting deletion of primary data. [2025-10-20T18:51:30.353437] [DEBUG] [LOCAL ] FSM: Deleted primary data file. [2025-10-20T18:51:30.353492] [INFO ] [MAIN ] FSM: Starting full primary refresh. [2025-10-20T18:51:30.654353] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-08. [2025-10-20T18:51:30.709613] [INFO ] [REMOTE ] FSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:30.738185] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:51:31.451345] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-01. [2025-10-20T18:51:31.706362] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:51:31.735310] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:51:32.251424] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-02. [2025-10-20T18:51:33.051374] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-06. [2025-10-20T18:51:33.186846] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T18:51:33.851390] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-02. [2025-10-20T18:51:33.943623] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T18:51:34.143122] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T18:51:34.651415] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-03. [2025-10-20T18:51:35.451369] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-07. [2025-10-20T18:51:35.607336] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T18:51:36.251434] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-09. [2025-10-20T18:51:36.388473] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T18:51:37.051354] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-03. [2025-10-20T18:51:37.135351] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T18:51:37.731350] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T18:51:37.851360] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-10. [2025-10-20T18:51:38.651418] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-08. [2025-10-20T18:51:38.841504] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T18:51:39.451408] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-04. [2025-10-20T18:51:39.530233] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T18:51:39.537152] [INFO ] [REMOTE ] FSCO: Full fetch complete (55405 records). [2025-10-20T18:51:39.537254] [DEBUG] [VALIDATOR ] FSCO: Validating primary data. [2025-10-20T18:51:39.927993] [DEBUG] [VALIDATOR ] FSCO: All expected trading days are present. [2025-10-20T18:51:39.928071] [DEBUG] [VALIDATOR ] FSCO: Primary data validation passed. [2025-10-20T18:51:39.928818] [INFO ] [MAIN ] FSCO: Fetched primary data is valid. [2025-10-20T18:51:39.928878] [DEBUG] [LOCAL ] FSCO: Saving primary data. [2025-10-20T18:51:39.930012] [DEBUG] [LOCAL ] FSCO: Trimmed 55405 records to 38696 (keeping 200 days of data) [2025-10-20T18:51:40.147917] [DEBUG] [LOCAL ] FSCO: Primary data saved (38696 records). [2025-10-20T18:51:40.148046] [INFO ] [MAIN ] FSCO: Full history saved. [2025-10-20T18:51:40.148123] [INFO ] [MAIN ] Update Progress: 871/2367 symbols completed. FSCO took 32.0s. Estimated remaining: 3h 11m 40s. Rate: 7.8 symbols/min [2025-10-20T18:51:40.253402] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-04. [2025-10-20T18:51:40.318293] [INFO ] [MAIN ] FSS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:40.376527] [DEBUG] [LOCAL ] FSS: Loaded 34103 existing records from primary storage. [2025-10-20T18:51:40.378095] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:51:40.388723] [INFO ] [MAIN ] FSS: Loaded 34103 existing records. [2025-10-20T18:51:40.567502] [WARN ] [MAIN ] FSS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:40.567713] [INFO ] [MAIN ] FSS: Wiping existing primary data before full refresh. [2025-10-20T18:51:40.567769] [INFO ] [LOCAL ] FSS: Starting deletion of primary data. [2025-10-20T18:51:40.567953] [DEBUG] [LOCAL ] FSS: Deleted primary data file. [2025-10-20T18:51:40.568009] [INFO ] [MAIN ] FSS: Starting full primary refresh. [2025-10-20T18:51:40.917291] [INFO ] [REMOTE ] FSS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:41.060252] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-09. [2025-10-20T18:51:41.065580] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T18:51:41.851351] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-01. [2025-10-20T18:51:42.049318] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:51:42.656859] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-05. [2025-10-20T18:51:42.677908] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:51:43.109736] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:51:43.451370] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-02. [2025-10-20T18:51:44.251371] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-10. [2025-10-20T18:51:44.310851] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T18:51:44.784704] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:51:45.051512] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-06. [2025-10-20T18:51:45.851437] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-05. [2025-10-20T18:51:46.197920] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T18:51:46.205579] [INFO ] [REMOTE ] FSK: Full fetch complete (72132 records). [2025-10-20T18:51:46.205726] [DEBUG] [VALIDATOR ] FSK: Validating primary data. [2025-10-20T18:51:46.598747] [DEBUG] [VALIDATOR ] FSK: All expected trading days are present. [2025-10-20T18:51:46.598878] [DEBUG] [VALIDATOR ] FSK: Primary data validation passed. [2025-10-20T18:51:46.599888] [INFO ] [MAIN ] FSK: Fetched primary data is valid. [2025-10-20T18:51:46.599953] [DEBUG] [LOCAL ] FSK: Saving primary data. [2025-10-20T18:51:46.601455] [DEBUG] [LOCAL ] FSK: Trimmed 72132 records to 49506 (keeping 200 days of data) [2025-10-20T18:51:46.655376] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-03. [2025-10-20T18:51:46.870329] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T18:51:46.965965] [DEBUG] [LOCAL ] FSK: Primary data saved (49506 records). [2025-10-20T18:51:46.966099] [INFO ] [MAIN ] FSK: Full history saved. [2025-10-20T18:51:46.966531] [INFO ] [MAIN ] Update Progress: 872/2367 symbols completed. FSK took 34.9s. Estimated remaining: 3h 11m 31s. Rate: 7.8 symbols/min [2025-10-20T18:51:47.133035] [INFO ] [MAIN ] FT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:47.142596] [DEBUG] [LOCAL ] FT: Loaded 6884 existing records from primary storage. [2025-10-20T18:51:47.142978] [INFO ] [MAIN ] FT: Loaded 6884 existing records. [2025-10-20T18:51:47.317982] [WARN ] [MAIN ] FT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:47.318169] [INFO ] [MAIN ] FT: Wiping existing primary data before full refresh. [2025-10-20T18:51:47.318214] [INFO ] [LOCAL ] FT: Starting deletion of primary data. [2025-10-20T18:51:47.318360] [DEBUG] [LOCAL ] FT: Deleted primary data file. [2025-10-20T18:51:47.318396] [INFO ] [MAIN ] FT: Starting full primary refresh. [2025-10-20T18:51:47.455343] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-07. [2025-10-20T18:51:47.702920] [INFO ] [REMOTE ] FT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:47.737350] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:51:48.123419] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:51:48.251346] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-01. [2025-10-20T18:51:48.816048] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T18:51:49.057294] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-02. [2025-10-20T18:51:49.064432] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T18:51:49.587661] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:51:49.851368] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-08. [2025-10-20T18:51:50.651371] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-06. [2025-10-20T18:51:51.451364] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-03. [2025-10-20T18:51:51.662269] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:51:52.044286] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T18:51:52.073666] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:51:52.251369] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-09. [2025-10-20T18:51:53.051363] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-04. [2025-10-20T18:51:53.851428] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-04. [2025-10-20T18:51:54.117441] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T18:51:54.202068] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T18:51:54.561103] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T18:51:54.651425] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-10. [2025-10-20T18:51:55.451376] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-05. [2025-10-20T18:51:56.143979] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T18:51:56.251434] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-06. [2025-10-20T18:51:56.623579] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T18:51:56.632997] [INFO ] [REMOTE ] FSM: Full fetch complete (77119 records). [2025-10-20T18:51:56.633165] [DEBUG] [VALIDATOR ] FSM: Validating primary data. [2025-10-20T18:51:56.917407] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T18:51:57.043291] [DEBUG] [VALIDATOR ] FSM: All expected trading days are present. [2025-10-20T18:51:57.043585] [DEBUG] [VALIDATOR ] FSM: Primary data validation passed. [2025-10-20T18:51:57.044536] [INFO ] [MAIN ] FSM: Fetched primary data is valid. [2025-10-20T18:51:57.044613] [DEBUG] [LOCAL ] FSM: Saving primary data. [2025-10-20T18:51:57.046214] [DEBUG] [LOCAL ] FSM: Trimmed 77119 records to 53303 (keeping 200 days of data) [2025-10-20T18:51:57.092198] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-05. [2025-10-20T18:51:57.354339] [DEBUG] [LOCAL ] FSM: Primary data saved (53303 records). [2025-10-20T18:51:57.354480] [INFO ] [MAIN ] FSM: Full history saved. [2025-10-20T18:51:57.354559] [INFO ] [MAIN ] Update Progress: 873/2367 symbols completed. FSM took 27.4s. Estimated remaining: 3h 11m 28s. Rate: 7.8 symbols/min [2025-10-20T18:51:57.523985] [INFO ] [MAIN ] FTHY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:57.537945] [DEBUG] [LOCAL ] FTHY: Loaded 12522 existing records from primary storage. [2025-10-20T18:51:57.538193] [INFO ] [MAIN ] FTHY: Loaded 12522 existing records. [2025-10-20T18:51:57.711891] [WARN ] [MAIN ] FTHY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:57.712169] [INFO ] [MAIN ] FTHY: Wiping existing primary data before full refresh. [2025-10-20T18:51:57.712217] [INFO ] [LOCAL ] FTHY: Starting deletion of primary data. [2025-10-20T18:51:57.712381] [DEBUG] [LOCAL ] FTHY: Deleted primary data file. [2025-10-20T18:51:57.712419] [INFO ] [MAIN ] FTHY: Starting full primary refresh. [2025-10-20T18:51:57.858814] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-07. [2025-10-20T18:51:58.074075] [INFO ] [REMOTE ] FTHY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:58.453707] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T18:51:58.651383] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-06. [2025-10-20T18:51:59.451367] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-01. [2025-10-20T18:51:59.667465] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:51:59.731177] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T18:52:00.251442] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-07. [2025-10-20T18:52:00.364877] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T18:52:01.051546] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-08. [2025-10-20T18:52:01.856990] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-02. [2025-10-20T18:52:01.879201] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T18:52:02.651432] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-08. [2025-10-20T18:52:02.674224] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T18:52:02.855842] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T18:52:03.451440] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-07. [2025-10-20T18:52:03.635105] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:52:04.183866] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:52:04.251366] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-08. [2025-10-20T18:52:05.044615] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T18:52:05.051330] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-09. [2025-10-20T18:52:05.793313] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T18:52:05.851464] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-03. [2025-10-20T18:52:06.599112] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T18:52:06.651424] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-09. [2025-10-20T18:52:07.451427] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-09. [2025-10-20T18:52:08.068970] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T18:52:08.251341] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-10. [2025-10-20T18:52:09.051361] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-10. [2025-10-20T18:52:09.277486] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:52:09.293514] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T18:52:09.295565] [INFO ] [REMOTE ] FT: Full fetch complete (9291 records). [2025-10-20T18:52:09.295638] [DEBUG] [VALIDATOR ] FT: Validating primary data. [2025-10-20T18:52:09.684042] [DEBUG] [VALIDATOR ] FT: All expected trading days are present. [2025-10-20T18:52:09.684119] [DEBUG] [VALIDATOR ] FT: Primary data validation passed. [2025-10-20T18:52:09.684354] [INFO ] [MAIN ] FT: Fetched primary data is valid. [2025-10-20T18:52:09.684392] [DEBUG] [LOCAL ] FT: Saving primary data. [2025-10-20T18:52:09.684857] [DEBUG] [LOCAL ] FT: Trimmed 9291 records to 6734 (keeping 200 days of data) [2025-10-20T18:52:09.722807] [DEBUG] [LOCAL ] FT: Primary data saved (6734 records). [2025-10-20T18:52:09.722910] [INFO ] [MAIN ] FT: Full history saved. [2025-10-20T18:52:09.722982] [INFO ] [MAIN ] Update Progress: 874/2367 symbols completed. FT took 22.8s. Estimated remaining: 3h 11m 28s. Rate: 7.8 symbols/min [2025-10-20T18:52:09.861671] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-10. [2025-10-20T18:52:09.894226] [INFO ] [MAIN ] FTI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:09.944560] [DEBUG] [LOCAL ] FTI: Loaded 54070 existing records from primary storage. [2025-10-20T18:52:09.944921] [INFO ] [MAIN ] FTI: Loaded 54070 existing records. [2025-10-20T18:52:10.132505] [WARN ] [MAIN ] FTI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:10.132794] [INFO ] [MAIN ] FTI: Wiping existing primary data before full refresh. [2025-10-20T18:52:10.132861] [INFO ] [LOCAL ] FTI: Starting deletion of primary data. [2025-10-20T18:52:10.133215] [DEBUG] [LOCAL ] FTI: Deleted primary data file. [2025-10-20T18:52:10.133284] [INFO ] [MAIN ] FTI: Starting full primary refresh. [2025-10-20T18:52:10.500438] [INFO ] [REMOTE ] FTI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:10.509513] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:52:10.514287] [INFO ] [REMOTE ] FSS: Full fetch complete (48636 records). [2025-10-20T18:52:10.514404] [DEBUG] [VALIDATOR ] FSS: Validating primary data. [2025-10-20T18:52:10.656488] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-01. [2025-10-20T18:52:10.903445] [DEBUG] [VALIDATOR ] FSS: All expected trading days are present. [2025-10-20T18:52:10.903518] [DEBUG] [VALIDATOR ] FSS: Primary data validation passed. [2025-10-20T18:52:10.904227] [INFO ] [MAIN ] FSS: Fetched primary data is valid. [2025-10-20T18:52:10.904289] [DEBUG] [LOCAL ] FSS: Saving primary data. [2025-10-20T18:52:10.905223] [DEBUG] [LOCAL ] FSS: Trimmed 48636 records to 33463 (keeping 200 days of data) [2025-10-20T18:52:11.104335] [DEBUG] [LOCAL ] FSS: Primary data saved (33463 records). [2025-10-20T18:52:11.104459] [INFO ] [MAIN ] FSS: Full history saved. [2025-10-20T18:52:11.104533] [INFO ] [MAIN ] Update Progress: 875/2367 symbols completed. FSS took 31.0s. Estimated remaining: 3h 11m 10s. Rate: 7.8 symbols/min [2025-10-20T18:52:11.270919] [INFO ] [MAIN ] FTK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:11.303232] [DEBUG] [LOCAL ] FTK: Loaded 33277 existing records from primary storage. [2025-10-20T18:52:11.303495] [INFO ] [MAIN ] FTK: Loaded 33277 existing records. [2025-10-20T18:52:11.456782] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-04. [2025-10-20T18:52:11.479841] [WARN ] [MAIN ] FTK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:11.480877] [INFO ] [MAIN ] FTK: Wiping existing primary data before full refresh. [2025-10-20T18:52:11.480927] [INFO ] [LOCAL ] FTK: Starting deletion of primary data. [2025-10-20T18:52:11.481086] [DEBUG] [LOCAL ] FTK: Deleted primary data file. [2025-10-20T18:52:11.481125] [INFO ] [MAIN ] FTK: Starting full primary refresh. [2025-10-20T18:52:11.850274] [INFO ] [REMOTE ] FTK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:11.871345] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T18:52:11.879404] [INFO ] [REMOTE ] FSLY: Full fetch complete (74955 records). [2025-10-20T18:52:11.879671] [DEBUG] [VALIDATOR ] FSLY: Validating primary data. [2025-10-20T18:52:12.265453] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T18:52:12.284275] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-01. [2025-10-20T18:52:12.319018] [DEBUG] [VALIDATOR ] FSLY: All expected trading days are present. [2025-10-20T18:52:12.319123] [DEBUG] [VALIDATOR ] FSLY: Primary data validation passed. [2025-10-20T18:52:12.320067] [INFO ] [MAIN ] FSLY: Fetched primary data is valid. [2025-10-20T18:52:12.320131] [DEBUG] [LOCAL ] FSLY: Saving primary data. [2025-10-20T18:52:12.321512] [DEBUG] [LOCAL ] FSLY: Trimmed 74955 records to 51345 (keeping 200 days of data) [2025-10-20T18:52:12.527344] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:52:12.625193] [DEBUG] [LOCAL ] FSLY: Primary data saved (51345 records). [2025-10-20T18:52:12.625326] [INFO ] [MAIN ] FSLY: Full history saved. [2025-10-20T18:52:12.625401] [INFO ] [MAIN ] Update Progress: 876/2367 symbols completed. FSLY took 44.2s. Estimated remaining: 3h 10m 52s. Rate: 7.8 symbols/min [2025-10-20T18:52:12.791287] [INFO ] [MAIN ] FTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:12.828872] [DEBUG] [LOCAL ] FTS: Loaded 43161 existing records from primary storage. [2025-10-20T18:52:12.829222] [INFO ] [MAIN ] FTS: Loaded 43161 existing records. [2025-10-20T18:52:13.005026] [WARN ] [MAIN ] FTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:13.005241] [INFO ] [MAIN ] FTS: Wiping existing primary data before full refresh. [2025-10-20T18:52:13.005286] [INFO ] [LOCAL ] FTS: Starting deletion of primary data. [2025-10-20T18:52:13.005711] [DEBUG] [LOCAL ] FTS: Deleted primary data file. [2025-10-20T18:52:13.005767] [INFO ] [MAIN ] FTS: Starting full primary refresh. [2025-10-20T18:52:13.058825] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-02. [2025-10-20T18:52:13.363203] [INFO ] [REMOTE ] FTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:13.426113] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T18:52:13.851408] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-01. [2025-10-20T18:52:14.651399] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-02. [2025-10-20T18:52:14.688182] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:52:15.345173] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T18:52:15.451380] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-02. [2025-10-20T18:52:15.706292] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:52:16.251426] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-03. [2025-10-20T18:52:16.995399] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:52:17.051449] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-05. [2025-10-20T18:52:17.851353] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-03. [2025-10-20T18:52:17.884444] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:52:17.958967] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T18:52:18.651358] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-04. [2025-10-20T18:52:19.254805] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:52:19.451365] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-06. [2025-10-20T18:52:20.240171] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T18:52:20.251318] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-07. [2025-10-20T18:52:20.423531] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:52:21.039825] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T18:52:21.051331] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-08. [2025-10-20T18:52:21.832287] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T18:52:21.851343] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-04. [2025-10-20T18:52:22.651388] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-03. [2025-10-20T18:52:23.368981] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T18:52:23.451334] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-05. [2025-10-20T18:52:23.560257] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T18:52:24.251406] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-04. [2025-10-20T18:52:24.926132] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T18:52:25.051363] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-09. [2025-10-20T18:52:25.432171] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T18:52:25.508543] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T18:52:25.851353] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-05. [2025-10-20T18:52:26.651429] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-06. [2025-10-20T18:52:26.954511] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T18:52:27.451375] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-06. [2025-10-20T18:52:28.224947] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T18:52:28.251341] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-07. [2025-10-20T18:52:28.928284] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:52:29.051337] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-07. [2025-10-20T18:52:29.851345] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-10. [2025-10-20T18:52:29.889987] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:52:30.413072] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T18:52:30.530169] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T18:52:30.532713] [INFO ] [REMOTE ] FTHY: Full fetch complete (17339 records). [2025-10-20T18:52:30.532784] [DEBUG] [VALIDATOR ] FTHY: Validating primary data. [2025-10-20T18:52:30.656454] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-08. [2025-10-20T18:52:30.897209] [DEBUG] [VALIDATOR ] FTHY: All expected trading days are present. [2025-10-20T18:52:30.897284] [DEBUG] [VALIDATOR ] FTHY: Primary data validation passed. [2025-10-20T18:52:30.897705] [INFO ] [MAIN ] FTHY: Fetched primary data is valid. [2025-10-20T18:52:30.897765] [DEBUG] [LOCAL ] FTHY: Saving primary data. [2025-10-20T18:52:30.898284] [DEBUG] [LOCAL ] FTHY: Trimmed 17339 records to 12520 (keeping 200 days of data) [2025-10-20T18:52:30.973804] [DEBUG] [LOCAL ] FTHY: Primary data saved (12520 records). [2025-10-20T18:52:30.973942] [INFO ] [MAIN ] FTHY: Full history saved. [2025-10-20T18:52:30.974022] [INFO ] [MAIN ] Update Progress: 877/2367 symbols completed. FTHY took 33.6s. Estimated remaining: 3h 11m 2s. Rate: 7.8 symbols/min [2025-10-20T18:52:31.451428] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-08. [2025-10-20T18:52:32.247841] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:52:32.256134] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-05. [2025-10-20T18:52:32.799307] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T18:52:33.051420] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-09. [2025-10-20T18:52:33.851375] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-09. [2025-10-20T18:52:34.095716] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T18:52:34.325381] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T18:52:34.651424] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-10. [2025-10-20T18:52:35.341284] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T18:52:35.451356] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-10. [2025-10-20T18:52:35.761559] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:52:35.768697] [INFO ] [REMOTE ] FTK: Full fetch complete (43913 records). [2025-10-20T18:52:35.768843] [DEBUG] [VALIDATOR ] FTK: Validating primary data. [2025-10-20T18:52:36.149439] [DEBUG] [VALIDATOR ] FTK: All expected trading days are present. [2025-10-20T18:52:36.149517] [DEBUG] [VALIDATOR ] FTK: Primary data validation passed. [2025-10-20T18:52:36.150202] [INFO ] [MAIN ] FTK: Fetched primary data is valid. [2025-10-20T18:52:36.150264] [DEBUG] [LOCAL ] FTK: Saving primary data. [2025-10-20T18:52:36.151194] [DEBUG] [LOCAL ] FTK: Trimmed 43913 records to 32881 (keeping 200 days of data) [2025-10-20T18:52:36.253692] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-06. [2025-10-20T18:52:36.338080] [DEBUG] [LOCAL ] FTK: Primary data saved (32881 records). [2025-10-20T18:52:36.338219] [INFO ] [MAIN ] FTK: Full history saved. [2025-10-20T18:52:36.338297] [INFO ] [MAIN ] Update Progress: 878/2367 symbols completed. FTK took 25.2s. Estimated remaining: 3h 10m 51s. Rate: 7.8 symbols/min [2025-10-20T18:52:37.122948] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T18:52:37.129346] [INFO ] [REMOTE ] FTS: Full fetch complete (60709 records). [2025-10-20T18:52:37.129462] [DEBUG] [VALIDATOR ] FTS: Validating primary data. [2025-10-20T18:52:37.511841] [DEBUG] [VALIDATOR ] FTS: All expected trading days are present. [2025-10-20T18:52:37.511943] [DEBUG] [VALIDATOR ] FTS: Primary data validation passed. [2025-10-20T18:52:37.512747] [INFO ] [MAIN ] FTS: Fetched primary data is valid. [2025-10-20T18:52:37.512808] [DEBUG] [LOCAL ] FTS: Saving primary data. [2025-10-20T18:52:37.513967] [DEBUG] [LOCAL ] FTS: Trimmed 60709 records to 42558 (keeping 200 days of data) [2025-10-20T18:52:37.775336] [DEBUG] [LOCAL ] FTS: Primary data saved (42558 records). [2025-10-20T18:52:37.775567] [INFO ] [MAIN ] FTS: Full history saved. [2025-10-20T18:52:37.775672] [INFO ] [MAIN ] Update Progress: 879/2367 symbols completed. FTS took 25.2s. Estimated remaining: 3h 10m 32s. Rate: 7.8 symbols/min [2025-10-20T18:52:38.068429] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T18:52:38.079463] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-07. [2025-10-20T18:52:40.035459] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T18:52:40.049349] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-08. [2025-10-20T18:52:41.908492] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T18:52:41.919946] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-09. [2025-10-20T18:52:43.517759] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T18:52:43.529189] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-10. [2025-10-20T18:52:45.240402] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T18:52:45.248250] [INFO ] [REMOTE ] FTI: Full fetch complete (77329 records). [2025-10-20T18:52:45.248398] [DEBUG] [VALIDATOR ] FTI: Validating primary data. [2025-10-20T18:52:45.640420] [DEBUG] [VALIDATOR ] FTI: All expected trading days are present. [2025-10-20T18:52:45.640525] [DEBUG] [VALIDATOR ] FTI: Primary data validation passed. [2025-10-20T18:52:45.641576] [INFO ] [MAIN ] FTI: Fetched primary data is valid. [2025-10-20T18:52:45.641668] [DEBUG] [LOCAL ] FTI: Saving primary data. [2025-10-20T18:52:45.643236] [DEBUG] [LOCAL ] FTI: Trimmed 77329 records to 53290 (keeping 200 days of data) [2025-10-20T18:52:45.959686] [DEBUG] [LOCAL ] FTI: Primary data saved (53290 records). [2025-10-20T18:52:45.959836] [INFO ] [MAIN ] FTI: Full history saved. [2025-10-20T18:52:45.960349] [INFO ] [MAIN ] Update Progress: 880/2367 symbols completed. FTI took 36.2s. Estimated remaining: 3h 10m 25s. Rate: 7.8 symbols/min [2025-10-20T18:52:46.629431] [INFO ] [MAIN ] FTWO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:46.693560] [INFO ] [MAIN ] FUL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:46.709087] [INFO ] [MAIN ] FUBO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:46.735321] [INFO ] [MAIN ] FTV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:46.739925] [DEBUG] [LOCAL ] FTWO: Loaded 1965 existing records from primary storage. [2025-10-20T18:52:46.740397] [INFO ] [MAIN ] FTWO: Loaded 1965 existing records. [2025-10-20T18:52:46.983423] [WARN ] [MAIN ] FTWO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:46.984072] [INFO ] [MAIN ] FTWO: Wiping existing primary data before full refresh. [2025-10-20T18:52:46.984124] [INFO ] [LOCAL ] FTWO: Starting deletion of primary data. [2025-10-20T18:52:46.984287] [DEBUG] [LOCAL ] FTWO: Deleted primary data file. [2025-10-20T18:52:46.984347] [INFO ] [MAIN ] FTWO: Starting full primary refresh. [2025-10-20T18:52:47.121028] [DEBUG] [LOCAL ] FUL: Loaded 32752 existing records from primary storage. [2025-10-20T18:52:47.126771] [INFO ] [MAIN ] FUL: Loaded 32752 existing records. [2025-10-20T18:52:47.127503] [DEBUG] [LOCAL ] FUBO: Loaded 54497 existing records from primary storage. [2025-10-20T18:52:47.167967] [DEBUG] [LOCAL ] FTV: Loaded 53827 existing records from primary storage. [2025-10-20T18:52:47.173335] [INFO ] [MAIN ] FUBO: Loaded 54497 existing records. [2025-10-20T18:52:47.179149] [INFO ] [MAIN ] FTV: Loaded 53827 existing records. [2025-10-20T18:52:47.807783] [WARN ] [MAIN ] FUL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:47.864625] [INFO ] [MAIN ] FUL: Wiping existing primary data before full refresh. [2025-10-20T18:52:48.025211] [INFO ] [LOCAL ] FUL: Starting deletion of primary data. [2025-10-20T18:52:48.008280] [WARN ] [MAIN ] FUBO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:48.004435] [WARN ] [MAIN ] FTV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:48.072021] [INFO ] [MAIN ] FUBO: Wiping existing primary data before full refresh. [2025-10-20T18:52:48.071868] [INFO ] [MAIN ] FTV: Wiping existing primary data before full refresh. [2025-10-20T18:52:48.072185] [INFO ] [LOCAL ] FTV: Starting deletion of primary data. [2025-10-20T18:52:48.071546] [INFO ] [REMOTE ] FTWO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:48.072287] [INFO ] [LOCAL ] FUBO: Starting deletion of primary data. [2025-10-20T18:52:48.072488] [DEBUG] [LOCAL ] FUL: Deleted primary data file. [2025-10-20T18:52:48.073137] [DEBUG] [LOCAL ] FUBO: Deleted primary data file. [2025-10-20T18:52:48.073005] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-01. [2025-10-20T18:52:48.073376] [INFO ] [MAIN ] FUL: Starting full primary refresh. [2025-10-20T18:52:48.073421] [INFO ] [MAIN ] FUBO: Starting full primary refresh. [2025-10-20T18:52:48.073469] [DEBUG] [LOCAL ] FTV: Deleted primary data file. [2025-10-20T18:52:48.104194] [INFO ] [MAIN ] FTV: Starting full primary refresh. [2025-10-20T18:52:48.961287] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T18:52:48.996707] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-02. [2025-10-20T18:52:49.170854] [INFO ] [REMOTE ] FUBO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:49.190861] [INFO ] [REMOTE ] FUL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:49.201566] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-01. [2025-10-20T18:52:49.202208] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-01. [2025-10-20T18:52:49.288296] [INFO ] [REMOTE ] FTV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:49.288698] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-01. [2025-10-20T18:52:49.648628] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T18:52:49.650275] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-03. [2025-10-20T18:52:50.092071] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-10-20T18:52:50.093309] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-04. [2025-10-20T18:52:50.526904] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T18:52:50.528875] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T18:52:50.531147] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-05. [2025-10-20T18:52:50.539963] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-02. [2025-10-20T18:52:50.922729] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:52:50.933246] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-02. [2025-10-20T18:52:50.947236] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T18:52:50.948393] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-06. [2025-10-20T18:52:51.456352] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T18:52:51.466492] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-03. [2025-10-20T18:52:51.636460] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-01 (2.43 seconds). [2025-10-20T18:52:51.649770] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T18:52:51.651264] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-02. [2025-10-20T18:52:51.651806] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-07. [2025-10-20T18:52:52.090455] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-10-20T18:52:52.091883] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-08. [2025-10-20T18:52:52.555441] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T18:52:52.556966] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-09. [2025-10-20T18:52:52.607569] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:52:52.868734] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-09 (0.31 seconds). [2025-10-20T18:52:52.988717] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:52:53.051456] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-03. [2025-10-20T18:52:53.412292] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T18:52:53.851366] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-03. [2025-10-20T18:52:54.631210] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T18:52:54.651351] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-04. [2025-10-20T18:52:55.451403] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-10. [2025-10-20T18:52:56.021672] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-03 (2.17 seconds). [2025-10-20T18:52:56.185927] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T18:52:56.187299] [INFO ] [REMOTE ] FTWO: Full fetch complete (3324 records). [2025-10-20T18:52:56.187539] [DEBUG] [VALIDATOR ] FTWO: Validating primary data. [2025-10-20T18:52:56.258320] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-04. [2025-10-20T18:52:56.323731] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:52:56.626300] [DEBUG] [VALIDATOR ] FTWO: All expected trading days are present. [2025-10-20T18:52:56.626408] [DEBUG] [VALIDATOR ] FTWO: Primary data validation passed. [2025-10-20T18:52:56.626590] [INFO ] [MAIN ] FTWO: Fetched primary data is valid. [2025-10-20T18:52:56.626680] [DEBUG] [LOCAL ] FTWO: Saving primary data. [2025-10-20T18:52:56.627080] [DEBUG] [LOCAL ] FTWO: Trimmed 3324 records to 1955 (keeping 200 days of data) [2025-10-20T18:52:56.639787] [DEBUG] [LOCAL ] FTWO: Primary data saved (1955 records). [2025-10-20T18:52:56.639849] [INFO ] [MAIN ] FTWO: Full history saved. [2025-10-20T18:52:56.639918] [INFO ] [MAIN ] Update Progress: 881/2367 symbols completed. FTWO took 10.7s. Estimated remaining: 3h 10m 23s. Rate: 7.8 symbols/min [2025-10-20T18:52:56.822013] [INFO ] [MAIN ] FUN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:56.870906] [DEBUG] [LOCAL ] FUN: Loaded 52679 existing records from primary storage. [2025-10-20T18:52:56.871239] [INFO ] [MAIN ] FUN: Loaded 52679 existing records. [2025-10-20T18:52:57.043476] [WARN ] [MAIN ] FUN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:57.043768] [INFO ] [MAIN ] FUN: Wiping existing primary data before full refresh. [2025-10-20T18:52:57.043821] [INFO ] [LOCAL ] FUN: Starting deletion of primary data. [2025-10-20T18:52:57.044015] [DEBUG] [LOCAL ] FUN: Deleted primary data file. [2025-10-20T18:52:57.044079] [INFO ] [MAIN ] FUN: Starting full primary refresh. [2025-10-20T18:52:57.056611] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-04. [2025-10-20T18:52:57.396218] [INFO ] [REMOTE ] FUN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:57.851375] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-01. [2025-10-20T18:52:58.493078] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T18:52:58.568581] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:52:58.651373] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-05. [2025-10-20T18:52:58.989751] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T18:52:59.451382] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-05. [2025-10-20T18:53:00.152267] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T18:53:00.251416] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-02. [2025-10-20T18:53:01.051445] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-06. [2025-10-20T18:53:01.852791] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-05. [2025-10-20T18:53:01.859124] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-05 (2.41 seconds). [2025-10-20T18:53:01.905027] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:53:02.606129] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T18:53:02.651369] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-07. [2025-10-20T18:53:03.240381] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T18:53:03.451435] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-06. [2025-10-20T18:53:04.251348] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-03. [2025-10-20T18:53:04.440377] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T18:53:05.051354] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-08. [2025-10-20T18:53:05.536155] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-06 (2.08 seconds). [2025-10-20T18:53:05.779930] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T18:53:05.851353] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-06. [2025-10-20T18:53:06.651346] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-07. [2025-10-20T18:53:06.868071] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T18:53:07.113573] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T18:53:07.451389] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-07. [2025-10-20T18:53:08.251341] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-04. [2025-10-20T18:53:08.691842] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:53:08.724102] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T18:53:09.051362] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-09. [2025-10-20T18:53:09.851358] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-08. [2025-10-20T18:53:09.924297] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:53:10.651349] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-08. [2025-10-20T18:53:10.814973] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T18:53:10.937423] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T18:53:11.451399] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-10. [2025-10-20T18:53:12.251374] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-09. [2025-10-20T18:53:12.981289] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-08 (2.33 seconds). [2025-10-20T18:53:13.051421] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-09. [2025-10-20T18:53:13.182180] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:53:13.190304] [INFO ] [REMOTE ] FTV: Full fetch complete (76640 records). [2025-10-20T18:53:13.190505] [DEBUG] [VALIDATOR ] FTV: Validating primary data. [2025-10-20T18:53:13.506023] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:53:13.623765] [DEBUG] [VALIDATOR ] FTV: All expected trading days are present. [2025-10-20T18:53:13.623859] [DEBUG] [VALIDATOR ] FTV: Primary data validation passed. [2025-10-20T18:53:13.624774] [INFO ] [MAIN ] FTV: Fetched primary data is valid. [2025-10-20T18:53:13.624848] [DEBUG] [LOCAL ] FTV: Saving primary data. [2025-10-20T18:53:13.626303] [DEBUG] [LOCAL ] FTV: Trimmed 76640 records to 53059 (keeping 200 days of data) [2025-10-20T18:53:13.857869] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-10. [2025-10-20T18:53:13.947034] [DEBUG] [LOCAL ] FTV: Primary data saved (53059 records). [2025-10-20T18:53:13.947177] [INFO ] [MAIN ] FTV: Full history saved. [2025-10-20T18:53:13.947257] [INFO ] [MAIN ] Update Progress: 882/2367 symbols completed. FTV took 28.0s. Estimated remaining: 3h 10m 31s. Rate: 7.8 symbols/min [2025-10-20T18:53:14.142829] [INFO ] [MAIN ] FVR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:14.168682] [DEBUG] [LOCAL ] FVR: Loaded 21415 existing records from primary storage. [2025-10-20T18:53:14.168941] [INFO ] [MAIN ] FVR: Loaded 21415 existing records. [2025-10-20T18:53:14.352546] [WARN ] [MAIN ] FVR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:14.352799] [INFO ] [MAIN ] FVR: Wiping existing primary data before full refresh. [2025-10-20T18:53:14.352850] [INFO ] [LOCAL ] FVR: Starting deletion of primary data. [2025-10-20T18:53:14.353053] [DEBUG] [LOCAL ] FVR: Deleted primary data file. [2025-10-20T18:53:14.353095] [INFO ] [MAIN ] FVR: Starting full primary refresh. [2025-10-20T18:53:14.656574] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-05. [2025-10-20T18:53:14.722139] [INFO ] [REMOTE ] FVR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:15.076308] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T18:53:15.170739] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T18:53:15.175986] [INFO ] [REMOTE ] FUL: Full fetch complete (47786 records). [2025-10-20T18:53:15.176162] [DEBUG] [VALIDATOR ] FUL: Validating primary data. [2025-10-20T18:53:15.456577] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-01. [2025-10-20T18:53:15.605192] [DEBUG] [VALIDATOR ] FUL: All expected trading days are present. [2025-10-20T18:53:15.605287] [DEBUG] [VALIDATOR ] FUL: Primary data validation passed. [2025-10-20T18:53:15.606033] [INFO ] [MAIN ] FUL: Fetched primary data is valid. [2025-10-20T18:53:15.606101] [DEBUG] [LOCAL ] FUL: Saving primary data. [2025-10-20T18:53:15.608122] [DEBUG] [LOCAL ] FUL: Trimmed 47786 records to 32097 (keeping 200 days of data) [2025-10-20T18:53:15.827297] [DEBUG] [LOCAL ] FUL: Primary data saved (32097 records). [2025-10-20T18:53:15.827516] [INFO ] [MAIN ] FUL: Full history saved. [2025-10-20T18:53:15.827623] [INFO ] [MAIN ] Update Progress: 883/2367 symbols completed. FUL took 29.9s. Estimated remaining: 3h 10m 14s. Rate: 7.8 symbols/min [2025-10-20T18:53:16.019499] [INFO ] [MAIN ] FVRR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:16.067208] [DEBUG] [LOCAL ] FVRR: Loaded 41187 existing records from primary storage. [2025-10-20T18:53:16.067696] [INFO ] [MAIN ] FVRR: Loaded 41187 existing records. [2025-10-20T18:53:16.256877] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-10. [2025-10-20T18:53:16.261372] [WARN ] [MAIN ] FVRR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:16.262823] [INFO ] [MAIN ] FVRR: Wiping existing primary data before full refresh. [2025-10-20T18:53:16.262880] [INFO ] [LOCAL ] FVRR: Starting deletion of primary data. [2025-10-20T18:53:16.263399] [DEBUG] [LOCAL ] FVRR: Deleted primary data file. [2025-10-20T18:53:16.263552] [INFO ] [MAIN ] FVRR: Starting full primary refresh. [2025-10-20T18:53:16.693189] [INFO ] [REMOTE ] FVRR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:16.694592] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:53:16.732173] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T18:53:17.051372] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-02. [2025-10-20T18:53:17.851433] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-06. [2025-10-20T18:53:17.967143] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T18:53:18.400719] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-10-20T18:53:18.408541] [INFO ] [REMOTE ] FUBO: Full fetch complete (77867 records). [2025-10-20T18:53:18.408720] [DEBUG] [VALIDATOR ] FUBO: Validating primary data. [2025-10-20T18:53:18.656535] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-03. [2025-10-20T18:53:18.805294] [DEBUG] [VALIDATOR ] FUBO: All expected trading days are present. [2025-10-20T18:53:18.805376] [DEBUG] [VALIDATOR ] FUBO: Primary data validation passed. [2025-10-20T18:53:18.806301] [INFO ] [MAIN ] FUBO: Fetched primary data is valid. [2025-10-20T18:53:18.806366] [DEBUG] [LOCAL ] FUBO: Saving primary data. [2025-10-20T18:53:18.807814] [DEBUG] [LOCAL ] FUBO: Trimmed 77867 records to 53718 (keeping 200 days of data) [2025-10-20T18:53:19.100944] [DEBUG] [LOCAL ] FUBO: Primary data saved (53718 records). [2025-10-20T18:53:19.101065] [INFO ] [MAIN ] FUBO: Full history saved. [2025-10-20T18:53:19.101141] [INFO ] [MAIN ] Update Progress: 884/2367 symbols completed. FUBO took 33.1s. Estimated remaining: 3h 9m 59s. Rate: 7.8 symbols/min [2025-10-20T18:53:19.272724] [INFO ] [MAIN ] FXED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:19.275455] [DEBUG] [LOCAL ] FXED: Loaded 1259 existing records from primary storage. [2025-10-20T18:53:19.275592] [INFO ] [MAIN ] FXED: Loaded 1259 existing records. [2025-10-20T18:53:19.453159] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-01. [2025-10-20T18:53:19.476738] [WARN ] [MAIN ] FXED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:19.478011] [INFO ] [MAIN ] FXED: Wiping existing primary data before full refresh. [2025-10-20T18:53:19.478463] [INFO ] [LOCAL ] FXED: Starting deletion of primary data. [2025-10-20T18:53:19.478860] [DEBUG] [LOCAL ] FXED: Deleted primary data file. [2025-10-20T18:53:19.479093] [INFO ] [MAIN ] FXED: Starting full primary refresh. [2025-10-20T18:53:19.791607] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:53:19.818435] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T18:53:19.920675] [INFO ] [REMOTE ] FXED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:20.251382] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-07. [2025-10-20T18:53:21.051413] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-01. [2025-10-20T18:53:21.091305] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T18:53:21.482486] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-01 (0.43 seconds). [2025-10-20T18:53:21.851386] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-02. [2025-10-20T18:53:22.032461] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T18:53:22.105320] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-02 (0.25 seconds). [2025-10-20T18:53:22.651369] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-03. [2025-10-20T18:53:22.998057] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T18:53:23.451373] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-04. [2025-10-20T18:53:24.251413] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-02. [2025-10-20T18:53:24.505193] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T18:53:25.051423] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-05. [2025-10-20T18:53:25.813264] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:53:25.851339] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-04. [2025-10-20T18:53:26.107208] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:53:26.268029] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T18:53:26.651386] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-03. [2025-10-20T18:53:27.451372] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-08. [2025-10-20T18:53:28.148074] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:53:28.251382] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-05. [2025-10-20T18:53:28.528778] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-10-20T18:53:29.051384] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-06. [2025-10-20T18:53:29.182626] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:53:29.401342] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-10-20T18:53:29.851406] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-04. [2025-10-20T18:53:30.651379] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-07. [2025-10-20T18:53:31.129591] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T18:53:31.456542] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-09. [2025-10-20T18:53:31.477178] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:53:32.251417] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-05. [2025-10-20T18:53:33.038925] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:53:33.051338] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-08. [2025-10-20T18:53:33.489689] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T18:53:33.727925] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:53:33.851371] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-06. [2025-10-20T18:53:34.651394] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-10. [2025-10-20T18:53:35.224340] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:53:35.451395] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-06. [2025-10-20T18:53:36.236461] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T18:53:36.244379] [INFO ] [REMOTE ] FUN: Full fetch complete (73439 records). [2025-10-20T18:53:36.244545] [DEBUG] [VALIDATOR ] FUN: Validating primary data. [2025-10-20T18:53:36.256590] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-09. [2025-10-20T18:53:36.676583] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T18:53:36.680380] [DEBUG] [VALIDATOR ] FUN: All expected trading days are present. [2025-10-20T18:53:36.686803] [DEBUG] [VALIDATOR ] FUN: Primary data validation passed. [2025-10-20T18:53:36.688008] [INFO ] [MAIN ] FUN: Fetched primary data is valid. [2025-10-20T18:53:36.688173] [DEBUG] [LOCAL ] FUN: Saving primary data. [2025-10-20T18:53:36.690435] [DEBUG] [LOCAL ] FUN: Trimmed 73439 records to 52012 (keeping 200 days of data) [2025-10-20T18:53:36.781939] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T18:53:36.997147] [DEBUG] [LOCAL ] FUN: Primary data saved (52012 records). [2025-10-20T18:53:36.997331] [INFO ] [MAIN ] FUN: Full history saved. [2025-10-20T18:53:36.998013] [INFO ] [MAIN ] Update Progress: 885/2367 symbols completed. FUN took 40.4s. Estimated remaining: 3h 10m 8s. Rate: 7.8 symbols/min [2025-10-20T18:53:37.060895] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-10. [2025-10-20T18:53:37.172056] [INFO ] [MAIN ] G: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:37.221112] [DEBUG] [LOCAL ] G: Loaded 50816 existing records from primary storage. [2025-10-20T18:53:37.221442] [INFO ] [MAIN ] G: Loaded 50816 existing records. [2025-10-20T18:53:37.398909] [WARN ] [MAIN ] G: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:37.399141] [INFO ] [MAIN ] G: Wiping existing primary data before full refresh. [2025-10-20T18:53:37.399191] [INFO ] [LOCAL ] G: Starting deletion of primary data. [2025-10-20T18:53:37.399400] [DEBUG] [LOCAL ] G: Deleted primary data file. [2025-10-20T18:53:37.399461] [INFO ] [MAIN ] G: Starting full primary refresh. [2025-10-20T18:53:37.578355] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T18:53:37.615291] [INFO ] [REMOTE ] FXED: Full fetch complete (1785 records). [2025-10-20T18:53:37.647271] [DEBUG] [VALIDATOR ] FXED: Validating primary data. [2025-10-20T18:53:37.857976] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-07. [2025-10-20T18:53:37.937816] [INFO ] [REMOTE ] G: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:38.177520] [DEBUG] [VALIDATOR ] FXED: All expected trading days are present. [2025-10-20T18:53:38.177601] [DEBUG] [VALIDATOR ] FXED: Primary data validation passed. [2025-10-20T18:53:38.177742] [INFO ] [MAIN ] FXED: Fetched primary data is valid. [2025-10-20T18:53:38.177817] [DEBUG] [LOCAL ] FXED: Saving primary data. [2025-10-20T18:53:38.178171] [DEBUG] [LOCAL ] FXED: Trimmed 1785 records to 1237 (keeping 200 days of data) [2025-10-20T18:53:38.186329] [DEBUG] [LOCAL ] FXED: Primary data saved (1237 records). [2025-10-20T18:53:38.186387] [INFO ] [MAIN ] FXED: Full history saved. [2025-10-20T18:53:38.186454] [INFO ] [MAIN ] Update Progress: 886/2367 symbols completed. FXED took 19.1s. Estimated remaining: 3h 9m 50s. Rate: 7.8 symbols/min [2025-10-20T18:53:38.381181] [INFO ] [MAIN ] GAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:38.409853] [DEBUG] [LOCAL ] GAB: Loaded 28162 existing records from primary storage. [2025-10-20T18:53:38.410148] [INFO ] [MAIN ] GAB: Loaded 28162 existing records. [2025-10-20T18:53:38.613447] [WARN ] [MAIN ] GAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:38.613759] [INFO ] [MAIN ] GAB: Wiping existing primary data before full refresh. [2025-10-20T18:53:38.613814] [INFO ] [LOCAL ] GAB: Starting deletion of primary data. [2025-10-20T18:53:38.614027] [DEBUG] [LOCAL ] GAB: Deleted primary data file. [2025-10-20T18:53:38.614083] [INFO ] [MAIN ] GAB: Starting full primary refresh. [2025-10-20T18:53:38.656489] [INFO ] [REMOTE ] G: Fetching primary data for 2025-01. [2025-10-20T18:53:39.034777] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T18:53:39.055133] [INFO ] [REMOTE ] GAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:39.451391] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-08. [2025-10-20T18:53:40.257249] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-01. [2025-10-20T18:53:40.274795] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T18:53:40.459330] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T18:53:41.051384] [INFO ] [REMOTE ] G: Fetching primary data for 2025-02. [2025-10-20T18:53:41.470961] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T18:53:41.851437] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-07. [2025-10-20T18:53:42.651415] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-02. [2025-10-20T18:53:42.715758] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:53:43.448544] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T18:53:43.457135] [INFO ] [REMOTE ] G: Fetching primary data for 2025-03. [2025-10-20T18:53:43.609475] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:53:44.251377] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-03. [2025-10-20T18:53:44.958491] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:53:45.051371] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-08. [2025-10-20T18:53:45.093626] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T18:53:45.851409] [INFO ] [REMOTE ] G: Fetching primary data for 2025-04. [2025-10-20T18:53:46.651392] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-09. [2025-10-20T18:53:46.778729] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:53:47.451384] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-09. [2025-10-20T18:53:47.513912] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T18:53:47.658689] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T18:53:48.251394] [INFO ] [REMOTE ] G: Fetching primary data for 2025-05. [2025-10-20T18:53:48.875602] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T18:53:49.051367] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-10. [2025-10-20T18:53:49.651331] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T18:53:49.851337] [INFO ] [REMOTE ] G: Fetching primary data for 2025-06. [2025-10-20T18:53:50.546191] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T18:53:50.555559] [INFO ] [REMOTE ] FVRR: Full fetch complete (59298 records). [2025-10-20T18:53:50.555770] [DEBUG] [VALIDATOR ] FVRR: Validating primary data. [2025-10-20T18:53:50.656634] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-04. [2025-10-20T18:53:50.972985] [DEBUG] [VALIDATOR ] FVRR: All expected trading days are present. [2025-10-20T18:53:50.973083] [DEBUG] [VALIDATOR ] FVRR: Primary data validation passed. [2025-10-20T18:53:50.973932] [INFO ] [MAIN ] FVRR: Fetched primary data is valid. [2025-10-20T18:53:50.973999] [DEBUG] [LOCAL ] FVRR: Saving primary data. [2025-10-20T18:53:50.975288] [DEBUG] [LOCAL ] FVRR: Trimmed 59298 records to 40712 (keeping 200 days of data) [2025-10-20T18:53:51.219273] [DEBUG] [LOCAL ] FVRR: Primary data saved (40712 records). [2025-10-20T18:53:51.219427] [INFO ] [MAIN ] FVRR: Full history saved. [2025-10-20T18:53:51.219941] [INFO ] [MAIN ] Update Progress: 887/2367 symbols completed. FVRR took 35.4s. Estimated remaining: 3h 9m 51s. Rate: 7.8 symbols/min [2025-10-20T18:53:51.400521] [INFO ] [MAIN ] GAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:51.407752] [DEBUG] [LOCAL ] GAM: Loaded 3971 existing records from primary storage. [2025-10-20T18:53:51.407966] [INFO ] [MAIN ] GAM: Loaded 3971 existing records. [2025-10-20T18:53:51.470122] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-10. [2025-10-20T18:53:51.630255] [WARN ] [MAIN ] GAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:51.648463] [INFO ] [MAIN ] GAM: Wiping existing primary data before full refresh. [2025-10-20T18:53:51.648621] [INFO ] [LOCAL ] GAM: Starting deletion of primary data. [2025-10-20T18:53:51.651880] [DEBUG] [LOCAL ] GAM: Deleted primary data file. [2025-10-20T18:53:51.652179] [INFO ] [MAIN ] GAM: Starting full primary refresh. [2025-10-20T18:53:51.659512] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T18:53:52.043395] [INFO ] [REMOTE ] GAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:52.063590] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T18:53:52.251435] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-01. [2025-10-20T18:53:52.611469] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T18:53:52.614391] [INFO ] [REMOTE ] FVR: Full fetch complete (29325 records). [2025-10-20T18:53:52.614469] [DEBUG] [VALIDATOR ] FVR: Validating primary data. [2025-10-20T18:53:52.749156] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T18:53:52.985192] [DEBUG] [VALIDATOR ] FVR: All expected trading days are present. [2025-10-20T18:53:52.985264] [DEBUG] [VALIDATOR ] FVR: Primary data validation passed. [2025-10-20T18:53:52.985815] [INFO ] [MAIN ] FVR: Fetched primary data is valid. [2025-10-20T18:53:52.985876] [DEBUG] [LOCAL ] FVR: Saving primary data. [2025-10-20T18:53:52.986510] [DEBUG] [LOCAL ] FVR: Trimmed 29325 records to 20901 (keeping 200 days of data) [2025-10-20T18:53:53.071593] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-05. [2025-10-20T18:53:53.113882] [DEBUG] [LOCAL ] FVR: Primary data saved (20901 records). [2025-10-20T18:53:53.113995] [INFO ] [MAIN ] FVR: Full history saved. [2025-10-20T18:53:53.114066] [INFO ] [MAIN ] Update Progress: 888/2367 symbols completed. FVR took 39.2s. Estimated remaining: 3h 9m 33s. Rate: 7.8 symbols/min [2025-10-20T18:53:53.278851] [INFO ] [MAIN ] GAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:53.329178] [DEBUG] [LOCAL ] GAP: Loaded 54433 existing records from primary storage. [2025-10-20T18:53:53.329501] [INFO ] [MAIN ] GAP: Loaded 54433 existing records. [2025-10-20T18:53:53.507018] [WARN ] [MAIN ] GAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:53.507320] [INFO ] [MAIN ] GAP: Wiping existing primary data before full refresh. [2025-10-20T18:53:53.507372] [INFO ] [LOCAL ] GAP: Starting deletion of primary data. [2025-10-20T18:53:53.507732] [DEBUG] [LOCAL ] GAP: Deleted primary data file. [2025-10-20T18:53:53.507795] [INFO ] [MAIN ] GAP: Starting full primary refresh. [2025-10-20T18:53:53.853316] [INFO ] [REMOTE ] G: Fetching primary data for 2025-07. [2025-10-20T18:53:53.888951] [INFO ] [REMOTE ] GAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:54.102018] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T18:53:54.651386] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-02. [2025-10-20T18:53:55.083818] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T18:53:55.181226] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:53:55.451421] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-06. [2025-10-20T18:53:56.251375] [INFO ] [REMOTE ] G: Fetching primary data for 2025-08. [2025-10-20T18:53:56.763159] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:53:57.051375] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-01. [2025-10-20T18:53:57.851384] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-07. [2025-10-20T18:53:57.923291] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T18:53:58.651462] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-03. [2025-10-20T18:53:58.762313] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T18:53:59.083506] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T18:53:59.357510] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T18:53:59.451396] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-08. [2025-10-20T18:54:00.251421] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-04. [2025-10-20T18:54:00.526445] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:54:01.037247] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T18:54:01.051369] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-05. [2025-10-20T18:54:01.624438] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T18:54:01.851384] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-06. [2025-10-20T18:54:02.467619] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T18:54:02.651371] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-07. [2025-10-20T18:54:03.195307] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-10-20T18:54:03.451427] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-02. [2025-10-20T18:54:04.251360] [INFO ] [REMOTE ] G: Fetching primary data for 2025-09. [2025-10-20T18:54:05.051432] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-08. [2025-10-20T18:54:05.083583] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:54:05.851405] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-09. [2025-10-20T18:54:05.908778] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:54:05.932523] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T18:54:06.651391] [INFO ] [REMOTE ] G: Fetching primary data for 2025-10. [2025-10-20T18:54:07.155354] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T18:54:07.451389] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-03. [2025-10-20T18:54:08.251457] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-09. [2025-10-20T18:54:08.491886] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:54:08.500230] [INFO ] [REMOTE ] G: Full fetch complete (72246 records). [2025-10-20T18:54:08.500404] [DEBUG] [VALIDATOR ] G: Validating primary data. [2025-10-20T18:54:08.786963] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-10-20T18:54:08.922597] [DEBUG] [VALIDATOR ] G: All expected trading days are present. [2025-10-20T18:54:08.922730] [DEBUG] [VALIDATOR ] G: Primary data validation passed. [2025-10-20T18:54:08.923638] [INFO ] [MAIN ] G: Fetched primary data is valid. [2025-10-20T18:54:08.923732] [DEBUG] [LOCAL ] G: Saving primary data. [2025-10-20T18:54:08.925156] [DEBUG] [LOCAL ] G: Trimmed 72246 records to 50114 (keeping 200 days of data) [2025-10-20T18:54:09.095985] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-10. [2025-10-20T18:54:09.226099] [DEBUG] [LOCAL ] G: Primary data saved (50114 records). [2025-10-20T18:54:09.226262] [INFO ] [MAIN ] G: Full history saved. [2025-10-20T18:54:09.227179] [INFO ] [MAIN ] Update Progress: 889/2367 symbols completed. G took 32.2s. Estimated remaining: 3h 9m 40s. Rate: 7.8 symbols/min [2025-10-20T18:54:09.400269] [INFO ] [MAIN ] GATX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:09.420343] [DEBUG] [LOCAL ] GATX: Loaded 18269 existing records from primary storage. [2025-10-20T18:54:09.420674] [INFO ] [MAIN ] GATX: Loaded 18269 existing records. [2025-10-20T18:54:09.588681] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T18:54:09.653461] [WARN ] [MAIN ] GATX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:09.653725] [INFO ] [MAIN ] GATX: Wiping existing primary data before full refresh. [2025-10-20T18:54:09.653798] [INFO ] [LOCAL ] GATX: Starting deletion of primary data. [2025-10-20T18:54:09.654296] [DEBUG] [LOCAL ] GATX: Deleted primary data file. [2025-10-20T18:54:09.654370] [INFO ] [MAIN ] GATX: Starting full primary refresh. [2025-10-20T18:54:09.863084] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-04. [2025-10-20T18:54:10.010844] [INFO ] [REMOTE ] GATX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:10.275758] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T18:54:10.280049] [INFO ] [REMOTE ] GAB: Full fetch complete (39400 records). [2025-10-20T18:54:10.280135] [DEBUG] [VALIDATOR ] GAB: Validating primary data. [2025-10-20T18:54:10.643049] [DEBUG] [VALIDATOR ] GAB: All expected trading days are present. [2025-10-20T18:54:10.643128] [DEBUG] [VALIDATOR ] GAB: Primary data validation passed. [2025-10-20T18:54:10.643768] [INFO ] [MAIN ] GAB: Fetched primary data is valid. [2025-10-20T18:54:10.643829] [DEBUG] [LOCAL ] GAB: Saving primary data. [2025-10-20T18:54:10.644631] [DEBUG] [LOCAL ] GAB: Trimmed 39400 records to 27744 (keeping 200 days of data) [2025-10-20T18:54:10.695576] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-10. [2025-10-20T18:54:10.811149] [DEBUG] [LOCAL ] GAB: Primary data saved (27744 records). [2025-10-20T18:54:10.811316] [INFO ] [MAIN ] GAB: Full history saved. [2025-10-20T18:54:10.811412] [INFO ] [MAIN ] Update Progress: 890/2367 symbols completed. GAB took 32.6s. Estimated remaining: 3h 9m 22s. Rate: 7.8 symbols/min [2025-10-20T18:54:10.983109] [INFO ] [MAIN ] GBAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:10.997839] [DEBUG] [LOCAL ] GBAB: Loaded 12974 existing records from primary storage. [2025-10-20T18:54:10.998067] [INFO ] [MAIN ] GBAB: Loaded 12974 existing records. [2025-10-20T18:54:11.173804] [WARN ] [MAIN ] GBAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:11.174002] [INFO ] [MAIN ] GBAB: Wiping existing primary data before full refresh. [2025-10-20T18:54:11.174047] [INFO ] [LOCAL ] GBAB: Starting deletion of primary data. [2025-10-20T18:54:11.174202] [DEBUG] [LOCAL ] GBAB: Deleted primary data file. [2025-10-20T18:54:11.174240] [INFO ] [MAIN ] GBAB: Starting full primary refresh. [2025-10-20T18:54:11.457372] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-01. [2025-10-20T18:54:11.450140] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T18:54:11.466097] [INFO ] [REMOTE ] GAM: Full fetch complete (6524 records). [2025-10-20T18:54:11.466189] [DEBUG] [VALIDATOR ] GAM: Validating primary data. [2025-10-20T18:54:11.652297] [INFO ] [REMOTE ] GBAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:11.926371] [DEBUG] [VALIDATOR ] GAM: All expected trading days are present. [2025-10-20T18:54:11.926538] [DEBUG] [VALIDATOR ] GAM: Primary data validation passed. [2025-10-20T18:54:11.926754] [INFO ] [MAIN ] GAM: Fetched primary data is valid. [2025-10-20T18:54:11.926828] [DEBUG] [LOCAL ] GAM: Saving primary data. [2025-10-20T18:54:11.927398] [DEBUG] [LOCAL ] GAM: Trimmed 6524 records to 3901 (keeping 200 days of data) [2025-10-20T18:54:11.979792] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T18:54:11.978462] [DEBUG] [LOCAL ] GAM: Primary data saved (3901 records). [2025-10-20T18:54:11.990346] [INFO ] [MAIN ] GAM: Full history saved. [2025-10-20T18:54:11.990827] [INFO ] [MAIN ] Update Progress: 891/2367 symbols completed. GAM took 20.8s. Estimated remaining: 3h 9m 3s. Rate: 7.8 symbols/min [2025-10-20T18:54:12.163572] [INFO ] [MAIN ] GBCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:12.199634] [DEBUG] [LOCAL ] GBCI: Loaded 41621 existing records from primary storage. [2025-10-20T18:54:12.199972] [INFO ] [MAIN ] GBCI: Loaded 41621 existing records. [2025-10-20T18:54:12.262024] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-01. [2025-10-20T18:54:12.379796] [WARN ] [MAIN ] GBCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:12.380062] [INFO ] [MAIN ] GBCI: Wiping existing primary data before full refresh. [2025-10-20T18:54:12.380184] [INFO ] [LOCAL ] GBCI: Starting deletion of primary data. [2025-10-20T18:54:12.389394] [DEBUG] [LOCAL ] GBCI: Deleted primary data file. [2025-10-20T18:54:12.389458] [INFO ] [MAIN ] GBCI: Starting full primary refresh. [2025-10-20T18:54:12.389764] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T18:54:12.747014] [INFO ] [REMOTE ] GBCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:13.051398] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-01. [2025-10-20T18:54:13.102365] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T18:54:13.851397] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-02. [2025-10-20T18:54:14.547144] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:54:14.651452] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-02. [2025-10-20T18:54:14.758890] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T18:54:15.451380] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-02. [2025-10-20T18:54:15.932790] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T18:54:16.251404] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-03. [2025-10-20T18:54:16.351863] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T18:54:17.051430] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-05. [2025-10-20T18:54:17.717759] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:54:17.851386] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-03. [2025-10-20T18:54:18.651394] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-04. [2025-10-20T18:54:18.860506] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T18:54:18.986280] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T18:54:19.451416] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-03. [2025-10-20T18:54:20.092458] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:54:20.120598] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T18:54:20.251416] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-06. [2025-10-20T18:54:21.051405] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-04. [2025-10-20T18:54:21.851370] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-05. [2025-10-20T18:54:22.227618] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-10-20T18:54:22.289684] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T18:54:22.651408] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-07. [2025-10-20T18:54:23.248759] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T18:54:23.451427] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-05. [2025-10-20T18:54:24.251351] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-04. [2025-10-20T18:54:24.459736] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:54:24.493225] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:54:24.865231] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:54:25.051422] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-05. [2025-10-20T18:54:25.839385] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T18:54:25.851369] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-06. [2025-10-20T18:54:26.651415] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-08. [2025-10-20T18:54:26.851515] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T18:54:27.451425] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-06. [2025-10-20T18:54:28.251403] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-06. [2025-10-20T18:54:28.559105] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T18:54:28.958286] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T18:54:29.051377] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-09. [2025-10-20T18:54:29.380958] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T18:54:29.851378] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-07. [2025-10-20T18:54:30.651374] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-07. [2025-10-20T18:54:30.836868] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T18:54:31.456525] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-10. [2025-10-20T18:54:31.471219] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T18:54:31.642888] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T18:54:32.251383] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-08. [2025-10-20T18:54:33.051372] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-07. [2025-10-20T18:54:33.291616] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:54:33.302739] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T18:54:33.307274] [INFO ] [REMOTE ] GAP: Full fetch complete (77812 records). [2025-10-20T18:54:33.307401] [DEBUG] [VALIDATOR ] GAP: Validating primary data. [2025-10-20T18:54:33.743276] [DEBUG] [VALIDATOR ] GAP: All expected trading days are present. [2025-10-20T18:54:33.743364] [DEBUG] [VALIDATOR ] GAP: Primary data validation passed. [2025-10-20T18:54:33.744364] [INFO ] [MAIN ] GAP: Fetched primary data is valid. [2025-10-20T18:54:33.744428] [DEBUG] [LOCAL ] GAP: Saving primary data. [2025-10-20T18:54:33.745933] [DEBUG] [LOCAL ] GAP: Trimmed 77812 records to 53653 (keeping 200 days of data) [2025-10-20T18:54:33.872169] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-08. [2025-10-20T18:54:34.096213] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T18:54:34.105399] [DEBUG] [LOCAL ] GAP: Primary data saved (53653 records). [2025-10-20T18:54:34.109285] [INFO ] [MAIN ] GAP: Full history saved. [2025-10-20T18:54:34.109952] [INFO ] [MAIN ] Update Progress: 892/2367 symbols completed. GAP took 41.0s. Estimated remaining: 3h 9m 20s. Rate: 7.8 symbols/min [2025-10-20T18:54:34.280183] [INFO ] [MAIN ] GBLI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:34.280408] [DEBUG] [LOCAL ] GBLI: No existing data file found in primary storage. [2025-10-20T18:54:34.280449] [WARN ] [MAIN ] GBLI: No existing data found, performing full refresh instead. [2025-10-20T18:54:34.280523] [INFO ] [MAIN ] GBLI: Wiping existing primary data before full refresh. [2025-10-20T18:54:34.280558] [INFO ] [LOCAL ] GBLI: Starting deletion of primary data. [2025-10-20T18:54:34.280597] [DEBUG] [LOCAL ] GBLI: No primary data file found to delete. [2025-10-20T18:54:34.280627] [INFO ] [MAIN ] GBLI: Starting full primary refresh. [2025-10-20T18:54:34.654977] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-08. [2025-10-20T18:54:34.658864] [INFO ] [REMOTE ] GBLI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:35.456587] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-09. [2025-10-20T18:54:35.486482] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T18:54:35.493608] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T18:54:36.251388] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-01. [2025-10-20T18:54:36.523861] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-01 (0.27 seconds). [2025-10-20T18:54:36.542797] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T18:54:37.051395] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-10. [2025-10-20T18:54:37.720714] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T18:54:37.723105] [INFO ] [REMOTE ] GATX: Full fetch complete (24864 records). [2025-10-20T18:54:37.723179] [DEBUG] [VALIDATOR ] GATX: Validating primary data. [2025-10-20T18:54:37.865803] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-02. [2025-10-20T18:54:38.093966] [DEBUG] [VALIDATOR ] GATX: All expected trading days are present. [2025-10-20T18:54:38.094045] [DEBUG] [VALIDATOR ] GATX: Primary data validation passed. [2025-10-20T18:54:38.094549] [INFO ] [MAIN ] GATX: Fetched primary data is valid. [2025-10-20T18:54:38.094588] [DEBUG] [LOCAL ] GATX: Saving primary data. [2025-10-20T18:54:38.095246] [DEBUG] [LOCAL ] GATX: Trimmed 24864 records to 18106 (keeping 200 days of data) [2025-10-20T18:54:38.183254] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T18:54:38.233384] [DEBUG] [LOCAL ] GATX: Primary data saved (18106 records). [2025-10-20T18:54:38.233555] [INFO ] [MAIN ] GATX: Full history saved. [2025-10-20T18:54:38.233679] [INFO ] [MAIN ] Update Progress: 893/2367 symbols completed. GATX took 29.0s. Estimated remaining: 3h 9m 6s. Rate: 7.8 symbols/min [2025-10-20T18:54:38.442932] [INFO ] [MAIN ] GBTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:38.488496] [DEBUG] [LOCAL ] GBTG: Loaded 40850 existing records from primary storage. [2025-10-20T18:54:38.489003] [INFO ] [MAIN ] GBTG: Loaded 40850 existing records. [2025-10-20T18:54:38.656516] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-03. [2025-10-20T18:54:38.723249] [WARN ] [MAIN ] GBTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:38.723716] [INFO ] [MAIN ] GBTG: Wiping existing primary data before full refresh. [2025-10-20T18:54:38.723791] [INFO ] [LOCAL ] GBTG: Starting deletion of primary data. [2025-10-20T18:54:38.724114] [DEBUG] [LOCAL ] GBTG: Deleted primary data file. [2025-10-20T18:54:38.724172] [INFO ] [MAIN ] GBTG: Starting full primary refresh. [2025-10-20T18:54:39.125472] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T18:54:39.253001] [INFO ] [REMOTE ] GBTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:39.451382] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-04. [2025-10-20T18:54:39.791939] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-10-20T18:54:40.251428] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-05. [2025-10-20T18:54:40.596465] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T18:54:41.051372] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-09. [2025-10-20T18:54:41.851394] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-09. [2025-10-20T18:54:42.395810] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T18:54:42.651346] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-01. [2025-10-20T18:54:42.685523] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:54:43.451347] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-10. [2025-10-20T18:54:43.906104] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T18:54:44.251373] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-02. [2025-10-20T18:54:45.051368] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-06. [2025-10-20T18:54:45.205132] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T18:54:45.212963] [INFO ] [REMOTE ] GBCI: Full fetch complete (59309 records). [2025-10-20T18:54:45.213119] [DEBUG] [VALIDATOR ] GBCI: Validating primary data. [2025-10-20T18:54:45.609334] [DEBUG] [VALIDATOR ] GBCI: All expected trading days are present. [2025-10-20T18:54:45.588996] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:54:45.624275] [DEBUG] [VALIDATOR ] GBCI: Primary data validation passed. [2025-10-20T18:54:45.628525] [INFO ] [MAIN ] GBCI: Fetched primary data is valid. [2025-10-20T18:54:45.628612] [DEBUG] [LOCAL ] GBCI: Saving primary data. [2025-10-20T18:54:45.630215] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:54:45.637612] [DEBUG] [LOCAL ] GBCI: Trimmed 59309 records to 41215 (keeping 200 days of data) [2025-10-20T18:54:45.851523] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-03. [2025-10-20T18:54:45.890935] [DEBUG] [LOCAL ] GBCI: Primary data saved (41215 records). [2025-10-20T18:54:45.891389] [INFO ] [MAIN ] GBCI: Full history saved. [2025-10-20T18:54:45.892122] [INFO ] [MAIN ] Update Progress: 894/2367 symbols completed. GBCI took 33.9s. Estimated remaining: 3h 8m 58s. Rate: 7.8 symbols/min [2025-10-20T18:54:46.059972] [INFO ] [MAIN ] GBX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:46.091210] [DEBUG] [LOCAL ] GBX: Loaded 29290 existing records from primary storage. [2025-10-20T18:54:46.091453] [INFO ] [MAIN ] GBX: Loaded 29290 existing records. [2025-10-20T18:54:46.293279] [WARN ] [MAIN ] GBX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:46.293595] [INFO ] [MAIN ] GBX: Wiping existing primary data before full refresh. [2025-10-20T18:54:46.293661] [INFO ] [LOCAL ] GBX: Starting deletion of primary data. [2025-10-20T18:54:46.294177] [DEBUG] [LOCAL ] GBX: Deleted primary data file. [2025-10-20T18:54:46.294233] [INFO ] [MAIN ] GBX: Starting full primary refresh. [2025-10-20T18:54:46.649580] [INFO ] [REMOTE ] GBX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:46.651327] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-01. [2025-10-20T18:54:47.187183] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:54:47.451372] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-04. [2025-10-20T18:54:47.609186] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T18:54:48.251363] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-07. [2025-10-20T18:54:48.590355] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T18:54:48.906262] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T18:54:49.051357] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-08. [2025-10-20T18:54:49.390226] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:54:49.851366] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-09. [2025-10-20T18:54:50.169461] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-10-20T18:54:50.651369] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-02. [2025-10-20T18:54:51.451381] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-10. [2025-10-20T18:54:51.568318] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T18:54:51.910318] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T18:54:51.911050] [INFO ] [REMOTE ] GBLI: Full fetch complete (1276 records). [2025-10-20T18:54:51.911117] [DEBUG] [VALIDATOR ] GBLI: Validating primary data. [2025-10-20T18:54:52.104726] [WARN ] [VALIDATOR ] GBLI: Validation failed - insufficient historical data. Have 173 unique days, need 200 days [2025-10-20T18:54:52.105020] [FATAL] [MAIN ] GBLI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:54:52.105094] [WARN ] [MAIN ] GBLI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:54:52.105168] [INFO ] [LOCAL ] GBLI: Starting deletion of primary data. [2025-10-20T18:54:52.105264] [DEBUG] [LOCAL ] GBLI: No primary data file found to delete. [2025-10-20T18:54:52.105321] [INFO ] [LOCAL ] GBLI: Starting deletion of derivative data. [2025-10-20T18:54:52.105365] [DEBUG] [LOCAL ] GBLI: No derivative data file found to delete. [2025-10-20T18:54:52.105421] [INFO ] [MAIN ] Update Progress: 895/2367 symbols completed. GBLI took 18.0s. Estimated remaining: 3h 8m 48s. Rate: 7.8 symbols/min [2025-10-20T18:54:52.256534] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-03. [2025-10-20T18:54:52.303050] [INFO ] [MAIN ] GCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:52.345193] [DEBUG] [LOCAL ] GCI: Loaded 44504 existing records from primary storage. [2025-10-20T18:54:52.345507] [INFO ] [MAIN ] GCI: Loaded 44504 existing records. [2025-10-20T18:54:52.535801] [WARN ] [MAIN ] GCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:52.536039] [INFO ] [MAIN ] GCI: Wiping existing primary data before full refresh. [2025-10-20T18:54:52.536087] [INFO ] [LOCAL ] GCI: Starting deletion of primary data. [2025-10-20T18:54:52.536328] [DEBUG] [LOCAL ] GCI: Deleted primary data file. [2025-10-20T18:54:52.536405] [INFO ] [MAIN ] GCI: Starting full primary refresh. [2025-10-20T18:54:52.887916] [INFO ] [REMOTE ] GCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:53.051355] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-05. [2025-10-20T18:54:53.521638] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T18:54:53.851404] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-10. [2025-10-20T18:54:54.585003] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T18:54:54.651400] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-04. [2025-10-20T18:54:54.690030] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:54:54.691992] [INFO ] [REMOTE ] GBAB: Full fetch complete (18819 records). [2025-10-20T18:54:54.692060] [DEBUG] [VALIDATOR ] GBAB: Validating primary data. [2025-10-20T18:54:55.053591] [DEBUG] [VALIDATOR ] GBAB: All expected trading days are present. [2025-10-20T18:54:55.053690] [DEBUG] [VALIDATOR ] GBAB: Primary data validation passed. [2025-10-20T18:54:55.054114] [INFO ] [MAIN ] GBAB: Fetched primary data is valid. [2025-10-20T18:54:55.054154] [DEBUG] [LOCAL ] GBAB: Saving primary data. [2025-10-20T18:54:55.054695] [DEBUG] [LOCAL ] GBAB: Trimmed 18819 records to 12847 (keeping 200 days of data) [2025-10-20T18:54:55.131626] [DEBUG] [LOCAL ] GBAB: Primary data saved (12847 records). [2025-10-20T18:54:55.131785] [INFO ] [MAIN ] GBAB: Full history saved. [2025-10-20T18:54:55.131861] [INFO ] [MAIN ] Update Progress: 896/2367 symbols completed. GBAB took 44.3s. Estimated remaining: 3h 8m 33s. Rate: 7.8 symbols/min [2025-10-20T18:54:55.302510] [INFO ] [MAIN ] GCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:55.325245] [DEBUG] [LOCAL ] GCO: Loaded 23845 existing records from primary storage. [2025-10-20T18:54:55.325506] [INFO ] [MAIN ] GCO: Loaded 23845 existing records. [2025-10-20T18:54:55.456479] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-01. [2025-10-20T18:54:55.508227] [WARN ] [MAIN ] GCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:55.508429] [INFO ] [MAIN ] GCO: Wiping existing primary data before full refresh. [2025-10-20T18:54:55.508476] [INFO ] [LOCAL ] GCO: Starting deletion of primary data. [2025-10-20T18:54:55.508637] [DEBUG] [LOCAL ] GCO: Deleted primary data file. [2025-10-20T18:54:55.508717] [INFO ] [MAIN ] GCO: Starting full primary refresh. [2025-10-20T18:54:55.875438] [INFO ] [REMOTE ] GCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:56.006962] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T18:54:56.251381] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-01. [2025-10-20T18:54:56.905465] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:54:57.051396] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-05. [2025-10-20T18:54:57.313458] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T18:54:57.851431] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-06. [2025-10-20T18:54:58.228931] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:54:58.651393] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-06. [2025-10-20T18:54:59.283864] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T18:54:59.343326] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T18:54:59.451445] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-07. [2025-10-20T18:55:00.251429] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-02. [2025-10-20T18:55:01.051415] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-07. [2025-10-20T18:55:01.417382] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T18:55:01.851403] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-08. [2025-10-20T18:55:02.142114] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T18:55:02.658465] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-02. [2025-10-20T18:55:02.676062] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T18:55:03.320168] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T18:55:03.451375] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-09. [2025-10-20T18:55:03.781991] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T18:55:04.251450] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-03. [2025-10-20T18:55:04.732455] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:55:05.051410] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-10. [2025-10-20T18:55:05.547431] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T18:55:05.851449] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-04. [2025-10-20T18:55:05.973021] [DEBUG] [REMOTE ] GCO: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T18:55:05.973331] [WARN ] [REMOTE ] GCO: No data returned for month 2025-04 [2025-10-20T18:55:05.973420] [FATAL] [REMOTE ] GCO: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:55:05.973495] [FATAL] [MAIN ] GCO: Failed to fetch primary data. [2025-10-20T18:55:05.973556] [WARN ] [MAIN ] GCO: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:55:05.973627] [INFO ] [LOCAL ] GCO: Starting deletion of primary data. [2025-10-20T18:55:05.973726] [DEBUG] [LOCAL ] GCO: No primary data file found to delete. [2025-10-20T18:55:05.973770] [INFO ] [LOCAL ] GCO: Starting deletion of derivative data. [2025-10-20T18:55:05.973917] [DEBUG] [LOCAL ] GCO: Deleted derivative data file. [2025-10-20T18:55:05.973999] [INFO ] [MAIN ] Update Progress: 897/2367 symbols completed. GCO took 10.8s. Estimated remaining: 3h 8m 30s. Rate: 7.8 symbols/min [2025-10-20T18:55:06.168110] [INFO ] [MAIN ] GCTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:06.191231] [DEBUG] [LOCAL ] GCTS: Loaded 20642 existing records from primary storage. [2025-10-20T18:55:06.191562] [INFO ] [MAIN ] GCTS: Loaded 20642 existing records. [2025-10-20T18:55:06.434813] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T18:55:06.439431] [WARN ] [MAIN ] GCTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:06.445596] [INFO ] [MAIN ] GCTS: Wiping existing primary data before full refresh. [2025-10-20T18:55:06.445998] [INFO ] [LOCAL ] GCTS: Starting deletion of primary data. [2025-10-20T18:55:06.446598] [DEBUG] [LOCAL ] GCTS: Deleted primary data file. [2025-10-20T18:55:06.446685] [INFO ] [MAIN ] GCTS: Starting full primary refresh. [2025-10-20T18:55:06.489476] [INFO ] [REMOTE ] GBTG: Full fetch complete (56818 records). [2025-10-20T18:55:06.510244] [DEBUG] [VALIDATOR ] GBTG: Validating primary data. [2025-10-20T18:55:06.661145] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-03. [2025-10-20T18:55:07.118363] [INFO ] [REMOTE ] GCTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:07.207489] [DEBUG] [VALIDATOR ] GBTG: All expected trading days are present. [2025-10-20T18:55:07.207579] [DEBUG] [VALIDATOR ] GBTG: Primary data validation passed. [2025-10-20T18:55:07.208400] [INFO ] [MAIN ] GBTG: Fetched primary data is valid. [2025-10-20T18:55:07.208463] [DEBUG] [LOCAL ] GBTG: Saving primary data. [2025-10-20T18:55:07.209826] [DEBUG] [LOCAL ] GBTG: Trimmed 56818 records to 40310 (keeping 200 days of data) [2025-10-20T18:55:07.435194] [DEBUG] [LOCAL ] GBTG: Primary data saved (40310 records). [2025-10-20T18:55:07.435336] [INFO ] [MAIN ] GBTG: Full history saved. [2025-10-20T18:55:07.435427] [INFO ] [MAIN ] Update Progress: 898/2367 symbols completed. GBTG took 29.2s. Estimated remaining: 3h 8m 12s. Rate: 7.8 symbols/min [2025-10-20T18:55:07.456560] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-01. [2025-10-20T18:55:07.606242] [INFO ] [MAIN ] GCV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:07.616313] [DEBUG] [LOCAL ] GCV: Loaded 7903 existing records from primary storage. [2025-10-20T18:55:07.616536] [INFO ] [MAIN ] GCV: Loaded 7903 existing records. [2025-10-20T18:55:07.792807] [WARN ] [MAIN ] GCV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:07.793013] [INFO ] [MAIN ] GCV: Wiping existing primary data before full refresh. [2025-10-20T18:55:07.793062] [INFO ] [LOCAL ] GCV: Starting deletion of primary data. [2025-10-20T18:55:07.793353] [DEBUG] [LOCAL ] GCV: Deleted primary data file. [2025-10-20T18:55:07.793409] [INFO ] [MAIN ] GCV: Starting full primary refresh. [2025-10-20T18:55:08.153060] [INFO ] [REMOTE ] GCV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:08.245839] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T18:55:08.255521] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-08. [2025-10-20T18:55:08.497730] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:55:09.051408] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-04. [2025-10-20T18:55:09.474835] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-10-20T18:55:09.851400] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-09. [2025-10-20T18:55:10.659836] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-01. [2025-10-20T18:55:10.678811] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:55:10.940072] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T18:55:11.260873] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T18:55:11.451386] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-10. [2025-10-20T18:55:12.251407] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-05. [2025-10-20T18:55:12.710174] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:55:12.714450] [INFO ] [REMOTE ] GBX: Full fetch complete (42830 records). [2025-10-20T18:55:12.714593] [DEBUG] [VALIDATOR ] GBX: Validating primary data. [2025-10-20T18:55:13.056538] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-02. [2025-10-20T18:55:13.169469] [DEBUG] [VALIDATOR ] GBX: All expected trading days are present. [2025-10-20T18:55:13.169566] [DEBUG] [VALIDATOR ] GBX: Primary data validation passed. [2025-10-20T18:55:13.170310] [INFO ] [MAIN ] GBX: Fetched primary data is valid. [2025-10-20T18:55:13.170378] [DEBUG] [LOCAL ] GBX: Saving primary data. [2025-10-20T18:55:13.171634] [DEBUG] [LOCAL ] GBX: Trimmed 42830 records to 28567 (keeping 200 days of data) [2025-10-20T18:55:13.381395] [DEBUG] [LOCAL ] GBX: Primary data saved (28567 records). [2025-10-20T18:55:13.381541] [INFO ] [MAIN ] GBX: Full history saved. [2025-10-20T18:55:13.382141] [INFO ] [MAIN ] Update Progress: 899/2367 symbols completed. GBX took 27.5s. Estimated remaining: 3h 8m 2s. Rate: 7.8 symbols/min [2025-10-20T18:55:13.574373] [INFO ] [MAIN ] GD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:13.623418] [DEBUG] [LOCAL ] GD: Loaded 47354 existing records from primary storage. [2025-10-20T18:55:13.623753] [INFO ] [MAIN ] GD: Loaded 47354 existing records. [2025-10-20T18:55:13.716795] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T18:55:13.852316] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-02. [2025-10-20T18:55:13.857203] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T18:55:13.890417] [WARN ] [MAIN ] GD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:13.890888] [INFO ] [MAIN ] GD: Wiping existing primary data before full refresh. [2025-10-20T18:55:13.891721] [INFO ] [LOCAL ] GD: Starting deletion of primary data. [2025-10-20T18:55:13.892103] [DEBUG] [LOCAL ] GD: Deleted primary data file. [2025-10-20T18:55:13.892175] [INFO ] [MAIN ] GD: Starting full primary refresh. [2025-10-20T18:55:14.282203] [INFO ] [REMOTE ] GD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:14.651426] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-06. [2025-10-20T18:55:14.787264] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T18:55:15.451459] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-01. [2025-10-20T18:55:16.261144] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-03. [2025-10-20T18:55:16.264305] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:55:17.051446] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-07. [2025-10-20T18:55:17.151067] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T18:55:17.295788] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:55:17.851417] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-03. [2025-10-20T18:55:18.360520] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T18:55:18.458155] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T18:55:18.651366] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-02. [2025-10-20T18:55:19.451378] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-04. [2025-10-20T18:55:20.052465] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T18:55:20.251396] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-04. [2025-10-20T18:55:20.388273] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T18:55:20.833940] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T18:55:21.051416] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-05. [2025-10-20T18:55:21.827574] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T18:55:21.851365] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-03. [2025-10-20T18:55:22.651422] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-06. [2025-10-20T18:55:23.243969] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T18:55:23.451346] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-07. [2025-10-20T18:55:23.646759] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T18:55:24.185574] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:55:24.251414] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-04. [2025-10-20T18:55:25.051423] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-08. [2025-10-20T18:55:25.851400] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-05. [2025-10-20T18:55:25.993776] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T18:55:26.651366] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-08. [2025-10-20T18:55:26.776136] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:55:27.059902] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T18:55:27.345465] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T18:55:27.451377] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-09. [2025-10-20T18:55:28.251433] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-09. [2025-10-20T18:55:28.998951] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T18:55:29.040395] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:55:29.051396] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-06. [2025-10-20T18:55:29.851377] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-10. [2025-10-20T18:55:30.447952] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:55:30.651388] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-10. [2025-10-20T18:55:31.221931] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:55:31.228304] [INFO ] [REMOTE ] GCI: Full fetch complete (63421 records). [2025-10-20T18:55:31.228435] [DEBUG] [VALIDATOR ] GCI: Validating primary data. [2025-10-20T18:55:31.457262] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-05. [2025-10-20T18:55:31.613612] [DEBUG] [VALIDATOR ] GCI: All expected trading days are present. [2025-10-20T18:55:31.613733] [DEBUG] [VALIDATOR ] GCI: Primary data validation passed. [2025-10-20T18:55:31.614550] [INFO ] [MAIN ] GCI: Fetched primary data is valid. [2025-10-20T18:55:31.614688] [DEBUG] [LOCAL ] GCI: Saving primary data. [2025-10-20T18:55:31.615928] [DEBUG] [LOCAL ] GCI: Trimmed 63421 records to 43764 (keeping 200 days of data) [2025-10-20T18:55:31.708690] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:55:31.713970] [INFO ] [REMOTE ] GCV: Full fetch complete (9865 records). [2025-10-20T18:55:31.715085] [DEBUG] [VALIDATOR ] GCV: Validating primary data. [2025-10-20T18:55:32.185156] [DEBUG] [LOCAL ] GCI: Primary data saved (43764 records). [2025-10-20T18:55:32.202398] [INFO ] [MAIN ] GCI: Full history saved. [2025-10-20T18:55:32.203824] [INFO ] [MAIN ] Update Progress: 900/2367 symbols completed. GCI took 40.1s. Estimated remaining: 3h 8m 12s. Rate: 7.8 symbols/min [2025-10-20T18:55:32.268476] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-07. [2025-10-20T18:55:32.289221] [DEBUG] [VALIDATOR ] GCV: All expected trading days are present. [2025-10-20T18:55:32.311275] [DEBUG] [VALIDATOR ] GCV: Primary data validation passed. [2025-10-20T18:55:32.325795] [INFO ] [MAIN ] GCV: Fetched primary data is valid. [2025-10-20T18:55:32.325892] [DEBUG] [LOCAL ] GCV: Saving primary data. [2025-10-20T18:55:32.330153] [DEBUG] [LOCAL ] GCV: Trimmed 9865 records to 7831 (keeping 200 days of data) [2025-10-20T18:55:32.445277] [INFO ] [MAIN ] GDDY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:32.451724] [DEBUG] [LOCAL ] GCV: Primary data saved (7831 records). [2025-10-20T18:55:32.452171] [INFO ] [MAIN ] GCV: Full history saved. [2025-10-20T18:55:32.452292] [INFO ] [MAIN ] Update Progress: 901/2367 symbols completed. GCV took 25.0s. Estimated remaining: 3h 7m 52s. Rate: 7.8 symbols/min [2025-10-20T18:55:32.662792] [INFO ] [MAIN ] GDIV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:32.685187] [DEBUG] [LOCAL ] GDIV: Loaded 3630 existing records from primary storage. [2025-10-20T18:55:32.682256] [DEBUG] [LOCAL ] GDDY: Loaded 50516 existing records from primary storage. [2025-10-20T18:55:32.685509] [INFO ] [MAIN ] GDIV: Loaded 3630 existing records. [2025-10-20T18:55:32.699691] [INFO ] [MAIN ] GDDY: Loaded 50516 existing records. [2025-10-20T18:55:33.069994] [WARN ] [MAIN ] GDDY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:33.085000] [INFO ] [MAIN ] GDDY: Wiping existing primary data before full refresh. [2025-10-20T18:55:33.107543] [INFO ] [LOCAL ] GDDY: Starting deletion of primary data. [2025-10-20T18:55:33.111922] [WARN ] [MAIN ] GDIV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:33.112540] [INFO ] [MAIN ] GDIV: Wiping existing primary data before full refresh. [2025-10-20T18:55:33.112612] [DEBUG] [LOCAL ] GDDY: Deleted primary data file. [2025-10-20T18:55:33.113030] [INFO ] [LOCAL ] GDIV: Starting deletion of primary data. [2025-10-20T18:55:33.113301] [DEBUG] [LOCAL ] GDIV: Deleted primary data file. [2025-10-20T18:55:33.113361] [INFO ] [MAIN ] GDIV: Starting full primary refresh. [2025-10-20T18:55:33.112916] [INFO ] [MAIN ] GDDY: Starting full primary refresh. [2025-10-20T18:55:33.727681] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:55:33.759317] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-08. [2025-10-20T18:55:33.895639] [INFO ] [REMOTE ] GDIV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:33.900503] [INFO ] [REMOTE ] GDDY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:33.901089] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-01. [2025-10-20T18:55:33.928886] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-05 (2.47 seconds). [2025-10-20T18:55:34.651378] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-01. [2025-10-20T18:55:34.794448] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:55:35.279225] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T18:55:35.446072] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:55:35.455230] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-06. [2025-10-20T18:55:36.251390] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-09. [2025-10-20T18:55:37.051369] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-02. [2025-10-20T18:55:37.170809] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:55:37.399924] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T18:55:37.851387] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-02. [2025-10-20T18:55:38.396013] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T18:55:38.651361] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-10. [2025-10-20T18:55:38.727523] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T18:55:39.451387] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-03. [2025-10-20T18:55:39.512247] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T18:55:39.515314] [INFO ] [REMOTE ] GCTS: Full fetch complete (26915 records). [2025-10-20T18:55:39.515398] [DEBUG] [VALIDATOR ] GCTS: Validating primary data. [2025-10-20T18:55:39.909585] [DEBUG] [VALIDATOR ] GCTS: All expected trading days are present. [2025-10-20T18:55:39.909688] [DEBUG] [VALIDATOR ] GCTS: Primary data validation passed. [2025-10-20T18:55:39.910213] [INFO ] [MAIN ] GCTS: Fetched primary data is valid. [2025-10-20T18:55:39.910253] [DEBUG] [LOCAL ] GCTS: Saving primary data. [2025-10-20T18:55:39.911023] [DEBUG] [LOCAL ] GCTS: Trimmed 26915 records to 20245 (keeping 200 days of data) [2025-10-20T18:55:40.021279] [DEBUG] [LOCAL ] GCTS: Primary data saved (20245 records). [2025-10-20T18:55:40.021408] [INFO ] [MAIN ] GCTS: Full history saved. [2025-10-20T18:55:40.021482] [INFO ] [MAIN ] Update Progress: 902/2367 symbols completed. GCTS took 34.0s. Estimated remaining: 3h 7m 45s. Rate: 7.8 symbols/min [2025-10-20T18:55:40.187767] [INFO ] [MAIN ] GDL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:40.191447] [DEBUG] [LOCAL ] GDL: Loaded 2550 existing records from primary storage. [2025-10-20T18:55:40.191578] [INFO ] [MAIN ] GDL: Loaded 2550 existing records. [2025-10-20T18:55:40.256614] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-03. [2025-10-20T18:55:40.368516] [WARN ] [MAIN ] GDL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:40.368753] [INFO ] [MAIN ] GDL: Wiping existing primary data before full refresh. [2025-10-20T18:55:40.368808] [INFO ] [LOCAL ] GDL: Starting deletion of primary data. [2025-10-20T18:55:40.368952] [DEBUG] [LOCAL ] GDL: Deleted primary data file. [2025-10-20T18:55:40.369007] [INFO ] [MAIN ] GDL: Starting full primary refresh. [2025-10-20T18:55:40.729014] [INFO ] [REMOTE ] GDL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:40.732271] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T18:55:41.051395] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-04. [2025-10-20T18:55:41.161800] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T18:55:41.590564] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T18:55:41.851420] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-05. [2025-10-20T18:55:42.451613] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T18:55:42.651389] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-04. [2025-10-20T18:55:43.451388] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-06. [2025-10-20T18:55:43.993708] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:55:44.235482] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T18:55:44.251355] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-05. [2025-10-20T18:55:45.051379] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-07. [2025-10-20T18:55:45.504626] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T18:55:45.595862] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-10-20T18:55:45.851369] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-06. [2025-10-20T18:55:46.651370] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-01. [2025-10-20T18:55:47.163329] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T18:55:47.298155] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T18:55:47.451429] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-07. [2025-10-20T18:55:48.251482] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-07. [2025-10-20T18:55:49.051421] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-08. [2025-10-20T18:55:49.112441] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T18:55:49.370396] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-10-20T18:55:49.374069] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:55:49.851398] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-02. [2025-10-20T18:55:50.265195] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T18:55:50.651368] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-03. [2025-10-20T18:55:51.097696] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T18:55:51.451397] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-08. [2025-10-20T18:55:52.251367] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-09. [2025-10-20T18:55:52.707807] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T18:55:53.051364] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-04. [2025-10-20T18:55:53.227097] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:55:53.359187] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-10-20T18:55:53.851381] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-09. [2025-10-20T18:55:54.651408] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-10. [2025-10-20T18:55:55.235118] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T18:55:55.236375] [INFO ] [REMOTE ] GDIV: Full fetch complete (5413 records). [2025-10-20T18:55:55.236474] [DEBUG] [VALIDATOR ] GDIV: Validating primary data. [2025-10-20T18:55:55.459060] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-08. [2025-10-20T18:55:55.662078] [DEBUG] [VALIDATOR ] GDIV: All expected trading days are present. [2025-10-20T18:55:55.662313] [DEBUG] [VALIDATOR ] GDIV: Primary data validation passed. [2025-10-20T18:55:55.664108] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T18:55:55.662911] [INFO ] [MAIN ] GDIV: Fetched primary data is valid. [2025-10-20T18:55:55.674036] [DEBUG] [LOCAL ] GDIV: Saving primary data. [2025-10-20T18:55:55.675036] [DEBUG] [LOCAL ] GDIV: Trimmed 5413 records to 3580 (keeping 200 days of data) [2025-10-20T18:55:55.699114] [DEBUG] [LOCAL ] GDIV: Primary data saved (3580 records). [2025-10-20T18:55:55.699245] [INFO ] [MAIN ] GDIV: Full history saved. [2025-10-20T18:55:55.699327] [INFO ] [MAIN ] Update Progress: 903/2367 symbols completed. GDIV took 23.2s. Estimated remaining: 3h 7m 50s. Rate: 7.8 symbols/min [2025-10-20T18:55:55.868964] [INFO ] [MAIN ] GDO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:55.876287] [DEBUG] [LOCAL ] GDO: Loaded 5892 existing records from primary storage. [2025-10-20T18:55:55.876495] [INFO ] [MAIN ] GDO: Loaded 5892 existing records. [2025-10-20T18:55:56.069398] [WARN ] [MAIN ] GDO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:56.069719] [INFO ] [MAIN ] GDO: Wiping existing primary data before full refresh. [2025-10-20T18:55:56.069778] [INFO ] [LOCAL ] GDO: Starting deletion of primary data. [2025-10-20T18:55:56.070179] [DEBUG] [LOCAL ] GDO: Deleted primary data file. [2025-10-20T18:55:56.070242] [INFO ] [MAIN ] GDO: Starting full primary refresh. [2025-10-20T18:55:56.264270] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-05. [2025-10-20T18:55:56.474151] [INFO ] [REMOTE ] GDO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:56.814070] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T18:55:57.051381] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-10. [2025-10-20T18:55:57.225344] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:55:57.851451] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-01. [2025-10-20T18:55:58.533761] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T18:55:58.651357] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-09. [2025-10-20T18:55:58.819014] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T18:55:58.826561] [INFO ] [REMOTE ] GDDY: Full fetch complete (72158 records). [2025-10-20T18:55:58.826744] [DEBUG] [VALIDATOR ] GDDY: Validating primary data. [2025-10-20T18:55:59.206135] [DEBUG] [VALIDATOR ] GDDY: All expected trading days are present. [2025-10-20T18:55:59.206223] [DEBUG] [VALIDATOR ] GDDY: Primary data validation passed. [2025-10-20T18:55:59.207122] [INFO ] [MAIN ] GDDY: Fetched primary data is valid. [2025-10-20T18:55:59.207188] [DEBUG] [LOCAL ] GDDY: Saving primary data. [2025-10-20T18:55:59.208685] [DEBUG] [LOCAL ] GDDY: Trimmed 72158 records to 49758 (keeping 200 days of data) [2025-10-20T18:55:59.451724] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-06. [2025-10-20T18:55:59.544347] [DEBUG] [LOCAL ] GDDY: Primary data saved (49758 records). [2025-10-20T18:55:59.544504] [INFO ] [MAIN ] GDDY: Full history saved. [2025-10-20T18:55:59.544593] [INFO ] [MAIN ] Update Progress: 904/2367 symbols completed. GDDY took 27.3s. Estimated remaining: 3h 7m 36s. Rate: 7.8 symbols/min [2025-10-20T18:55:59.739041] [INFO ] [MAIN ] GDOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:59.774572] [DEBUG] [LOCAL ] GDOT: Loaded 38316 existing records from primary storage. [2025-10-20T18:55:59.774881] [INFO ] [MAIN ] GDOT: Loaded 38316 existing records. [2025-10-20T18:55:59.879828] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T18:55:59.959947] [WARN ] [MAIN ] GDOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:59.960298] [INFO ] [MAIN ] GDOT: Wiping existing primary data before full refresh. [2025-10-20T18:55:59.960439] [INFO ] [LOCAL ] GDOT: Starting deletion of primary data. [2025-10-20T18:55:59.960757] [DEBUG] [LOCAL ] GDOT: Deleted primary data file. [2025-10-20T18:55:59.960971] [INFO ] [MAIN ] GDOT: Starting full primary refresh. [2025-10-20T18:56:00.255044] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-02. [2025-10-20T18:56:00.267449] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:56:00.354595] [INFO ] [REMOTE ] GDOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:00.823258] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T18:56:01.051393] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-07. [2025-10-20T18:56:01.773360] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T18:56:01.851366] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-01. [2025-10-20T18:56:02.651380] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-03. [2025-10-20T18:56:03.126297] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T18:56:03.275166] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:56:03.451345] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-04. [2025-10-20T18:56:04.128382] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T18:56:04.251411] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-02. [2025-10-20T18:56:05.051412] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-08. [2025-10-20T18:56:05.465758] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T18:56:05.551104] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-10-20T18:56:05.851381] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-05. [2025-10-20T18:56:06.418623] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T18:56:06.651413] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-06. [2025-10-20T18:56:07.240515] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T18:56:07.451382] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-03. [2025-10-20T18:56:08.251483] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-07. [2025-10-20T18:56:08.866327] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:56:08.922569] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:56:09.051420] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-08. [2025-10-20T18:56:09.835357] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T18:56:09.851396] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-10. [2025-10-20T18:56:10.651373] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-04. [2025-10-20T18:56:11.451437] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-09. [2025-10-20T18:56:11.687253] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:56:11.695001] [INFO ] [REMOTE ] GD: Full fetch complete (69721 records). [2025-10-20T18:56:11.695174] [DEBUG] [VALIDATOR ] GD: Validating primary data. [2025-10-20T18:56:12.045075] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T18:56:12.176858] [DEBUG] [VALIDATOR ] GD: All expected trading days are present. [2025-10-20T18:56:12.179386] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T18:56:12.179631] [DEBUG] [VALIDATOR ] GD: Primary data validation passed. [2025-10-20T18:56:12.180860] [INFO ] [MAIN ] GD: Fetched primary data is valid. [2025-10-20T18:56:12.189227] [DEBUG] [LOCAL ] GD: Saving primary data. [2025-10-20T18:56:12.191325] [DEBUG] [LOCAL ] GD: Trimmed 69721 records to 46577 (keeping 200 days of data) [2025-10-20T18:56:12.256496] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-05. [2025-10-20T18:56:12.488442] [DEBUG] [LOCAL ] GD: Primary data saved (46577 records). [2025-10-20T18:56:12.488608] [INFO ] [MAIN ] GD: Full history saved. [2025-10-20T18:56:12.488947] [INFO ] [MAIN ] Update Progress: 905/2367 symbols completed. GD took 59.1s. Estimated remaining: 3h 7m 37s. Rate: 7.8 symbols/min [2025-10-20T18:56:12.654795] [INFO ] [MAIN ] GDV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:12.675674] [DEBUG] [LOCAL ] GDV: Loaded 19698 existing records from primary storage. [2025-10-20T18:56:12.675923] [INFO ] [MAIN ] GDV: Loaded 19698 existing records. [2025-10-20T18:56:12.846507] [WARN ] [MAIN ] GDV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:12.846746] [INFO ] [MAIN ] GDV: Wiping existing primary data before full refresh. [2025-10-20T18:56:12.846796] [INFO ] [LOCAL ] GDV: Starting deletion of primary data. [2025-10-20T18:56:12.847183] [DEBUG] [LOCAL ] GDV: Deleted primary data file. [2025-10-20T18:56:12.847238] [INFO ] [MAIN ] GDV: Starting full primary refresh. [2025-10-20T18:56:13.056691] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-09. [2025-10-20T18:56:13.232768] [INFO ] [REMOTE ] GDV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:13.732967] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:56:13.800565] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T18:56:13.851412] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-10. [2025-10-20T18:56:14.276187] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T18:56:14.277324] [INFO ] [REMOTE ] GDL: Full fetch complete (3499 records). [2025-10-20T18:56:14.277401] [DEBUG] [VALIDATOR ] GDL: Validating primary data. [2025-10-20T18:56:14.656550] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-06. [2025-10-20T18:56:14.687270] [DEBUG] [VALIDATOR ] GDL: All expected trading days are present. [2025-10-20T18:56:14.687349] [DEBUG] [VALIDATOR ] GDL: Primary data validation passed. [2025-10-20T18:56:14.687469] [INFO ] [MAIN ] GDL: Fetched primary data is valid. [2025-10-20T18:56:14.687506] [DEBUG] [LOCAL ] GDL: Saving primary data. [2025-10-20T18:56:14.687890] [DEBUG] [LOCAL ] GDL: Trimmed 3499 records to 2525 (keeping 200 days of data) [2025-10-20T18:56:14.703558] [DEBUG] [LOCAL ] GDL: Primary data saved (2525 records). [2025-10-20T18:56:14.703628] [INFO ] [MAIN ] GDL: Full history saved. [2025-10-20T18:56:14.703734] [INFO ] [MAIN ] Update Progress: 906/2367 symbols completed. GDL took 34.7s. Estimated remaining: 3h 7m 20s. Rate: 7.8 symbols/min [2025-10-20T18:56:14.867514] [INFO ] [MAIN ] GE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:14.917841] [DEBUG] [LOCAL ] GE: Loaded 54478 existing records from primary storage. [2025-10-20T18:56:14.918158] [INFO ] [MAIN ] GE: Loaded 54478 existing records. [2025-10-20T18:56:15.121227] [WARN ] [MAIN ] GE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:15.121492] [INFO ] [MAIN ] GE: Wiping existing primary data before full refresh. [2025-10-20T18:56:15.121549] [INFO ] [LOCAL ] GE: Starting deletion of primary data. [2025-10-20T18:56:15.121837] [DEBUG] [LOCAL ] GE: Deleted primary data file. [2025-10-20T18:56:15.121905] [INFO ] [MAIN ] GE: Starting full primary refresh. [2025-10-20T18:56:15.451556] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-01. [2025-10-20T18:56:15.471952] [INFO ] [REMOTE ] GE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:16.181991] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T18:56:16.240311] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T18:56:16.251329] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-02. [2025-10-20T18:56:17.051432] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-01. [2025-10-20T18:56:17.237570] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T18:56:17.851371] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-03. [2025-10-20T18:56:18.651398] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-10. [2025-10-20T18:56:18.851621] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T18:56:19.162007] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-10-20T18:56:19.265704] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T18:56:19.266891] [INFO ] [REMOTE ] GDO: Full fetch complete (8492 records). [2025-10-20T18:56:19.266960] [DEBUG] [VALIDATOR ] GDO: Validating primary data. [2025-10-20T18:56:19.456591] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-02. [2025-10-20T18:56:19.622968] [DEBUG] [VALIDATOR ] GDO: All expected trading days are present. [2025-10-20T18:56:19.623041] [DEBUG] [VALIDATOR ] GDO: Primary data validation passed. [2025-10-20T18:56:19.623245] [INFO ] [MAIN ] GDO: Fetched primary data is valid. [2025-10-20T18:56:19.623283] [DEBUG] [LOCAL ] GDO: Saving primary data. [2025-10-20T18:56:19.623689] [DEBUG] [LOCAL ] GDO: Trimmed 8492 records to 5840 (keeping 200 days of data) [2025-10-20T18:56:19.657432] [DEBUG] [LOCAL ] GDO: Primary data saved (5840 records). [2025-10-20T18:56:19.657517] [INFO ] [MAIN ] GDO: Full history saved. [2025-10-20T18:56:19.657589] [INFO ] [MAIN ] Update Progress: 907/2367 symbols completed. GDO took 24.0s. Estimated remaining: 3h 7m 8s. Rate: 7.8 symbols/min [2025-10-20T18:56:19.823254] [INFO ] [MAIN ] GEF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:19.843075] [DEBUG] [LOCAL ] GEF: Loaded 21098 existing records from primary storage. [2025-10-20T18:56:19.843319] [INFO ] [MAIN ] GEF: Loaded 21098 existing records. [2025-10-20T18:56:20.019125] [WARN ] [MAIN ] GEF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:20.019339] [INFO ] [MAIN ] GEF: Wiping existing primary data before full refresh. [2025-10-20T18:56:20.019385] [INFO ] [LOCAL ] GEF: Starting deletion of primary data. [2025-10-20T18:56:20.019716] [DEBUG] [LOCAL ] GEF: Deleted primary data file. [2025-10-20T18:56:20.019771] [INFO ] [MAIN ] GEF: Starting full primary refresh. [2025-10-20T18:56:20.256456] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-07. [2025-10-20T18:56:20.368747] [INFO ] [REMOTE ] GEF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:21.051369] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-04. [2025-10-20T18:56:21.498291] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-10-20T18:56:21.772148] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T18:56:21.851333] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-08. [2025-10-20T18:56:22.213508] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T18:56:22.651375] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-03. [2025-10-20T18:56:23.278579] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T18:56:23.451382] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-05. [2025-10-20T18:56:24.251350] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-01. [2025-10-20T18:56:24.490163] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T18:56:24.539480] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T18:56:25.051341] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-04. [2025-10-20T18:56:25.278223] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T18:56:25.851405] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-02. [2025-10-20T18:56:26.651386] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-09. [2025-10-20T18:56:26.704880] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T18:56:27.266005] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-04 (2.21 seconds). [2025-10-20T18:56:27.451351] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-03. [2025-10-20T18:56:28.156024] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:56:28.251342] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-10. [2025-10-20T18:56:28.459236] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T18:56:29.051373] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-06. [2025-10-20T18:56:29.720053] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T18:56:29.725941] [INFO ] [REMOTE ] GDOT: Full fetch complete (53599 records). [2025-10-20T18:56:29.726075] [DEBUG] [VALIDATOR ] GDOT: Validating primary data. [2025-10-20T18:56:29.856631] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-04. [2025-10-20T18:56:30.121803] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T18:56:30.144887] [DEBUG] [VALIDATOR ] GDOT: All expected trading days are present. [2025-10-20T18:56:30.144972] [DEBUG] [VALIDATOR ] GDOT: Primary data validation passed. [2025-10-20T18:56:30.145802] [INFO ] [MAIN ] GDOT: Fetched primary data is valid. [2025-10-20T18:56:30.145868] [DEBUG] [LOCAL ] GDOT: Saving primary data. [2025-10-20T18:56:30.147047] [DEBUG] [LOCAL ] GDOT: Trimmed 53599 records to 37755 (keeping 200 days of data) [2025-10-20T18:56:30.359907] [DEBUG] [LOCAL ] GDOT: Primary data saved (37755 records). [2025-10-20T18:56:30.360056] [INFO ] [MAIN ] GDOT: Full history saved. [2025-10-20T18:56:30.360133] [INFO ] [MAIN ] Update Progress: 908/2367 symbols completed. GDOT took 30.8s. Estimated remaining: 3h 7m 5s. Rate: 7.8 symbols/min [2025-10-20T18:56:30.535022] [INFO ] [MAIN ] GEL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:30.568224] [DEBUG] [LOCAL ] GEL: Loaded 32124 existing records from primary storage. [2025-10-20T18:56:30.568508] [INFO ] [MAIN ] GEL: Loaded 32124 existing records. [2025-10-20T18:56:30.656733] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-07. [2025-10-20T18:56:30.753246] [WARN ] [MAIN ] GEL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:30.753533] [INFO ] [MAIN ] GEL: Wiping existing primary data before full refresh. [2025-10-20T18:56:30.753580] [INFO ] [LOCAL ] GEL: Starting deletion of primary data. [2025-10-20T18:56:30.753869] [DEBUG] [LOCAL ] GEL: Deleted primary data file. [2025-10-20T18:56:30.753936] [INFO ] [MAIN ] GEL: Starting full primary refresh. [2025-10-20T18:56:31.015324] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T18:56:31.130711] [INFO ] [REMOTE ] GEL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:31.451344] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-05. [2025-10-20T18:56:31.670696] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T18:56:32.251390] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-01. [2025-10-20T18:56:32.277225] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:56:33.051370] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-06. [2025-10-20T18:56:33.383740] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T18:56:33.851393] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-02. [2025-10-20T18:56:33.888630] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T18:56:34.651421] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-07. [2025-10-20T18:56:35.168438] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:56:35.451345] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-03. [2025-10-20T18:56:35.701354] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T18:56:36.251424] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-08. [2025-10-20T18:56:36.970752] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:56:37.051362] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-04. [2025-10-20T18:56:37.195471] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T18:56:37.851390] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-09. [2025-10-20T18:56:38.075814] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T18:56:38.651395] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-08. [2025-10-20T18:56:38.787723] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T18:56:39.451410] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-05. [2025-10-20T18:56:39.467567] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T18:56:40.251379] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-10. [2025-10-20T18:56:41.051395] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-09. [2025-10-20T18:56:41.279346] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T18:56:41.391330] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T18:56:41.394766] [INFO ] [REMOTE ] GDV: Full fetch complete (28419 records). [2025-10-20T18:56:41.394889] [DEBUG] [VALIDATOR ] GDV: Validating primary data. [2025-10-20T18:56:41.782917] [DEBUG] [VALIDATOR ] GDV: All expected trading days are present. [2025-10-20T18:56:41.783000] [DEBUG] [VALIDATOR ] GDV: Primary data validation passed. [2025-10-20T18:56:41.783603] [INFO ] [MAIN ] GDV: Fetched primary data is valid. [2025-10-20T18:56:41.783667] [DEBUG] [LOCAL ] GDV: Saving primary data. [2025-10-20T18:56:41.784521] [DEBUG] [LOCAL ] GDV: Trimmed 28419 records to 19418 (keeping 200 days of data) [2025-10-20T18:56:41.868888] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-05. [2025-10-20T18:56:41.906407] [DEBUG] [LOCAL ] GDV: Primary data saved (19418 records). [2025-10-20T18:56:41.906563] [INFO ] [MAIN ] GDV: Full history saved. [2025-10-20T18:56:41.906676] [INFO ] [MAIN ] Update Progress: 909/2367 symbols completed. GDV took 29.4s. Estimated remaining: 3h 7m 4s. Rate: 7.8 symbols/min [2025-10-20T18:56:42.080355] [INFO ] [MAIN ] GENI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:42.130522] [DEBUG] [LOCAL ] GENI: Loaded 53717 existing records from primary storage. [2025-10-20T18:56:42.130977] [INFO ] [MAIN ] GENI: Loaded 53717 existing records. [2025-10-20T18:56:42.235032] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:56:42.345818] [WARN ] [MAIN ] GENI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:42.345975] [INFO ] [MAIN ] GENI: Wiping existing primary data before full refresh. [2025-10-20T18:56:42.346021] [INFO ] [LOCAL ] GENI: Starting deletion of primary data. [2025-10-20T18:56:42.346320] [DEBUG] [LOCAL ] GENI: Deleted primary data file. [2025-10-20T18:56:42.346376] [INFO ] [MAIN ] GENI: Starting full primary refresh. [2025-10-20T18:56:42.658885] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-06. [2025-10-20T18:56:42.699522] [INFO ] [REMOTE ] GENI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:43.451392] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-01. [2025-10-20T18:56:43.483828] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T18:56:44.251358] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-06. [2025-10-20T18:56:44.895078] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-06 (2.24 seconds). [2025-10-20T18:56:44.974842] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T18:56:45.051334] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-07. [2025-10-20T18:56:45.658376] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:56:45.851362] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-07. [2025-10-20T18:56:46.651359] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-10. [2025-10-20T18:56:46.901841] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:56:47.282799] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:56:47.451397] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-02. [2025-10-20T18:56:47.678582] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:56:47.682155] [INFO ] [REMOTE ] GEF: Full fetch complete (29381 records). [2025-10-20T18:56:47.682252] [DEBUG] [VALIDATOR ] GEF: Validating primary data. [2025-10-20T18:56:48.046797] [DEBUG] [VALIDATOR ] GEF: All expected trading days are present. [2025-10-20T18:56:48.046877] [DEBUG] [VALIDATOR ] GEF: Primary data validation passed. [2025-10-20T18:56:48.047435] [INFO ] [MAIN ] GEF: Fetched primary data is valid. [2025-10-20T18:56:48.047473] [DEBUG] [LOCAL ] GEF: Saving primary data. [2025-10-20T18:56:48.048155] [DEBUG] [LOCAL ] GEF: Trimmed 29381 records to 20852 (keeping 200 days of data) [2025-10-20T18:56:48.171352] [DEBUG] [LOCAL ] GEF: Primary data saved (20852 records). [2025-10-20T18:56:48.171472] [INFO ] [MAIN ] GEF: Full history saved. [2025-10-20T18:56:48.171547] [INFO ] [MAIN ] Update Progress: 910/2367 symbols completed. GEF took 28.5s. Estimated remaining: 3h 6m 54s. Rate: 7.8 symbols/min [2025-10-20T18:56:48.260739] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-08. [2025-10-20T18:56:48.343502] [INFO ] [MAIN ] GEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:48.396438] [DEBUG] [LOCAL ] GEO: Loaded 52550 existing records from primary storage. [2025-10-20T18:56:48.396780] [INFO ] [MAIN ] GEO: Loaded 52550 existing records. [2025-10-20T18:56:48.577177] [WARN ] [MAIN ] GEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:48.577406] [INFO ] [MAIN ] GEO: Wiping existing primary data before full refresh. [2025-10-20T18:56:48.577482] [INFO ] [LOCAL ] GEO: Starting deletion of primary data. [2025-10-20T18:56:48.577759] [DEBUG] [LOCAL ] GEO: Deleted primary data file. [2025-10-20T18:56:48.577834] [INFO ] [MAIN ] GEO: Starting full primary refresh. [2025-10-20T18:56:48.951960] [INFO ] [REMOTE ] GEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:48.983762] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:56:49.051416] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-08. [2025-10-20T18:56:49.851378] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-03. [2025-10-20T18:56:50.428991] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T18:56:50.576118] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-08 (2.32 seconds). [2025-10-20T18:56:50.651361] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-09. [2025-10-20T18:56:51.435069] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T18:56:51.451337] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-04. [2025-10-20T18:56:52.251358] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-01. [2025-10-20T18:56:52.819304] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T18:56:53.040766] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:56:53.051507] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-09. [2025-10-20T18:56:53.837356] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:56:53.851331] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-05. [2025-10-20T18:56:53.951438] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:56:54.651448] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-02. [2025-10-20T18:56:55.395411] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:56:55.451381] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-06. [2025-10-20T18:56:56.251383] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-10. [2025-10-20T18:56:56.323426] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:56:57.007907] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T18:56:57.051377] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-10. [2025-10-20T18:56:57.426402] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T18:56:57.430094] [INFO ] [REMOTE ] GEL: Full fetch complete (48079 records). [2025-10-20T18:56:57.430184] [DEBUG] [VALIDATOR ] GEL: Validating primary data. [2025-10-20T18:56:57.832276] [DEBUG] [VALIDATOR ] GEL: All expected trading days are present. [2025-10-20T18:56:57.832363] [DEBUG] [VALIDATOR ] GEL: Primary data validation passed. [2025-10-20T18:56:57.833091] [INFO ] [MAIN ] GEL: Fetched primary data is valid. [2025-10-20T18:56:57.833178] [DEBUG] [LOCAL ] GEL: Saving primary data. [2025-10-20T18:56:57.834248] [DEBUG] [LOCAL ] GEL: Trimmed 48079 records to 31500 (keeping 200 days of data) [2025-10-20T18:56:57.886129] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-03. [2025-10-20T18:56:58.053403] [DEBUG] [LOCAL ] GEL: Primary data saved (31500 records). [2025-10-20T18:56:58.054149] [INFO ] [MAIN ] GEL: Full history saved. [2025-10-20T18:56:58.054952] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T18:56:58.062588] [INFO ] [MAIN ] Update Progress: 911/2367 symbols completed. GEL took 27.7s. Estimated remaining: 3h 6m 50s. Rate: 7.8 symbols/min [2025-10-20T18:56:58.121148] [INFO ] [REMOTE ] GE: Full fetch complete (77865 records). [2025-10-20T18:56:58.136478] [DEBUG] [VALIDATOR ] GE: Validating primary data. [2025-10-20T18:56:58.404755] [INFO ] [MAIN ] GES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:58.653242] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-07. [2025-10-20T18:56:58.657104] [DEBUG] [VALIDATOR ] GE: All expected trading days are present. [2025-10-20T18:56:58.658207] [DEBUG] [VALIDATOR ] GE: Primary data validation passed. [2025-10-20T18:56:58.659962] [INFO ] [MAIN ] GE: Fetched primary data is valid. [2025-10-20T18:56:58.672516] [DEBUG] [LOCAL ] GE: Saving primary data. [2025-10-20T18:56:58.673746] [DEBUG] [LOCAL ] GES: Loaded 38876 existing records from primary storage. [2025-10-20T18:56:58.674351] [INFO ] [MAIN ] GES: Loaded 38876 existing records. [2025-10-20T18:56:58.745387] [DEBUG] [LOCAL ] GE: Trimmed 77865 records to 53698 (keeping 200 days of data) [2025-10-20T18:56:58.949203] [WARN ] [MAIN ] GES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:58.952125] [INFO ] [MAIN ] GES: Wiping existing primary data before full refresh. [2025-10-20T18:56:58.959457] [INFO ] [LOCAL ] GES: Starting deletion of primary data. [2025-10-20T18:56:58.961394] [DEBUG] [LOCAL ] GES: Deleted primary data file. [2025-10-20T18:56:58.961987] [INFO ] [MAIN ] GES: Starting full primary refresh. [2025-10-20T18:56:59.353494] [INFO ] [REMOTE ] GES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:59.451993] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-01. [2025-10-20T18:56:59.567202] [DEBUG] [LOCAL ] GE: Primary data saved (53698 records). [2025-10-20T18:56:59.567351] [INFO ] [MAIN ] GE: Full history saved. [2025-10-20T18:56:59.567430] [INFO ] [MAIN ] Update Progress: 912/2367 symbols completed. GE took 44.9s. Estimated remaining: 3h 6m 32s. Rate: 7.8 symbols/min [2025-10-20T18:56:59.739982] [INFO ] [MAIN ] GETY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:59.772500] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:56:59.814959] [DEBUG] [LOCAL ] GETY: Loaded 35677 existing records from primary storage. [2025-10-20T18:56:59.815291] [INFO ] [MAIN ] GETY: Loaded 35677 existing records. [2025-10-20T18:56:59.995422] [WARN ] [MAIN ] GETY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:59.995633] [INFO ] [MAIN ] GETY: Wiping existing primary data before full refresh. [2025-10-20T18:56:59.995729] [INFO ] [LOCAL ] GETY: Starting deletion of primary data. [2025-10-20T18:56:59.995970] [DEBUG] [LOCAL ] GETY: Deleted primary data file. [2025-10-20T18:56:59.996025] [INFO ] [MAIN ] GETY: Starting full primary refresh. [2025-10-20T18:57:00.253223] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-04. [2025-10-20T18:57:00.371441] [INFO ] [REMOTE ] GETY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:00.403750] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:57:01.051348] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-08. [2025-10-20T18:57:01.485367] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-10-20T18:57:01.851380] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-01. [2025-10-20T18:57:02.414304] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T18:57:02.651356] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-05. [2025-10-20T18:57:02.833969] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:57:03.451376] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-02. [2025-10-20T18:57:03.576412] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T18:57:04.251348] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-02. [2025-10-20T18:57:04.393117] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:57:04.850939] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T18:57:05.051386] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-09. [2025-10-20T18:57:05.787843] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:57:05.851377] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-03. [2025-10-20T18:57:06.651376] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-06. [2025-10-20T18:57:06.886074] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:57:07.440480] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:57:07.451337] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-04. [2025-10-20T18:57:08.251354] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-03. [2025-10-20T18:57:08.290835] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:57:08.913267] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:57:09.051385] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-10. [2025-10-20T18:57:09.851350] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-05. [2025-10-20T18:57:09.949100] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:57:10.651417] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-07. [2025-10-20T18:57:10.780307] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:57:10.788201] [INFO ] [REMOTE ] GENI: Full fetch complete (76088 records). [2025-10-20T18:57:10.788355] [DEBUG] [VALIDATOR ] GENI: Validating primary data. [2025-10-20T18:57:11.191284] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:57:11.227906] [DEBUG] [VALIDATOR ] GENI: All expected trading days are present. [2025-10-20T18:57:11.228027] [DEBUG] [VALIDATOR ] GENI: Primary data validation passed. [2025-10-20T18:57:11.229448] [INFO ] [MAIN ] GENI: Fetched primary data is valid. [2025-10-20T18:57:11.229516] [DEBUG] [LOCAL ] GENI: Saving primary data. [2025-10-20T18:57:11.231022] [DEBUG] [LOCAL ] GENI: Trimmed 76088 records to 52943 (keeping 200 days of data) [2025-10-20T18:57:11.452010] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-04. [2025-10-20T18:57:11.545381] [DEBUG] [LOCAL ] GENI: Primary data saved (52943 records). [2025-10-20T18:57:11.545537] [INFO ] [MAIN ] GENI: Full history saved. [2025-10-20T18:57:11.546119] [INFO ] [MAIN ] Update Progress: 913/2367 symbols completed. GENI took 29.6s. Estimated remaining: 3h 6m 31s. Rate: 7.8 symbols/min [2025-10-20T18:57:11.711484] [INFO ] [MAIN ] GEV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:11.763104] [DEBUG] [LOCAL ] GEV: Loaded 53772 existing records from primary storage. [2025-10-20T18:57:11.763405] [INFO ] [MAIN ] GEV: Loaded 53772 existing records. [2025-10-20T18:57:11.939820] [WARN ] [MAIN ] GEV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:57:11.940030] [INFO ] [MAIN ] GEV: Wiping existing primary data before full refresh. [2025-10-20T18:57:11.940077] [INFO ] [LOCAL ] GEV: Starting deletion of primary data. [2025-10-20T18:57:11.940264] [DEBUG] [LOCAL ] GEV: Deleted primary data file. [2025-10-20T18:57:11.940301] [INFO ] [MAIN ] GEV: Starting full primary refresh. [2025-10-20T18:57:12.262328] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-06. [2025-10-20T18:57:12.303681] [INFO ] [REMOTE ] GEV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:12.552836] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T18:57:13.051347] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-01. [2025-10-20T18:57:13.140204] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:57:13.705327] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T18:57:13.851360] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-08. [2025-10-20T18:57:14.651379] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-05. [2025-10-20T18:57:15.252899] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-01 (2.20 seconds). [2025-10-20T18:57:15.331614] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:57:15.451396] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-09. [2025-10-20T18:57:16.064932] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T18:57:16.251378] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-06. [2025-10-20T18:57:17.051012] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T18:57:17.060955] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-07. [2025-10-20T18:57:17.794774] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T18:57:17.851338] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-07. [2025-10-20T18:57:18.488906] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:57:18.651398] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-08. [2025-10-20T18:57:19.333032] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T18:57:19.451367] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-02. [2025-10-20T18:57:20.117216] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T18:57:20.251383] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-08. [2025-10-20T18:57:21.051392] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-10. [2025-10-20T18:57:21.657074] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-10-20T18:57:21.775382] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T18:57:21.851386] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-09. [2025-10-20T18:57:22.651345] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-09. [2025-10-20T18:57:22.995041] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T18:57:23.004039] [INFO ] [REMOTE ] GEO: Full fetch complete (75543 records). [2025-10-20T18:57:23.004244] [DEBUG] [VALIDATOR ] GEO: Validating primary data. [2025-10-20T18:57:23.418819] [DEBUG] [VALIDATOR ] GEO: All expected trading days are present. [2025-10-20T18:57:23.433451] [DEBUG] [VALIDATOR ] GEO: Primary data validation passed. [2025-10-20T18:57:23.435437] [INFO ] [MAIN ] GEO: Fetched primary data is valid. [2025-10-20T18:57:23.436559] [DEBUG] [LOCAL ] GEO: Saving primary data. [2025-10-20T18:57:23.436451] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:57:23.445938] [DEBUG] [LOCAL ] GEO: Trimmed 75543 records to 51753 (keeping 200 days of data) [2025-10-20T18:57:23.493566] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-03. [2025-10-20T18:57:23.754670] [DEBUG] [LOCAL ] GEO: Primary data saved (51753 records). [2025-10-20T18:57:23.754866] [INFO ] [MAIN ] GEO: Full history saved. [2025-10-20T18:57:23.755521] [INFO ] [MAIN ] Update Progress: 914/2367 symbols completed. GEO took 35.6s. Estimated remaining: 3h 6m 31s. Rate: 7.8 symbols/min [2025-10-20T18:57:23.929961] [INFO ] [MAIN ] GF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:23.930170] [DEBUG] [LOCAL ] GF: No existing data file found in primary storage. [2025-10-20T18:57:23.930357] [WARN ] [MAIN ] GF: No existing data found, performing full refresh instead. [2025-10-20T18:57:23.930587] [INFO ] [MAIN ] GF: Wiping existing primary data before full refresh. [2025-10-20T18:57:23.930762] [INFO ] [LOCAL ] GF: Starting deletion of primary data. [2025-10-20T18:57:23.930838] [DEBUG] [LOCAL ] GF: No primary data file found to delete. [2025-10-20T18:57:23.930888] [INFO ] [MAIN ] GF: Starting full primary refresh. [2025-10-20T18:57:24.104387] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T18:57:24.256466] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-10. [2025-10-20T18:57:24.325578] [INFO ] [REMOTE ] GF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:25.051434] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-01. [2025-10-20T18:57:25.540717] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T18:57:25.602510] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T18:57:25.646319] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T18:57:25.654457] [INFO ] [REMOTE ] GES: Full fetch complete (58555 records). [2025-10-20T18:57:25.654702] [DEBUG] [VALIDATOR ] GES: Validating primary data. [2025-10-20T18:57:25.858540] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-02. [2025-10-20T18:57:26.039841] [DEBUG] [VALIDATOR ] GES: All expected trading days are present. [2025-10-20T18:57:26.039933] [DEBUG] [VALIDATOR ] GES: Primary data validation passed. [2025-10-20T18:57:26.040780] [INFO ] [MAIN ] GES: Fetched primary data is valid. [2025-10-20T18:57:26.040852] [DEBUG] [LOCAL ] GES: Saving primary data. [2025-10-20T18:57:26.042119] [DEBUG] [LOCAL ] GES: Trimmed 58555 records to 38154 (keeping 200 days of data) [2025-10-20T18:57:26.270841] [DEBUG] [LOCAL ] GES: Primary data saved (38154 records). [2025-10-20T18:57:26.270997] [INFO ] [MAIN ] GES: Full history saved. [2025-10-20T18:57:26.271077] [INFO ] [MAIN ] Update Progress: 915/2367 symbols completed. GES took 28.2s. Estimated remaining: 3h 6m 15s. Rate: 7.8 symbols/min [2025-10-20T18:57:26.443784] [INFO ] [MAIN ] GFF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:26.473277] [DEBUG] [LOCAL ] GFF: Loaded 30283 existing records from primary storage. [2025-10-20T18:57:26.473553] [INFO ] [MAIN ] GFF: Loaded 30283 existing records. [2025-10-20T18:57:26.582090] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T18:57:26.657203] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-03. [2025-10-20T18:57:26.663922] [WARN ] [MAIN ] GFF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:57:26.664982] [INFO ] [MAIN ] GFF: Wiping existing primary data before full refresh. [2025-10-20T18:57:26.665034] [INFO ] [LOCAL ] GFF: Starting deletion of primary data. [2025-10-20T18:57:26.665244] [DEBUG] [LOCAL ] GFF: Deleted primary data file. [2025-10-20T18:57:26.665300] [INFO ] [MAIN ] GFF: Starting full primary refresh. [2025-10-20T18:57:27.017897] [INFO ] [REMOTE ] GFF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:27.451397] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-10. [2025-10-20T18:57:27.541999] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T18:57:28.251433] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-01. [2025-10-20T18:57:28.800696] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T18:57:28.806348] [INFO ] [REMOTE ] GETY: Full fetch complete (55671 records). [2025-10-20T18:57:28.806491] [DEBUG] [VALIDATOR ] GETY: Validating primary data. [2025-10-20T18:57:29.056708] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-04. [2025-10-20T18:57:29.187823] [DEBUG] [VALIDATOR ] GETY: All expected trading days are present. [2025-10-20T18:57:29.188088] [DEBUG] [VALIDATOR ] GETY: Primary data validation passed. [2025-10-20T18:57:29.189022] [INFO ] [MAIN ] GETY: Fetched primary data is valid. [2025-10-20T18:57:29.189115] [DEBUG] [LOCAL ] GETY: Saving primary data. [2025-10-20T18:57:29.191022] [DEBUG] [LOCAL ] GETY: Trimmed 55671 records to 35268 (keeping 200 days of data) [2025-10-20T18:57:29.291948] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:57:29.417068] [DEBUG] [LOCAL ] GETY: Primary data saved (35268 records). [2025-10-20T18:57:29.417209] [INFO ] [MAIN ] GETY: Full history saved. [2025-10-20T18:57:29.417287] [INFO ] [MAIN ] Update Progress: 916/2367 symbols completed. GETY took 29.8s. Estimated remaining: 3h 5m 60s. Rate: 7.8 symbols/min [2025-10-20T18:57:29.588693] [INFO ] [MAIN ] GFI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:29.637430] [DEBUG] [LOCAL ] GFI: Loaded 53895 existing records from primary storage. [2025-10-20T18:57:29.637787] [INFO ] [MAIN ] GFI: Loaded 53895 existing records. [2025-10-20T18:57:29.813085] [WARN ] [MAIN ] GFI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:57:29.813307] [INFO ] [MAIN ] GFI: Wiping existing primary data before full refresh. [2025-10-20T18:57:29.813353] [INFO ] [LOCAL ] GFI: Starting deletion of primary data. [2025-10-20T18:57:29.813853] [DEBUG] [LOCAL ] GFI: Deleted primary data file. [2025-10-20T18:57:29.813909] [INFO ] [MAIN ] GFI: Starting full primary refresh. [2025-10-20T18:57:29.856610] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-02. [2025-10-20T18:57:30.166922] [INFO ] [REMOTE ] GFI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:30.651382] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-01. [2025-10-20T18:57:31.219288] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T18:57:31.349032] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T18:57:31.451348] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-05. [2025-10-20T18:57:32.251364] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-03. [2025-10-20T18:57:32.514159] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T18:57:33.051380] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-02. [2025-10-20T18:57:33.249627] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T18:57:33.845860] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-05 (2.39 seconds). [2025-10-20T18:57:33.857783] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-04. [2025-10-20T18:57:34.651541] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-06. [2025-10-20T18:57:34.884455] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T18:57:35.298244] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T18:57:35.451369] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-03. [2025-10-20T18:57:36.251366] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-04. [2025-10-20T18:57:36.528462] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T18:57:36.867846] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T18:57:37.051356] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-05. [2025-10-20T18:57:37.204623] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:57:37.671705] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T18:57:37.851373] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-06. [2025-10-20T18:57:38.498054] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T18:57:38.651371] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-04. [2025-10-20T18:57:39.451383] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-07. [2025-10-20T18:57:39.899426] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T18:57:40.251365] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-08. [2025-10-20T18:57:40.688578] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T18:57:40.736751] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T18:57:41.051380] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-05. [2025-10-20T18:57:41.851438] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-09. [2025-10-20T18:57:42.379903] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T18:57:42.385006] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:57:42.651399] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-10. [2025-10-20T18:57:43.053178] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T18:57:43.055559] [INFO ] [REMOTE ] GF: Full fetch complete (4810 records). [2025-10-20T18:57:43.055776] [DEBUG] [VALIDATOR ] GF: Validating primary data. [2025-10-20T18:57:43.424762] [DEBUG] [VALIDATOR ] GF: All expected trading days are present. [2025-10-20T18:57:43.424841] [DEBUG] [VALIDATOR ] GF: Primary data validation passed. [2025-10-20T18:57:43.424985] [INFO ] [MAIN ] GF: Fetched primary data is valid. [2025-10-20T18:57:43.425022] [DEBUG] [LOCAL ] GF: Saving primary data. [2025-10-20T18:57:43.425373] [DEBUG] [LOCAL ] GF: Trimmed 4810 records to 2662 (keeping 200 days of data) [2025-10-20T18:57:43.440823] [DEBUG] [LOCAL ] GF: Primary data saved (2662 records). [2025-10-20T18:57:43.440888] [INFO ] [MAIN ] GF: Full history saved. [2025-10-20T18:57:43.440948] [INFO ] [MAIN ] Update Progress: 917/2367 symbols completed. GF took 19.7s. Estimated remaining: 3h 6m 2s. Rate: 7.8 symbols/min [2025-10-20T18:57:43.451369] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-05. [2025-10-20T18:57:44.251382] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-06. [2025-10-20T18:57:45.051438] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-07. [2025-10-20T18:57:45.538432] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:57:45.572221] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T18:57:45.851375] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-07. [2025-10-20T18:57:46.651386] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-06. [2025-10-20T18:57:46.966380] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:57:47.451345] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-08. [2025-10-20T18:57:47.593489] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-07 (2.54 seconds). [2025-10-20T18:57:48.251393] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-08. [2025-10-20T18:57:48.568025] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T18:57:48.737901] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T18:57:49.051406] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-09. [2025-10-20T18:57:49.851442] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-07. [2025-10-20T18:57:50.197184] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T18:57:50.651354] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-10. [2025-10-20T18:57:50.715877] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-08 (2.46 seconds). [2025-10-20T18:57:51.451345] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-09. [2025-10-20T18:57:51.519547] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T18:57:51.523621] [INFO ] [REMOTE ] GFF: Full fetch complete (43805 records). [2025-10-20T18:57:51.523757] [DEBUG] [VALIDATOR ] GFF: Validating primary data. [2025-10-20T18:57:51.938316] [DEBUG] [VALIDATOR ] GFF: All expected trading days are present. [2025-10-20T18:57:51.941436] [DEBUG] [VALIDATOR ] GFF: Primary data validation passed. [2025-10-20T18:57:51.942136] [INFO ] [MAIN ] GFF: Fetched primary data is valid. [2025-10-20T18:57:51.942383] [DEBUG] [LOCAL ] GFF: Saving primary data. [2025-10-20T18:57:51.944556] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-10-20T18:57:51.959304] [DEBUG] [LOCAL ] GFF: Trimmed 43805 records to 29664 (keeping 200 days of data) [2025-10-20T18:57:52.148975] [DEBUG] [LOCAL ] GFF: Primary data saved (29664 records). [2025-10-20T18:57:52.149131] [INFO ] [MAIN ] GFF: Full history saved. [2025-10-20T18:57:52.149210] [INFO ] [MAIN ] Update Progress: 918/2367 symbols completed. GFF took 25.9s. Estimated remaining: 3h 5m 56s. Rate: 7.8 symbols/min [2025-10-20T18:57:52.251460] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-08. [2025-10-20T18:57:53.848087] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T18:57:53.861300] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-09. [2025-10-20T18:57:53.942608] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-09 (2.49 seconds). [2025-10-20T18:57:53.954086] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-10. [2025-10-20T18:57:55.850839] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T18:57:55.863448] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-10. [2025-10-20T18:57:56.607944] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-10 (2.65 seconds). [2025-10-20T18:57:56.615848] [INFO ] [REMOTE ] GEV: Full fetch complete (77040 records). [2025-10-20T18:57:56.616011] [DEBUG] [VALIDATOR ] GEV: Validating primary data. [2025-10-20T18:57:57.004562] [DEBUG] [VALIDATOR ] GEV: All expected trading days are present. [2025-10-20T18:57:57.004663] [DEBUG] [VALIDATOR ] GEV: Primary data validation passed. [2025-10-20T18:57:57.005667] [INFO ] [MAIN ] GEV: Fetched primary data is valid. [2025-10-20T18:57:57.005734] [DEBUG] [LOCAL ] GEV: Saving primary data. [2025-10-20T18:57:57.007240] [DEBUG] [LOCAL ] GEV: Trimmed 77040 records to 52992 (keeping 200 days of data) [2025-10-20T18:57:57.338852] [DEBUG] [LOCAL ] GEV: Primary data saved (52992 records). [2025-10-20T18:57:57.338986] [INFO ] [MAIN ] GEV: Full history saved. [2025-10-20T18:57:57.339490] [INFO ] [MAIN ] Update Progress: 919/2367 symbols completed. GEV took 45.8s. Estimated remaining: 3h 5m 44s. Rate: 7.8 symbols/min [2025-10-20T18:57:58.074132] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T18:57:58.081898] [INFO ] [REMOTE ] GFI: Full fetch complete (76940 records). [2025-10-20T18:57:58.082013] [DEBUG] [VALIDATOR ] GFI: Validating primary data. [2025-10-20T18:57:58.472705] [DEBUG] [VALIDATOR ] GFI: All expected trading days are present. [2025-10-20T18:57:58.472789] [DEBUG] [VALIDATOR ] GFI: Primary data validation passed. [2025-10-20T18:57:58.473709] [INFO ] [MAIN ] GFI: Fetched primary data is valid. [2025-10-20T18:57:58.473773] [DEBUG] [LOCAL ] GFI: Saving primary data. [2025-10-20T18:57:58.475162] [DEBUG] [LOCAL ] GFI: Trimmed 76940 records to 53116 (keeping 200 days of data) [2025-10-20T18:57:58.795886] [DEBUG] [LOCAL ] GFI: Primary data saved (53116 records). [2025-10-20T18:57:58.796016] [INFO ] [MAIN ] GFI: Full history saved. [2025-10-20T18:57:58.796094] [INFO ] [MAIN ] Update Progress: 920/2367 symbols completed. GFI took 29.4s. Estimated remaining: 3h 5m 27s. Rate: 7.8 symbols/min [2025-10-20T18:57:59.371060] [INFO ] [MAIN ] GGG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:59.439494] [INFO ] [MAIN ] GFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:59.476927] [INFO ] [MAIN ] GFL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:59.503479] [INFO ] [MAIN ] GGB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:59.515460] [DEBUG] [LOCAL ] GFR: Loaded 7837 existing records from primary storage. [2025-10-20T18:57:59.515912] [INFO ] [MAIN ] GFR: Loaded 7837 existing records. [2025-10-20T18:57:59.752473] [DEBUG] [LOCAL ] GGG: Loaded 41252 existing records from primary storage. [2025-10-20T18:57:59.755668] [WARN ] [MAIN ] GFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:57:59.756235] [INFO ] [MAIN ] GGG: Loaded 41252 existing records. [2025-10-20T18:57:59.756139] [INFO ] [MAIN ] GFR: Wiping existing primary data before full refresh. [2025-10-20T18:57:59.756374] [INFO ] [LOCAL ] GFR: Starting deletion of primary data. [2025-10-20T18:57:59.841978] [DEBUG] [LOCAL ] GFR: Deleted primary data file. [2025-10-20T18:57:59.832688] [DEBUG] [LOCAL ] GGB: Loaded 52241 existing records from primary storage. [2025-10-20T18:57:59.860177] [INFO ] [MAIN ] GFR: Starting full primary refresh. [2025-10-20T18:57:59.917260] [DEBUG] [LOCAL ] GFL: Loaded 48336 existing records from primary storage. [2025-10-20T18:57:59.917501] [INFO ] [MAIN ] GGB: Loaded 52241 existing records. [2025-10-20T18:57:59.973190] [INFO ] [MAIN ] GFL: Loaded 48336 existing records. [2025-10-20T18:58:00.383246] [WARN ] [MAIN ] GGG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:00.414142] [INFO ] [MAIN ] GGG: Wiping existing primary data before full refresh. [2025-10-20T18:58:00.421115] [INFO ] [LOCAL ] GGG: Starting deletion of primary data. [2025-10-20T18:58:00.513517] [DEBUG] [LOCAL ] GGG: Deleted primary data file. [2025-10-20T18:58:00.561153] [INFO ] [MAIN ] GGG: Starting full primary refresh. [2025-10-20T18:58:00.683774] [WARN ] [MAIN ] GFL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:00.655554] [WARN ] [MAIN ] GGB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:00.771509] [INFO ] [MAIN ] GFL: Wiping existing primary data before full refresh. [2025-10-20T18:58:00.805400] [INFO ] [MAIN ] GGB: Wiping existing primary data before full refresh. [2025-10-20T18:58:00.819677] [INFO ] [LOCAL ] GGB: Starting deletion of primary data. [2025-10-20T18:58:00.824916] [INFO ] [LOCAL ] GFL: Starting deletion of primary data. [2025-10-20T18:58:00.836922] [DEBUG] [LOCAL ] GGB: Deleted primary data file. [2025-10-20T18:58:00.847704] [INFO ] [MAIN ] GGB: Starting full primary refresh. [2025-10-20T18:58:00.874073] [DEBUG] [LOCAL ] GFL: Deleted primary data file. [2025-10-20T18:58:00.884452] [INFO ] [MAIN ] GFL: Starting full primary refresh. [2025-10-20T18:58:01.025199] [INFO ] [REMOTE ] GFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:01.054254] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-01. [2025-10-20T18:58:01.711090] [INFO ] [REMOTE ] GGG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:01.734822] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-01. [2025-10-20T18:58:01.968376] [INFO ] [REMOTE ] GGB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:01.978605] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-01. [2025-10-20T18:58:01.998772] [INFO ] [REMOTE ] GFL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:01.999090] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-01. [2025-10-20T18:58:02.160173] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:58:02.162600] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-02. [2025-10-20T18:58:02.719805] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T18:58:02.722184] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-03. [2025-10-20T18:58:03.517761] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T18:58:03.526952] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-02. [2025-10-20T18:58:03.589729] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T18:58:03.599142] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-02. [2025-10-20T18:58:03.608436] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T18:58:03.612215] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-04. [2025-10-20T18:58:03.644715] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T18:58:03.655048] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-02. [2025-10-20T18:58:04.171207] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T18:58:04.174475] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-05. [2025-10-20T18:58:04.649421] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:58:04.657695] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-03. [2025-10-20T18:58:04.755888] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:58:04.945509] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T18:58:05.051341] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-06. [2025-10-20T18:58:05.288072] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:58:05.756831] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T18:58:05.851384] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-07. [2025-10-20T18:58:06.245097] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:58:06.317344] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T18:58:06.651422] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-03. [2025-10-20T18:58:07.451397] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-08. [2025-10-20T18:58:08.048025] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T18:58:08.262192] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-03. [2025-10-20T18:58:08.268614] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:58:09.051422] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-04. [2025-10-20T18:58:09.851383] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-09. [2025-10-20T18:58:10.012742] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:58:10.545941] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T18:58:10.651416] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-04. [2025-10-20T18:58:10.798239] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T18:58:11.451374] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-10. [2025-10-20T18:58:12.251379] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-04. [2025-10-20T18:58:12.259720] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T18:58:12.261293] [INFO ] [REMOTE ] GFR: Full fetch complete (12050 records). [2025-10-20T18:58:12.261373] [DEBUG] [VALIDATOR ] GFR: Validating primary data. [2025-10-20T18:58:12.604957] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T18:58:12.696111] [DEBUG] [VALIDATOR ] GFR: All expected trading days are present. [2025-10-20T18:58:12.696194] [DEBUG] [VALIDATOR ] GFR: Primary data validation passed. [2025-10-20T18:58:12.696565] [INFO ] [MAIN ] GFR: Fetched primary data is valid. [2025-10-20T18:58:12.696603] [DEBUG] [LOCAL ] GFR: Saving primary data. [2025-10-20T18:58:12.697054] [DEBUG] [LOCAL ] GFR: Trimmed 12050 records to 7510 (keeping 200 days of data) [2025-10-20T18:58:12.746470] [DEBUG] [LOCAL ] GFR: Primary data saved (7510 records). [2025-10-20T18:58:12.746719] [INFO ] [MAIN ] GFR: Full history saved. [2025-10-20T18:58:12.746833] [INFO ] [MAIN ] Update Progress: 921/2367 symbols completed. GFR took 14.0s. Estimated remaining: 3h 5m 29s. Rate: 7.8 symbols/min [2025-10-20T18:58:12.927471] [INFO ] [MAIN ] GGT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:12.927674] [DEBUG] [LOCAL ] GGT: No existing data file found in primary storage. [2025-10-20T18:58:12.927728] [WARN ] [MAIN ] GGT: No existing data found, performing full refresh instead. [2025-10-20T18:58:12.927836] [INFO ] [MAIN ] GGT: Wiping existing primary data before full refresh. [2025-10-20T18:58:12.927877] [INFO ] [LOCAL ] GGT: Starting deletion of primary data. [2025-10-20T18:58:12.927917] [DEBUG] [LOCAL ] GGT: No primary data file found to delete. [2025-10-20T18:58:12.927948] [INFO ] [MAIN ] GGT: Starting full primary refresh. [2025-10-20T18:58:13.066367] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-05. [2025-10-20T18:58:13.285620] [INFO ] [REMOTE ] GGT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:13.851386] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-01. [2025-10-20T18:58:13.997403] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T18:58:14.644492] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:58:14.652859] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-05. [2025-10-20T18:58:14.659556] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T18:58:15.451411] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-06. [2025-10-20T18:58:16.251419] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-02. [2025-10-20T18:58:16.369387] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T18:58:16.915334] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T18:58:17.047674] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T18:58:17.051324] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-07. [2025-10-20T18:58:17.851437] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-06. [2025-10-20T18:58:18.651396] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-05. [2025-10-20T18:58:18.701872] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:58:19.451364] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-03. [2025-10-20T18:58:19.505155] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:58:20.019550] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T18:58:20.251368] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-07. [2025-10-20T18:58:20.335536] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:58:21.051376] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-04. [2025-10-20T18:58:21.851448] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-06. [2025-10-20T18:58:21.874329] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T18:58:22.129391] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T18:58:22.651454] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-08. [2025-10-20T18:58:23.331059] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:58:23.451356] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-07. [2025-10-20T18:58:23.519410] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T18:58:24.251417] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-08. [2025-10-20T18:58:24.699202] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T18:58:25.051364] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-08. [2025-10-20T18:58:25.846066] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:58:25.856400] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-05. [2025-10-20T18:58:26.625465] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:58:26.651444] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-09. [2025-10-20T18:58:26.692733] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T18:58:27.451400] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-06. [2025-10-20T18:58:28.077889] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T18:58:28.251371] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-09. [2025-10-20T18:58:28.358793] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T18:58:29.051362] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-07. [2025-10-20T18:58:29.851963] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:58:29.851627] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-10. [2025-10-20T18:58:29.956213] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T18:58:30.651403] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-08. [2025-10-20T18:58:31.439674] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T18:58:31.445501] [INFO ] [REMOTE ] GGG: Full fetch complete (59462 records). [2025-10-20T18:58:31.445691] [DEBUG] [VALIDATOR ] GGG: Validating primary data. [2025-10-20T18:58:31.457355] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-09. [2025-10-20T18:58:31.577151] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:58:31.852318] [DEBUG] [VALIDATOR ] GGG: All expected trading days are present. [2025-10-20T18:58:31.852404] [DEBUG] [VALIDATOR ] GGG: Primary data validation passed. [2025-10-20T18:58:31.853203] [INFO ] [MAIN ] GGG: Fetched primary data is valid. [2025-10-20T18:58:31.853264] [DEBUG] [LOCAL ] GGG: Saving primary data. [2025-10-20T18:58:31.854476] [DEBUG] [LOCAL ] GGG: Trimmed 59462 records to 40585 (keeping 200 days of data) [2025-10-20T18:58:32.105568] [DEBUG] [LOCAL ] GGG: Primary data saved (40585 records). [2025-10-20T18:58:32.105718] [INFO ] [MAIN ] GGG: Full history saved. [2025-10-20T18:58:32.105802] [INFO ] [MAIN ] Update Progress: 922/2367 symbols completed. GGG took 33.3s. Estimated remaining: 3h 5m 40s. Rate: 7.8 symbols/min [2025-10-20T18:58:32.257245] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-10. [2025-10-20T18:58:32.276980] [INFO ] [MAIN ] GGZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:32.281032] [DEBUG] [LOCAL ] GGZ: Loaded 2448 existing records from primary storage. [2025-10-20T18:58:32.281210] [INFO ] [MAIN ] GGZ: Loaded 2448 existing records. [2025-10-20T18:58:32.460281] [WARN ] [MAIN ] GGZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:32.460523] [INFO ] [MAIN ] GGZ: Wiping existing primary data before full refresh. [2025-10-20T18:58:32.460581] [INFO ] [LOCAL ] GGZ: Starting deletion of primary data. [2025-10-20T18:58:32.460736] [DEBUG] [LOCAL ] GGZ: Deleted primary data file. [2025-10-20T18:58:32.460794] [INFO ] [MAIN ] GGZ: Starting full primary refresh. [2025-10-20T18:58:32.813773] [INFO ] [REMOTE ] GGZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:33.051344] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-01. [2025-10-20T18:58:33.262034] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T18:58:33.688715] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T18:58:33.851352] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-02. [2025-10-20T18:58:34.131975] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:58:34.139506] [INFO ] [REMOTE ] GGB: Full fetch complete (74358 records). [2025-10-20T18:58:34.139664] [DEBUG] [VALIDATOR ] GGB: Validating primary data. [2025-10-20T18:58:34.243060] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T18:58:34.528396] [DEBUG] [VALIDATOR ] GGB: All expected trading days are present. [2025-10-20T18:58:34.528468] [DEBUG] [VALIDATOR ] GGB: Primary data validation passed. [2025-10-20T18:58:34.529380] [INFO ] [MAIN ] GGB: Fetched primary data is valid. [2025-10-20T18:58:34.529442] [DEBUG] [LOCAL ] GGB: Saving primary data. [2025-10-20T18:58:34.530808] [DEBUG] [LOCAL ] GGB: Trimmed 74358 records to 51536 (keeping 200 days of data) [2025-10-20T18:58:34.690236] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-10. [2025-10-20T18:58:34.821869] [DEBUG] [LOCAL ] GGB: Primary data saved (51536 records). [2025-10-20T18:58:34.821982] [INFO ] [MAIN ] GGB: Full history saved. [2025-10-20T18:58:34.822484] [INFO ] [MAIN ] Update Progress: 923/2367 symbols completed. GGB took 36.0s. Estimated remaining: 3h 5m 24s. Rate: 7.8 symbols/min [2025-10-20T18:58:34.991554] [INFO ] [MAIN ] GHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:34.997262] [DEBUG] [LOCAL ] GHC: Loaded 2977 existing records from primary storage. [2025-10-20T18:58:34.997449] [INFO ] [MAIN ] GHC: Loaded 2977 existing records. [2025-10-20T18:58:35.175528] [WARN ] [MAIN ] GHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:35.175951] [INFO ] [MAIN ] GHC: Wiping existing primary data before full refresh. [2025-10-20T18:58:35.176003] [INFO ] [LOCAL ] GHC: Starting deletion of primary data. [2025-10-20T18:58:35.176234] [DEBUG] [LOCAL ] GHC: Deleted primary data file. [2025-10-20T18:58:35.176289] [INFO ] [MAIN ] GHC: Starting full primary refresh. [2025-10-20T18:58:35.456572] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-09. [2025-10-20T18:58:35.538764] [INFO ] [REMOTE ] GHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:36.251373] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-01. [2025-10-20T18:58:36.368006] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T18:58:36.377512] [INFO ] [REMOTE ] GFL: Full fetch complete (69606 records). [2025-10-20T18:58:36.377746] [DEBUG] [VALIDATOR ] GFL: Validating primary data. [2025-10-20T18:58:36.512991] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T18:58:36.802480] [DEBUG] [VALIDATOR ] GFL: All expected trading days are present. [2025-10-20T18:58:36.805969] [DEBUG] [VALIDATOR ] GFL: Primary data validation passed. [2025-10-20T18:58:36.806084] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T18:58:36.807153] [INFO ] [MAIN ] GFL: Fetched primary data is valid. [2025-10-20T18:58:36.808364] [DEBUG] [LOCAL ] GFL: Saving primary data. [2025-10-20T18:58:36.809833] [DEBUG] [LOCAL ] GFL: Trimmed 69606 records to 47569 (keeping 200 days of data) [2025-10-20T18:58:37.064528] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-03. [2025-10-20T18:58:37.090444] [DEBUG] [LOCAL ] GFL: Primary data saved (47569 records). [2025-10-20T18:58:37.090752] [INFO ] [MAIN ] GFL: Full history saved. [2025-10-20T18:58:37.090842] [INFO ] [MAIN ] Update Progress: 924/2367 symbols completed. GFL took 38.3s. Estimated remaining: 3h 5m 8s. Rate: 7.8 symbols/min [2025-10-20T18:58:37.255274] [INFO ] [MAIN ] GHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:37.261090] [DEBUG] [LOCAL ] GHG: Loaded 3317 existing records from primary storage. [2025-10-20T18:58:37.261268] [INFO ] [MAIN ] GHG: Loaded 3317 existing records. [2025-10-20T18:58:37.437148] [WARN ] [MAIN ] GHG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:37.437332] [INFO ] [MAIN ] GHG: Wiping existing primary data before full refresh. [2025-10-20T18:58:37.437379] [INFO ] [LOCAL ] GHG: Starting deletion of primary data. [2025-10-20T18:58:37.437570] [DEBUG] [LOCAL ] GHG: Deleted primary data file. [2025-10-20T18:58:37.437625] [INFO ] [MAIN ] GHG: Starting full primary refresh. [2025-10-20T18:58:37.713546] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T18:58:37.789623] [INFO ] [REMOTE ] GHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:37.851345] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-01. [2025-10-20T18:58:38.182946] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T18:58:38.651365] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-04. [2025-10-20T18:58:39.261864] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:58:39.451382] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-02. [2025-10-20T18:58:39.813028] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:58:40.251369] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-10. [2025-10-20T18:58:41.051355] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-03. [2025-10-20T18:58:41.238123] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T18:58:41.241570] [INFO ] [REMOTE ] GGT: Full fetch complete (18982 records). [2025-10-20T18:58:41.241682] [DEBUG] [VALIDATOR ] GGT: Validating primary data. [2025-10-20T18:58:41.533026] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T18:58:41.618164] [DEBUG] [VALIDATOR ] GGT: All expected trading days are present. [2025-10-20T18:58:41.618249] [DEBUG] [VALIDATOR ] GGT: Primary data validation passed. [2025-10-20T18:58:41.618779] [INFO ] [MAIN ] GGT: Fetched primary data is valid. [2025-10-20T18:58:41.618841] [DEBUG] [LOCAL ] GGT: Saving primary data. [2025-10-20T18:58:41.619524] [DEBUG] [LOCAL ] GGT: Trimmed 18982 records to 14586 (keeping 200 days of data) [2025-10-20T18:58:41.711835] [DEBUG] [LOCAL ] GGT: Primary data saved (14586 records). [2025-10-20T18:58:41.711996] [INFO ] [MAIN ] GGT: Full history saved. [2025-10-20T18:58:41.712083] [INFO ] [MAIN ] Update Progress: 925/2367 symbols completed. GGT took 29.0s. Estimated remaining: 3h 4m 55s. Rate: 7.8 symbols/min [2025-10-20T18:58:41.854256] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-05. [2025-10-20T18:58:41.932014] [INFO ] [MAIN ] GHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:41.942209] [DEBUG] [LOCAL ] GHI: Loaded 7591 existing records from primary storage. [2025-10-20T18:58:41.942442] [INFO ] [MAIN ] GHI: Loaded 7591 existing records. [2025-10-20T18:58:42.129986] [WARN ] [MAIN ] GHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:42.131096] [INFO ] [MAIN ] GHI: Wiping existing primary data before full refresh. [2025-10-20T18:58:42.131158] [INFO ] [LOCAL ] GHI: Starting deletion of primary data. [2025-10-20T18:58:42.131911] [DEBUG] [LOCAL ] GHI: Deleted primary data file. [2025-10-20T18:58:42.131969] [INFO ] [MAIN ] GHI: Starting full primary refresh. [2025-10-20T18:58:42.457915] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T18:58:42.551075] [INFO ] [REMOTE ] GHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:42.651377] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-06. [2025-10-20T18:58:43.163302] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T18:58:43.451378] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-01. [2025-10-20T18:58:44.222832] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:58:44.251374] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-02. [2025-10-20T18:58:44.857082] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T18:58:45.051388] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-07. [2025-10-20T18:58:45.416422] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-10-20T18:58:45.851364] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-08. [2025-10-20T18:58:46.272442] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T18:58:46.651368] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-02. [2025-10-20T18:58:47.190366] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T18:58:47.451370] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-09. [2025-10-20T18:58:47.896146] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T18:58:48.251385] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-03. [2025-10-20T18:58:48.976832] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T18:58:49.051396] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-04. [2025-10-20T18:58:49.667502] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T18:58:49.851357] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-10. [2025-10-20T18:58:50.468366] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T18:58:50.469344] [INFO ] [REMOTE ] GGZ: Full fetch complete (3895 records). [2025-10-20T18:58:50.469413] [DEBUG] [VALIDATOR ] GGZ: Validating primary data. [2025-10-20T18:58:50.659156] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-04. [2025-10-20T18:58:50.831546] [DEBUG] [VALIDATOR ] GGZ: All expected trading days are present. [2025-10-20T18:58:50.831623] [DEBUG] [VALIDATOR ] GGZ: Primary data validation passed. [2025-10-20T18:58:50.831782] [INFO ] [MAIN ] GGZ: Fetched primary data is valid. [2025-10-20T18:58:50.831838] [DEBUG] [LOCAL ] GGZ: Saving primary data. [2025-10-20T18:58:50.832182] [DEBUG] [LOCAL ] GGZ: Trimmed 3895 records to 2412 (keeping 200 days of data) [2025-10-20T18:58:50.846787] [DEBUG] [LOCAL ] GGZ: Primary data saved (2412 records). [2025-10-20T18:58:50.846850] [INFO ] [MAIN ] GGZ: Full history saved. [2025-10-20T18:58:50.846914] [INFO ] [MAIN ] Update Progress: 926/2367 symbols completed. GGZ took 18.7s. Estimated remaining: 3h 4m 50s. Rate: 7.8 symbols/min [2025-10-20T18:58:51.012981] [INFO ] [MAIN ] GHLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:51.018673] [DEBUG] [LOCAL ] GHLD: Loaded 4348 existing records from primary storage. [2025-10-20T18:58:51.018855] [INFO ] [MAIN ] GHLD: Loaded 4348 existing records. [2025-10-20T18:58:51.192999] [WARN ] [MAIN ] GHLD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:51.193218] [INFO ] [MAIN ] GHLD: Wiping existing primary data before full refresh. [2025-10-20T18:58:51.193263] [INFO ] [LOCAL ] GHLD: Starting deletion of primary data. [2025-10-20T18:58:51.193417] [DEBUG] [LOCAL ] GHLD: Deleted primary data file. [2025-10-20T18:58:51.193471] [INFO ] [MAIN ] GHLD: Starting full primary refresh. [2025-10-20T18:58:51.455757] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-03. [2025-10-20T18:58:51.534982] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T18:58:51.564805] [INFO ] [REMOTE ] GHLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:51.995527] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T18:58:52.251421] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-01. [2025-10-20T18:58:52.701853] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T18:58:53.051412] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-04. [2025-10-20T18:58:53.605423] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T18:58:53.851429] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-05. [2025-10-20T18:58:54.475736] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T18:58:54.651432] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-05. [2025-10-20T18:58:55.167538] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T18:58:55.451369] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-06. [2025-10-20T18:58:56.122376] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T18:58:56.251371] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-05. [2025-10-20T18:58:56.684977] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T18:58:57.051378] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-06. [2025-10-20T18:58:57.492694] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T18:58:57.851367] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-07. [2025-10-20T18:58:58.472471] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-10-20T18:58:58.651359] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-08. [2025-10-20T18:58:59.134129] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T18:58:59.451366] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-02. [2025-10-20T18:58:59.810529] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:59:00.251374] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-06. [2025-10-20T18:59:00.891692] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T18:59:01.051414] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-07. [2025-10-20T18:59:01.851410] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-03. [2025-10-20T18:59:01.902134] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T18:59:02.207562] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-10-20T18:59:02.651405] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-07. [2025-10-20T18:59:03.161775] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T18:59:03.451409] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-09. [2025-10-20T18:59:03.883714] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-10-20T18:59:04.251404] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-04. [2025-10-20T18:59:04.852687] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T18:59:05.051377] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-08. [2025-10-20T18:59:05.789554] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T18:59:05.851382] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-09. [2025-10-20T18:59:06.651350] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-05. [2025-10-20T18:59:06.718280] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T18:59:07.140535] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T18:59:07.451387] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-10. [2025-10-20T18:59:08.070256] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T18:59:08.071328] [INFO ] [REMOTE ] GHG: Full fetch complete (4168 records). [2025-10-20T18:59:08.071396] [DEBUG] [VALIDATOR ] GHG: Validating primary data. [2025-10-20T18:59:08.257008] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-10. [2025-10-20T18:59:08.437166] [DEBUG] [VALIDATOR ] GHG: All expected trading days are present. [2025-10-20T18:59:08.437251] [DEBUG] [VALIDATOR ] GHG: Primary data validation passed. [2025-10-20T18:59:08.437393] [INFO ] [MAIN ] GHG: Fetched primary data is valid. [2025-10-20T18:59:08.437436] [DEBUG] [LOCAL ] GHG: Saving primary data. [2025-10-20T18:59:08.437849] [DEBUG] [LOCAL ] GHG: Trimmed 4168 records to 3282 (keeping 200 days of data) [2025-10-20T18:59:08.455772] [DEBUG] [LOCAL ] GHG: Primary data saved (3282 records). [2025-10-20T18:59:08.455842] [INFO ] [MAIN ] GHG: Full history saved. [2025-10-20T18:59:08.455911] [INFO ] [MAIN ] Update Progress: 927/2367 symbols completed. GHG took 31.4s. Estimated remaining: 3h 4m 58s. Rate: 7.8 symbols/min [2025-10-20T18:59:08.626460] [INFO ] [MAIN ] GHM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:08.641889] [DEBUG] [LOCAL ] GHM: Loaded 14580 existing records from primary storage. [2025-10-20T18:59:08.642116] [INFO ] [MAIN ] GHM: Loaded 14580 existing records. [2025-10-20T18:59:08.820118] [WARN ] [MAIN ] GHM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:08.820348] [INFO ] [MAIN ] GHM: Wiping existing primary data before full refresh. [2025-10-20T18:59:08.820398] [INFO ] [LOCAL ] GHM: Starting deletion of primary data. [2025-10-20T18:59:08.821241] [DEBUG] [LOCAL ] GHM: Deleted primary data file. [2025-10-20T18:59:08.821301] [INFO ] [MAIN ] GHM: Starting full primary refresh. [2025-10-20T18:59:09.059108] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-06. [2025-10-20T18:59:09.176020] [INFO ] [REMOTE ] GHM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:09.337827] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T18:59:09.340019] [INFO ] [REMOTE ] GHI: Full fetch complete (11152 records). [2025-10-20T18:59:09.340089] [DEBUG] [VALIDATOR ] GHI: Validating primary data. [2025-10-20T18:59:09.718865] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T18:59:09.719260] [DEBUG] [VALIDATOR ] GHI: All expected trading days are present. [2025-10-20T18:59:09.721618] [DEBUG] [VALIDATOR ] GHI: Primary data validation passed. [2025-10-20T18:59:09.722013] [INFO ] [MAIN ] GHI: Fetched primary data is valid. [2025-10-20T18:59:09.722063] [DEBUG] [LOCAL ] GHI: Saving primary data. [2025-10-20T18:59:09.722553] [DEBUG] [LOCAL ] GHI: Trimmed 11152 records to 7558 (keeping 200 days of data) [2025-10-20T18:59:09.767005] [DEBUG] [LOCAL ] GHI: Primary data saved (7558 records). [2025-10-20T18:59:09.767119] [INFO ] [MAIN ] GHI: Full history saved. [2025-10-20T18:59:09.767193] [INFO ] [MAIN ] Update Progress: 928/2367 symbols completed. GHI took 28.1s. Estimated remaining: 3h 4m 40s. Rate: 7.8 symbols/min [2025-10-20T18:59:09.856549] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-08. [2025-10-20T18:59:09.937753] [INFO ] [MAIN ] GHY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:09.959958] [DEBUG] [LOCAL ] GHY: Loaded 21330 existing records from primary storage. [2025-10-20T18:59:09.960206] [INFO ] [MAIN ] GHY: Loaded 21330 existing records. [2025-10-20T18:59:10.139109] [WARN ] [MAIN ] GHY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:10.139402] [INFO ] [MAIN ] GHY: Wiping existing primary data before full refresh. [2025-10-20T18:59:10.139454] [INFO ] [LOCAL ] GHY: Starting deletion of primary data. [2025-10-20T18:59:10.139691] [DEBUG] [LOCAL ] GHY: Deleted primary data file. [2025-10-20T18:59:10.139747] [INFO ] [MAIN ] GHY: Starting full primary refresh. [2025-10-20T18:59:10.407183] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T18:59:10.539336] [INFO ] [REMOTE ] GHY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:10.651409] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-01. [2025-10-20T18:59:11.451373] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-09. [2025-10-20T18:59:11.813481] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:59:12.047030] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T18:59:12.251365] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-07. [2025-10-20T18:59:12.822594] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T18:59:13.051388] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-10. [2025-10-20T18:59:13.839267] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T18:59:13.840455] [INFO ] [REMOTE ] GHC: Full fetch complete (3659 records). [2025-10-20T18:59:13.840523] [DEBUG] [VALIDATOR ] GHC: Validating primary data. [2025-10-20T18:59:13.861155] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-08. [2025-10-20T18:59:14.204821] [DEBUG] [VALIDATOR ] GHC: All expected trading days are present. [2025-10-20T18:59:14.204892] [DEBUG] [VALIDATOR ] GHC: Primary data validation passed. [2025-10-20T18:59:14.205018] [INFO ] [MAIN ] GHC: Fetched primary data is valid. [2025-10-20T18:59:14.205056] [DEBUG] [LOCAL ] GHC: Saving primary data. [2025-10-20T18:59:14.205402] [DEBUG] [LOCAL ] GHC: Trimmed 3659 records to 2956 (keeping 200 days of data) [2025-10-20T18:59:14.224686] [DEBUG] [LOCAL ] GHC: Primary data saved (2956 records). [2025-10-20T18:59:14.224757] [INFO ] [MAIN ] GHC: Full history saved. [2025-10-20T18:59:14.224824] [INFO ] [MAIN ] Update Progress: 929/2367 symbols completed. GHC took 39.4s. Estimated remaining: 3h 4m 27s. Rate: 7.8 symbols/min [2025-10-20T18:59:14.401012] [INFO ] [MAIN ] GIB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:14.433201] [DEBUG] [LOCAL ] GIB: Loaded 32143 existing records from primary storage. [2025-10-20T18:59:14.433496] [INFO ] [MAIN ] GIB: Loaded 32143 existing records. [2025-10-20T18:59:14.633354] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T18:59:14.627605] [WARN ] [MAIN ] GIB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:14.633687] [INFO ] [MAIN ] GIB: Wiping existing primary data before full refresh. [2025-10-20T18:59:14.636080] [INFO ] [LOCAL ] GIB: Starting deletion of primary data. [2025-10-20T18:59:14.636303] [DEBUG] [LOCAL ] GIB: Deleted primary data file. [2025-10-20T18:59:14.636359] [INFO ] [MAIN ] GIB: Starting full primary refresh. [2025-10-20T18:59:14.656503] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-09. [2025-10-20T18:59:15.008224] [INFO ] [REMOTE ] GIB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:15.307551] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T18:59:15.451372] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-10. [2025-10-20T18:59:16.172151] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T18:59:16.173344] [INFO ] [REMOTE ] GHLD: Full fetch complete (5084 records). [2025-10-20T18:59:16.173409] [DEBUG] [VALIDATOR ] GHLD: Validating primary data. [2025-10-20T18:59:16.256610] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-01. [2025-10-20T18:59:16.533314] [DEBUG] [VALIDATOR ] GHLD: All expected trading days are present. [2025-10-20T18:59:16.533391] [DEBUG] [VALIDATOR ] GHLD: Primary data validation passed. [2025-10-20T18:59:16.533567] [INFO ] [MAIN ] GHLD: Fetched primary data is valid. [2025-10-20T18:59:16.533605] [DEBUG] [LOCAL ] GHLD: Saving primary data. [2025-10-20T18:59:16.533993] [DEBUG] [LOCAL ] GHLD: Trimmed 5084 records to 4301 (keeping 200 days of data) [2025-10-20T18:59:16.558844] [DEBUG] [LOCAL ] GHLD: Primary data saved (4301 records). [2025-10-20T18:59:16.558918] [INFO ] [MAIN ] GHLD: Full history saved. [2025-10-20T18:59:16.559009] [INFO ] [MAIN ] Update Progress: 930/2367 symbols completed. GHLD took 25.7s. Estimated remaining: 3h 4m 11s. Rate: 7.8 symbols/min [2025-10-20T18:59:16.725507] [INFO ] [MAIN ] GIC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:16.739085] [DEBUG] [LOCAL ] GIC: Loaded 13587 existing records from primary storage. [2025-10-20T18:59:16.739277] [INFO ] [MAIN ] GIC: Loaded 13587 existing records. [2025-10-20T18:59:16.917763] [WARN ] [MAIN ] GIC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:16.918001] [INFO ] [MAIN ] GIC: Wiping existing primary data before full refresh. [2025-10-20T18:59:16.918047] [INFO ] [LOCAL ] GIC: Starting deletion of primary data. [2025-10-20T18:59:16.918247] [DEBUG] [LOCAL ] GIC: Deleted primary data file. [2025-10-20T18:59:16.918283] [INFO ] [MAIN ] GIC: Starting full primary refresh. [2025-10-20T18:59:17.058727] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-01. [2025-10-20T18:59:17.292168] [INFO ] [REMOTE ] GIC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:17.292850] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:59:17.851387] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-01. [2025-10-20T18:59:18.385826] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T18:59:18.651380] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-02. [2025-10-20T18:59:18.842187] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T18:59:19.451348] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-02. [2025-10-20T18:59:19.695190] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:59:20.251356] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-02. [2025-10-20T18:59:20.415756] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:59:21.051375] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-03. [2025-10-20T18:59:21.231035] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:59:21.851359] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-03. [2025-10-20T18:59:22.049678] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T18:59:22.651400] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-02. [2025-10-20T18:59:22.982492] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T18:59:23.451365] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-04. [2025-10-20T18:59:23.975784] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:59:24.251413] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-03. [2025-10-20T18:59:24.515772] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T18:59:25.051384] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-03. [2025-10-20T18:59:25.724314] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:59:25.771634] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T18:59:25.851391] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-05. [2025-10-20T18:59:26.651401] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-04. [2025-10-20T18:59:26.934717] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T18:59:27.451375] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-06. [2025-10-20T18:59:27.715715] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T18:59:28.251428] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-04. [2025-10-20T18:59:28.368943] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T18:59:29.051402] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-04. [2025-10-20T18:59:29.736884] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:59:29.851346] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-05. [2025-10-20T18:59:29.905366] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T18:59:30.651442] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-07. [2025-10-20T18:59:31.188337] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:59:31.451356] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-05. [2025-10-20T18:59:31.749560] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:59:32.251487] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-05. [2025-10-20T18:59:32.277700] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:59:33.050873] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T18:59:33.053661] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-08. [2025-10-20T18:59:33.851379] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-06. [2025-10-20T18:59:34.095393] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:59:34.651353] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-09. [2025-10-20T18:59:34.991074] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T18:59:35.451401] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-06. [2025-10-20T18:59:35.718804] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:59:36.251440] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-07. [2025-10-20T18:59:36.711506] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T18:59:37.051409] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-06. [2025-10-20T18:59:37.146425] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T18:59:37.851379] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-08. [2025-10-20T18:59:37.898024] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T18:59:38.651347] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-07. [2025-10-20T18:59:38.809939] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T18:59:39.451346] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-09. [2025-10-20T18:59:39.463573] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T18:59:40.251355] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-08. [2025-10-20T18:59:40.333022] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T18:59:41.051338] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-10. [2025-10-20T18:59:41.335883] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:59:41.851371] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-10. [2025-10-20T18:59:42.052698] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T18:59:42.055705] [INFO ] [REMOTE ] GHM: Full fetch complete (22030 records). [2025-10-20T18:59:42.055788] [DEBUG] [VALIDATOR ] GHM: Validating primary data. [2025-10-20T18:59:42.456816] [DEBUG] [VALIDATOR ] GHM: All expected trading days are present. [2025-10-20T18:59:42.456886] [DEBUG] [VALIDATOR ] GHM: Primary data validation passed. [2025-10-20T18:59:42.457449] [INFO ] [MAIN ] GHM: Fetched primary data is valid. [2025-10-20T18:59:42.457529] [DEBUG] [LOCAL ] GHM: Saving primary data. [2025-10-20T18:59:42.458218] [DEBUG] [LOCAL ] GHM: Trimmed 22030 records to 14369 (keeping 200 days of data) [2025-10-20T18:59:42.540365] [DEBUG] [LOCAL ] GHM: Primary data saved (14369 records). [2025-10-20T18:59:42.540494] [INFO ] [MAIN ] GHM: Full history saved. [2025-10-20T18:59:42.540569] [INFO ] [MAIN ] Update Progress: 931/2367 symbols completed. GHM took 34.1s. Estimated remaining: 3h 4m 32s. Rate: 7.8 symbols/min [2025-10-20T18:59:42.656570] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-09. [2025-10-20T18:59:42.708589] [INFO ] [MAIN ] GIL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:42.749140] [DEBUG] [LOCAL ] GIL: Loaded 44482 existing records from primary storage. [2025-10-20T18:59:42.749450] [INFO ] [MAIN ] GIL: Loaded 44482 existing records. [2025-10-20T18:59:42.926323] [WARN ] [MAIN ] GIL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:42.926505] [INFO ] [MAIN ] GIL: Wiping existing primary data before full refresh. [2025-10-20T18:59:42.926552] [INFO ] [LOCAL ] GIL: Starting deletion of primary data. [2025-10-20T18:59:42.926753] [DEBUG] [LOCAL ] GIL: Deleted primary data file. [2025-10-20T18:59:42.926814] [INFO ] [MAIN ] GIL: Starting full primary refresh. [2025-10-20T18:59:43.112591] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:59:43.196546] [INFO ] [REMOTE ] GHY: Full fetch complete (32455 records). [2025-10-20T18:59:43.212368] [DEBUG] [VALIDATOR ] GHY: Validating primary data. [2025-10-20T18:59:43.423165] [INFO ] [REMOTE ] GIL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:43.463816] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-07. [2025-10-20T18:59:43.700170] [DEBUG] [VALIDATOR ] GHY: All expected trading days are present. [2025-10-20T18:59:43.700251] [DEBUG] [VALIDATOR ] GHY: Primary data validation passed. [2025-10-20T18:59:43.701004] [INFO ] [MAIN ] GHY: Fetched primary data is valid. [2025-10-20T18:59:43.701066] [DEBUG] [LOCAL ] GHY: Saving primary data. [2025-10-20T18:59:43.702014] [DEBUG] [LOCAL ] GHY: Trimmed 32455 records to 20931 (keeping 200 days of data) [2025-10-20T18:59:43.795399] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T18:59:43.839921] [DEBUG] [LOCAL ] GHY: Primary data saved (20931 records). [2025-10-20T18:59:43.840060] [INFO ] [MAIN ] GHY: Full history saved. [2025-10-20T18:59:43.840137] [INFO ] [MAIN ] Update Progress: 932/2367 symbols completed. GHY took 34.1s. Estimated remaining: 3h 4m 14s. Rate: 7.8 symbols/min [2025-10-20T18:59:44.016866] [INFO ] [MAIN ] GIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:44.067447] [DEBUG] [LOCAL ] GIS: Loaded 54452 existing records from primary storage. [2025-10-20T18:59:44.067809] [INFO ] [MAIN ] GIS: Loaded 54452 existing records. [2025-10-20T18:59:44.241693] [WARN ] [MAIN ] GIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:44.241900] [INFO ] [MAIN ] GIS: Wiping existing primary data before full refresh. [2025-10-20T18:59:44.241946] [INFO ] [LOCAL ] GIS: Starting deletion of primary data. [2025-10-20T18:59:44.242137] [DEBUG] [LOCAL ] GIS: Deleted primary data file. [2025-10-20T18:59:44.242174] [INFO ] [MAIN ] GIS: Starting full primary refresh. [2025-10-20T18:59:44.256614] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-10. [2025-10-20T18:59:44.611608] [INFO ] [REMOTE ] GIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:44.904383] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:59:45.051357] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-01. [2025-10-20T18:59:45.167459] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T18:59:45.169607] [INFO ] [REMOTE ] GIC: Full fetch complete (19871 records). [2025-10-20T18:59:45.169733] [DEBUG] [VALIDATOR ] GIC: Validating primary data. [2025-10-20T18:59:45.527903] [DEBUG] [VALIDATOR ] GIC: All expected trading days are present. [2025-10-20T18:59:45.527978] [DEBUG] [VALIDATOR ] GIC: Primary data validation passed. [2025-10-20T18:59:45.528389] [INFO ] [MAIN ] GIC: Fetched primary data is valid. [2025-10-20T18:59:45.528427] [DEBUG] [LOCAL ] GIC: Saving primary data. [2025-10-20T18:59:45.529008] [DEBUG] [LOCAL ] GIC: Trimmed 19871 records to 13388 (keeping 200 days of data) [2025-10-20T18:59:45.609443] [DEBUG] [LOCAL ] GIC: Primary data saved (13388 records). [2025-10-20T18:59:45.609589] [INFO ] [MAIN ] GIC: Full history saved. [2025-10-20T18:59:45.609697] [INFO ] [MAIN ] Update Progress: 933/2367 symbols completed. GIC took 29.1s. Estimated remaining: 3h 3m 57s. Rate: 7.8 symbols/min [2025-10-20T18:59:45.782177] [INFO ] [MAIN ] GKOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:45.822749] [DEBUG] [LOCAL ] GKOS: Loaded 43111 existing records from primary storage. [2025-10-20T18:59:45.823081] [INFO ] [MAIN ] GKOS: Loaded 43111 existing records. [2025-10-20T18:59:45.858050] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-08. [2025-10-20T18:59:46.017264] [WARN ] [MAIN ] GKOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:46.017502] [INFO ] [MAIN ] GKOS: Wiping existing primary data before full refresh. [2025-10-20T18:59:46.017549] [INFO ] [LOCAL ] GKOS: Starting deletion of primary data. [2025-10-20T18:59:46.017971] [DEBUG] [LOCAL ] GKOS: Deleted primary data file. [2025-10-20T18:59:46.018026] [INFO ] [MAIN ] GKOS: Starting full primary refresh. [2025-10-20T18:59:46.366703] [INFO ] [REMOTE ] GKOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:46.651375] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-01. [2025-10-20T18:59:47.057691] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T18:59:47.339761] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:59:47.451372] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-09. [2025-10-20T18:59:48.168935] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T18:59:48.251373] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-02. [2025-10-20T18:59:48.778195] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T18:59:49.051416] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-01. [2025-10-20T18:59:49.851367] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-02. [2025-10-20T18:59:50.079525] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T18:59:50.279403] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T18:59:50.651387] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-10. [2025-10-20T18:59:51.192817] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T18:59:51.451379] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-03. [2025-10-20T18:59:52.106848] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:59:52.111557] [INFO ] [REMOTE ] GIB: Full fetch complete (46256 records). [2025-10-20T18:59:52.111687] [DEBUG] [VALIDATOR ] GIB: Validating primary data. [2025-10-20T18:59:52.256496] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-03. [2025-10-20T18:59:52.492918] [DEBUG] [VALIDATOR ] GIB: All expected trading days are present. [2025-10-20T18:59:52.492999] [DEBUG] [VALIDATOR ] GIB: Primary data validation passed. [2025-10-20T18:59:52.493727] [INFO ] [MAIN ] GIB: Fetched primary data is valid. [2025-10-20T18:59:52.493787] [DEBUG] [LOCAL ] GIB: Saving primary data. [2025-10-20T18:59:52.494746] [DEBUG] [LOCAL ] GIB: Trimmed 46256 records to 31496 (keeping 200 days of data) [2025-10-20T18:59:52.683731] [DEBUG] [LOCAL ] GIB: Primary data saved (31496 records). [2025-10-20T18:59:52.683848] [INFO ] [MAIN ] GIB: Full history saved. [2025-10-20T18:59:52.684425] [INFO ] [MAIN ] Update Progress: 934/2367 symbols completed. GIB took 38.5s. Estimated remaining: 3h 3m 49s. Rate: 7.8 symbols/min [2025-10-20T18:59:52.854438] [INFO ] [MAIN ] GL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:52.892826] [DEBUG] [LOCAL ] GL: Loaded 38100 existing records from primary storage. [2025-10-20T18:59:52.893185] [INFO ] [MAIN ] GL: Loaded 38100 existing records. [2025-10-20T18:59:53.065814] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-02. [2025-10-20T18:59:53.070564] [WARN ] [MAIN ] GL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:53.071951] [INFO ] [MAIN ] GL: Wiping existing primary data before full refresh. [2025-10-20T18:59:53.072109] [INFO ] [LOCAL ] GL: Starting deletion of primary data. [2025-10-20T18:59:53.072472] [DEBUG] [LOCAL ] GL: Deleted primary data file. [2025-10-20T18:59:53.072546] [INFO ] [MAIN ] GL: Starting full primary refresh. [2025-10-20T18:59:53.391997] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T18:59:53.484602] [INFO ] [REMOTE ] GL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:53.509313] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T18:59:53.851414] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-04. [2025-10-20T18:59:54.586004] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:59:54.651384] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-03. [2025-10-20T18:59:55.451372] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-04. [2025-10-20T18:59:55.506211] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:59:56.220317] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T18:59:56.251353] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-04. [2025-10-20T18:59:57.051388] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-01. [2025-10-20T18:59:57.480090] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T18:59:57.851437] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-05. [2025-10-20T18:59:57.895065] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T18:59:58.621863] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:59:58.651329] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-02. [2025-10-20T18:59:59.366941] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T18:59:59.451372] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-06. [2025-10-20T19:00:00.064408] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T19:00:00.251379] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-05. [2025-10-20T19:00:01.032018] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:00:01.051335] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-05. [2025-10-20T19:00:01.851362] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-03. [2025-10-20T19:00:02.147308] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:00:02.651393] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-07. [2025-10-20T19:00:03.447971] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:00:03.456231] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-06. [2025-10-20T19:00:04.251394] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-04. [2025-10-20T19:00:04.342888] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-05 (3.29 seconds). [2025-10-20T19:00:04.548930] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T19:00:05.051348] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-06. [2025-10-20T19:00:05.514979] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T19:00:05.851464] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-07. [2025-10-20T19:00:06.651495] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-08. [2025-10-20T19:00:07.314433] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-06 (2.26 seconds). [2025-10-20T19:00:07.351936] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-04 (3.10 seconds). [2025-10-20T19:00:07.451352] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-05. [2025-10-20T19:00:07.625988] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:00:08.251415] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-07. [2025-10-20T19:00:08.427828] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T19:00:08.872008] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T19:00:09.051443] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-06. [2025-10-20T19:00:09.851370] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-09. [2025-10-20T19:00:10.305073] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T19:00:10.476666] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-10-20T19:00:10.651364] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-08. [2025-10-20T19:00:11.436337] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T19:00:11.451335] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-10. [2025-10-20T19:00:11.966952] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T19:00:12.251354] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-09. [2025-10-20T19:00:12.665197] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T19:00:12.672044] [INFO ] [REMOTE ] GIL: Full fetch complete (62443 records). [2025-10-20T19:00:12.672164] [DEBUG] [VALIDATOR ] GIL: Validating primary data. [2025-10-20T19:00:13.054172] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-08. [2025-10-20T19:00:13.055378] [DEBUG] [VALIDATOR ] GIL: All expected trading days are present. [2025-10-20T19:00:13.055460] [DEBUG] [VALIDATOR ] GIL: Primary data validation passed. [2025-10-20T19:00:13.056293] [INFO ] [MAIN ] GIL: Fetched primary data is valid. [2025-10-20T19:00:13.056355] [DEBUG] [LOCAL ] GIL: Saving primary data. [2025-10-20T19:00:13.057573] [DEBUG] [LOCAL ] GIL: Trimmed 62443 records to 43989 (keeping 200 days of data) [2025-10-20T19:00:13.358815] [DEBUG] [LOCAL ] GIL: Primary data saved (43989 records). [2025-10-20T19:00:13.359118] [INFO ] [MAIN ] GIL: Full history saved. [2025-10-20T19:00:13.367611] [INFO ] [MAIN ] Update Progress: 935/2367 symbols completed. GIL took 30.8s. Estimated remaining: 3h 4m 1s. Rate: 7.8 symbols/min [2025-10-20T19:00:13.401484] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:00:13.573175] [INFO ] [MAIN ] GLOB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:13.619885] [DEBUG] [LOCAL ] GLOB: Loaded 48655 existing records from primary storage. [2025-10-20T19:00:13.620196] [INFO ] [MAIN ] GLOB: Loaded 48655 existing records. [2025-10-20T19:00:13.796476] [WARN ] [MAIN ] GLOB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:13.796639] [INFO ] [MAIN ] GLOB: Wiping existing primary data before full refresh. [2025-10-20T19:00:13.796732] [INFO ] [LOCAL ] GLOB: Starting deletion of primary data. [2025-10-20T19:00:13.796960] [DEBUG] [LOCAL ] GLOB: Deleted primary data file. [2025-10-20T19:00:13.797017] [INFO ] [MAIN ] GLOB: Starting full primary refresh. [2025-10-20T19:00:13.865738] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-10. [2025-10-20T19:00:14.187432] [INFO ] [REMOTE ] GLOB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:14.651349] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-07. [2025-10-20T19:00:15.186882] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T19:00:15.451362] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-09. [2025-10-20T19:00:16.251369] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-01. [2025-10-20T19:00:16.366467] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T19:00:17.051358] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-08. [2025-10-20T19:00:17.127573] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-10 (3.26 seconds). [2025-10-20T19:00:17.133799] [INFO ] [REMOTE ] GKOS: Full fetch complete (60424 records). [2025-10-20T19:00:17.133904] [DEBUG] [VALIDATOR ] GKOS: Validating primary data. [2025-10-20T19:00:17.530867] [DEBUG] [VALIDATOR ] GKOS: All expected trading days are present. [2025-10-20T19:00:17.531111] [DEBUG] [VALIDATOR ] GKOS: Primary data validation passed. [2025-10-20T19:00:17.532189] [INFO ] [MAIN ] GKOS: Fetched primary data is valid. [2025-10-20T19:00:17.532397] [DEBUG] [LOCAL ] GKOS: Saving primary data. [2025-10-20T19:00:17.534043] [DEBUG] [LOCAL ] GKOS: Trimmed 60424 records to 42502 (keeping 200 days of data) [2025-10-20T19:00:17.656150] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-10-20T19:00:17.806418] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:00:17.852844] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-10. [2025-10-20T19:00:17.858183] [DEBUG] [LOCAL ] GKOS: Primary data saved (42502 records). [2025-10-20T19:00:17.858324] [INFO ] [MAIN ] GKOS: Full history saved. [2025-10-20T19:00:17.858403] [INFO ] [MAIN ] Update Progress: 936/2367 symbols completed. GKOS took 32.2s. Estimated remaining: 3h 3m 48s. Rate: 7.8 symbols/min [2025-10-20T19:00:18.030694] [INFO ] [MAIN ] GLP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:18.041097] [DEBUG] [LOCAL ] GLP: Loaded 9431 existing records from primary storage. [2025-10-20T19:00:18.041366] [INFO ] [MAIN ] GLP: Loaded 9431 existing records. [2025-10-20T19:00:18.226047] [WARN ] [MAIN ] GLP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:18.226495] [INFO ] [MAIN ] GLP: Wiping existing primary data before full refresh. [2025-10-20T19:00:18.226600] [INFO ] [LOCAL ] GLP: Starting deletion of primary data. [2025-10-20T19:00:18.226957] [DEBUG] [LOCAL ] GLP: Deleted primary data file. [2025-10-20T19:00:18.227062] [INFO ] [MAIN ] GLP: Starting full primary refresh. [2025-10-20T19:00:18.358862] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:00:18.614533] [INFO ] [REMOTE ] GLP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:18.651378] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-01. [2025-10-20T19:00:19.439067] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:00:19.451319] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-02. [2025-10-20T19:00:20.229628] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:00:20.251346] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-03. [2025-10-20T19:00:20.796282] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-10 (2.94 seconds). [2025-10-20T19:00:20.803816] [INFO ] [REMOTE ] GIS: Full fetch complete (77451 records). [2025-10-20T19:00:20.803939] [DEBUG] [VALIDATOR ] GIS: Validating primary data. [2025-10-20T19:00:20.804793] [FATAL] [VALIDATOR ] GIS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T19:00:20.804881] [FATAL] [MAIN ] GIS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:00:20.804925] [WARN ] [MAIN ] GIS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:00:20.805005] [INFO ] [LOCAL ] GIS: Starting deletion of primary data. [2025-10-20T19:00:20.805079] [DEBUG] [LOCAL ] GIS: No primary data file found to delete. [2025-10-20T19:00:20.805158] [INFO ] [LOCAL ] GIS: Starting deletion of derivative data. [2025-10-20T19:00:20.805341] [DEBUG] [LOCAL ] GIS: Deleted derivative data file. [2025-10-20T19:00:20.805430] [INFO ] [MAIN ] Update Progress: 937/2367 symbols completed. GIS took 37.0s. Estimated remaining: 3h 3m 33s. Rate: 7.8 symbols/min [2025-10-20T19:00:20.986458] [INFO ] [MAIN ] GLW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:21.035313] [DEBUG] [LOCAL ] GLW: Loaded 54416 existing records from primary storage. [2025-10-20T19:00:21.035623] [INFO ] [MAIN ] GLW: Loaded 54416 existing records. [2025-10-20T19:00:21.056604] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-09. [2025-10-20T19:00:21.158543] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T19:00:21.226919] [WARN ] [MAIN ] GLW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:21.227118] [INFO ] [MAIN ] GLW: Wiping existing primary data before full refresh. [2025-10-20T19:00:21.227164] [INFO ] [LOCAL ] GLW: Starting deletion of primary data. [2025-10-20T19:00:21.227354] [DEBUG] [LOCAL ] GLW: Deleted primary data file. [2025-10-20T19:00:21.227391] [INFO ] [MAIN ] GLW: Starting full primary refresh. [2025-10-20T19:00:21.573636] [INFO ] [REMOTE ] GLW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:21.851390] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-04. [2025-10-20T19:00:22.587771] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T19:00:22.616959] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T19:00:22.651341] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-05. [2025-10-20T19:00:23.451365] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-02. [2025-10-20T19:00:23.566085] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T19:00:24.251493] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-06. [2025-10-20T19:00:24.800222] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T19:00:25.025716] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T19:00:25.051333] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-07. [2025-10-20T19:00:25.750664] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T19:00:25.851352] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-08. [2025-10-20T19:00:26.651406] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-10. [2025-10-20T19:00:26.707200] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T19:00:27.451404] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-03. [2025-10-20T19:00:28.099272] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T19:00:28.104823] [INFO ] [REMOTE ] GL: Full fetch complete (55054 records). [2025-10-20T19:00:28.104917] [DEBUG] [VALIDATOR ] GL: Validating primary data. [2025-10-20T19:00:28.277178] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-09. [2025-10-20T19:00:28.486712] [DEBUG] [VALIDATOR ] GL: All expected trading days are present. [2025-10-20T19:00:28.486833] [DEBUG] [VALIDATOR ] GL: Primary data validation passed. [2025-10-20T19:00:28.487588] [INFO ] [MAIN ] GL: Fetched primary data is valid. [2025-10-20T19:00:28.487629] [DEBUG] [LOCAL ] GL: Saving primary data. [2025-10-20T19:00:28.488825] [DEBUG] [LOCAL ] GL: Trimmed 55054 records to 37483 (keeping 200 days of data) [2025-10-20T19:00:28.763966] [DEBUG] [LOCAL ] GL: Primary data saved (37483 records). [2025-10-20T19:00:28.764123] [INFO ] [MAIN ] GL: Full history saved. [2025-10-20T19:00:28.764749] [INFO ] [MAIN ] Update Progress: 938/2367 symbols completed. GL took 36.1s. Estimated remaining: 3h 3m 26s. Rate: 7.8 symbols/min [2025-10-20T19:00:29.008697] [INFO ] [MAIN ] GM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:29.065818] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-01. [2025-10-20T19:00:29.072164] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:00:29.135898] [DEBUG] [LOCAL ] GM: Loaded 54509 existing records from primary storage. [2025-10-20T19:00:29.136294] [INFO ] [MAIN ] GM: Loaded 54509 existing records. [2025-10-20T19:00:29.314724] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T19:00:29.429177] [WARN ] [MAIN ] GM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:29.429429] [INFO ] [MAIN ] GM: Wiping existing primary data before full refresh. [2025-10-20T19:00:29.429479] [INFO ] [LOCAL ] GM: Starting deletion of primary data. [2025-10-20T19:00:29.429753] [DEBUG] [LOCAL ] GM: Deleted primary data file. [2025-10-20T19:00:29.429811] [INFO ] [MAIN ] GM: Starting full primary refresh. [2025-10-20T19:00:29.856623] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-04. [2025-10-20T19:00:29.967188] [INFO ] [REMOTE ] GM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:30.651360] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-10. [2025-10-20T19:00:30.950569] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T19:00:31.069085] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T19:00:31.445319] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T19:00:31.447873] [INFO ] [REMOTE ] GLP: Full fetch complete (14304 records). [2025-10-20T19:00:31.448141] [DEBUG] [VALIDATOR ] GLP: Validating primary data. [2025-10-20T19:00:31.460495] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-01. [2025-10-20T19:00:31.885980] [DEBUG] [VALIDATOR ] GLP: All expected trading days are present. [2025-10-20T19:00:31.886345] [DEBUG] [VALIDATOR ] GLP: Primary data validation passed. [2025-10-20T19:00:31.887235] [INFO ] [MAIN ] GLP: Fetched primary data is valid. [2025-10-20T19:00:31.887307] [DEBUG] [LOCAL ] GLP: Saving primary data. [2025-10-20T19:00:31.888204] [DEBUG] [LOCAL ] GLP: Trimmed 14304 records to 9318 (keeping 200 days of data) [2025-10-20T19:00:31.954964] [DEBUG] [LOCAL ] GLP: Primary data saved (9318 records). [2025-10-20T19:00:31.955124] [INFO ] [MAIN ] GLP: Full history saved. [2025-10-20T19:00:31.955209] [INFO ] [MAIN ] Update Progress: 939/2367 symbols completed. GLP took 14.1s. Estimated remaining: 3h 3m 11s. Rate: 7.8 symbols/min [2025-10-20T19:00:32.124520] [INFO ] [MAIN ] GME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:32.172610] [DEBUG] [LOCAL ] GME: Loaded 54561 existing records from primary storage. [2025-10-20T19:00:32.172947] [INFO ] [MAIN ] GME: Loaded 54561 existing records. [2025-10-20T19:00:32.259799] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-02. [2025-10-20T19:00:32.346023] [WARN ] [MAIN ] GME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:32.346205] [INFO ] [MAIN ] GME: Wiping existing primary data before full refresh. [2025-10-20T19:00:32.346252] [INFO ] [LOCAL ] GME: Starting deletion of primary data. [2025-10-20T19:00:32.346954] [DEBUG] [LOCAL ] GME: Deleted primary data file. [2025-10-20T19:00:32.347013] [INFO ] [MAIN ] GME: Starting full primary refresh. [2025-10-20T19:00:32.686122] [INFO ] [REMOTE ] GME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:33.051445] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-01. [2025-10-20T19:00:33.761197] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-01 (2.30 seconds). [2025-10-20T19:00:33.833507] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:00:33.851343] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-02. [2025-10-20T19:00:34.651432] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-03. [2025-10-20T19:00:35.444245] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-01 (2.39 seconds). [2025-10-20T19:00:35.455709] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-05. [2025-10-20T19:00:35.773170] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-10-20T19:00:36.251357] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-02. [2025-10-20T19:00:36.380632] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:00:37.051371] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-04. [2025-10-20T19:00:37.279338] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T19:00:37.851427] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-03. [2025-10-20T19:00:38.651380] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-06. [2025-10-20T19:00:38.782207] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-02 (2.53 seconds). [2025-10-20T19:00:39.090686] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T19:00:39.451338] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-05. [2025-10-20T19:00:40.015664] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-03 (2.16 seconds). [2025-10-20T19:00:40.116577] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:00:40.251376] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-04. [2025-10-20T19:00:41.051448] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-03. [2025-10-20T19:00:41.390056] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T19:00:41.851356] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-06. [2025-10-20T19:00:42.538819] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T19:00:42.651360] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-05. [2025-10-20T19:00:43.451347] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-07. [2025-10-20T19:00:43.593304] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-03 (2.54 seconds). [2025-10-20T19:00:43.637546] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T19:00:44.251349] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-07. [2025-10-20T19:00:44.724718] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-10-20T19:00:45.051369] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-06. [2025-10-20T19:00:45.177423] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T19:00:45.851376] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-08. [2025-10-20T19:00:46.046893] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T19:00:46.651417] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-08. [2025-10-20T19:00:46.890510] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T19:00:47.451347] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-04. [2025-10-20T19:00:47.646434] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T19:00:48.251334] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-09. [2025-10-20T19:00:48.645357] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T19:00:49.051451] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-07. [2025-10-20T19:00:49.851400] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-09. [2025-10-20T19:00:50.010321] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T19:00:50.080535] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-04 (2.63 seconds). [2025-10-20T19:00:50.651376] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-10. [2025-10-20T19:00:50.875772] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T19:00:51.451403] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-05. [2025-10-20T19:00:52.057024] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T19:00:52.090539] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T19:00:52.098044] [INFO ] [REMOTE ] GLOB: Full fetch complete (65972 records). [2025-10-20T19:00:52.098198] [DEBUG] [VALIDATOR ] GLOB: Validating primary data. [2025-10-20T19:00:52.261671] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-08. [2025-10-20T19:00:52.501175] [DEBUG] [VALIDATOR ] GLOB: All expected trading days are present. [2025-10-20T19:00:52.501257] [DEBUG] [VALIDATOR ] GLOB: Primary data validation passed. [2025-10-20T19:00:52.502173] [INFO ] [MAIN ] GLOB: Fetched primary data is valid. [2025-10-20T19:00:52.502237] [DEBUG] [LOCAL ] GLOB: Saving primary data. [2025-10-20T19:00:52.503517] [DEBUG] [LOCAL ] GLOB: Trimmed 65972 records to 48058 (keeping 200 days of data) [2025-10-20T19:00:52.785496] [DEBUG] [LOCAL ] GLOB: Primary data saved (48058 records). [2025-10-20T19:00:52.785674] [INFO ] [MAIN ] GLOB: Full history saved. [2025-10-20T19:00:52.786534] [INFO ] [MAIN ] Update Progress: 940/2367 symbols completed. GLOB took 39.4s. Estimated remaining: 3h 3m 24s. Rate: 7.8 symbols/min [2025-10-20T19:00:52.961381] [INFO ] [MAIN ] GMED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:53.007446] [DEBUG] [LOCAL ] GMED: Loaded 48880 existing records from primary storage. [2025-10-20T19:00:53.007796] [INFO ] [MAIN ] GMED: Loaded 48880 existing records. [2025-10-20T19:00:53.071675] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-10. [2025-10-20T19:00:53.189851] [WARN ] [MAIN ] GMED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:53.190089] [INFO ] [MAIN ] GMED: Wiping existing primary data before full refresh. [2025-10-20T19:00:53.190136] [INFO ] [LOCAL ] GMED: Starting deletion of primary data. [2025-10-20T19:00:53.190347] [DEBUG] [LOCAL ] GMED: Deleted primary data file. [2025-10-20T19:00:53.190403] [INFO ] [MAIN ] GMED: Starting full primary refresh. [2025-10-20T19:00:53.553768] [INFO ] [REMOTE ] GMED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:53.781974] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-05 (2.33 seconds). [2025-10-20T19:00:53.851411] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-06. [2025-10-20T19:00:53.957760] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T19:00:53.968516] [INFO ] [REMOTE ] GLW: Full fetch complete (77794 records). [2025-10-20T19:00:53.968713] [DEBUG] [VALIDATOR ] GLW: Validating primary data. [2025-10-20T19:00:54.431505] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-08 (2.17 seconds). [2025-10-20T19:00:54.483144] [DEBUG] [VALIDATOR ] GLW: All expected trading days are present. [2025-10-20T19:00:54.483269] [DEBUG] [VALIDATOR ] GLW: Primary data validation passed. [2025-10-20T19:00:54.484263] [INFO ] [MAIN ] GLW: Fetched primary data is valid. [2025-10-20T19:00:54.484330] [DEBUG] [LOCAL ] GLW: Saving primary data. [2025-10-20T19:00:54.485903] [DEBUG] [LOCAL ] GLW: Trimmed 77794 records to 53635 (keeping 200 days of data) [2025-10-20T19:00:54.660326] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-01. [2025-10-20T19:00:54.833055] [DEBUG] [LOCAL ] GLW: Primary data saved (53635 records). [2025-10-20T19:00:54.833194] [INFO ] [MAIN ] GLW: Full history saved. [2025-10-20T19:00:54.833271] [INFO ] [MAIN ] Update Progress: 941/2367 symbols completed. GLW took 34.0s. Estimated remaining: 3h 3m 7s. Rate: 7.8 symbols/min [2025-10-20T19:00:55.000382] [INFO ] [MAIN ] GMRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:55.034914] [DEBUG] [LOCAL ] GMRE: Loaded 37414 existing records from primary storage. [2025-10-20T19:00:55.035197] [INFO ] [MAIN ] GMRE: Loaded 37414 existing records. [2025-10-20T19:00:55.208796] [WARN ] [MAIN ] GMRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:55.209054] [INFO ] [MAIN ] GMRE: Wiping existing primary data before full refresh. [2025-10-20T19:00:55.209111] [INFO ] [LOCAL ] GMRE: Starting deletion of primary data. [2025-10-20T19:00:55.209320] [DEBUG] [LOCAL ] GMRE: Deleted primary data file. [2025-10-20T19:00:55.209377] [INFO ] [MAIN ] GMRE: Starting full primary refresh. [2025-10-20T19:00:55.458452] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-09. [2025-10-20T19:00:55.585845] [INFO ] [REMOTE ] GMRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:56.251346] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-01. [2025-10-20T19:00:56.349437] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:00:56.393751] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-06 (2.54 seconds). [2025-10-20T19:00:57.051377] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-07. [2025-10-20T19:00:57.494738] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T19:00:57.534415] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T19:00:57.851417] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-10. [2025-10-20T19:00:58.651379] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-02. [2025-10-20T19:00:58.764897] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T19:00:58.773231] [INFO ] [REMOTE ] GM: Full fetch complete (77907 records). [2025-10-20T19:00:58.773368] [DEBUG] [VALIDATOR ] GM: Validating primary data. [2025-10-20T19:00:59.176742] [DEBUG] [VALIDATOR ] GM: All expected trading days are present. [2025-10-20T19:00:59.176819] [DEBUG] [VALIDATOR ] GM: Primary data validation passed. [2025-10-20T19:00:59.177717] [INFO ] [MAIN ] GM: Fetched primary data is valid. [2025-10-20T19:00:59.177780] [DEBUG] [LOCAL ] GM: Saving primary data. [2025-10-20T19:00:59.179212] [DEBUG] [LOCAL ] GM: Trimmed 77907 records to 53729 (keeping 200 days of data) [2025-10-20T19:00:59.455814] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-02. [2025-10-20T19:00:59.510905] [DEBUG] [LOCAL ] GM: Primary data saved (53729 records). [2025-10-20T19:00:59.511050] [INFO ] [MAIN ] GM: Full history saved. [2025-10-20T19:00:59.511528] [INFO ] [MAIN ] Update Progress: 942/2367 symbols completed. GM took 30.7s. Estimated remaining: 3h 2m 55s. Rate: 7.8 symbols/min [2025-10-20T19:00:59.680618] [INFO ] [MAIN ] GNE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:59.702160] [DEBUG] [LOCAL ] GNE: Loaded 18212 existing records from primary storage. [2025-10-20T19:00:59.702411] [INFO ] [MAIN ] GNE: Loaded 18212 existing records. [2025-10-20T19:00:59.860182] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-07 (2.81 seconds). [2025-10-20T19:00:59.941165] [WARN ] [MAIN ] GNE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:59.941944] [INFO ] [MAIN ] GNE: Wiping existing primary data before full refresh. [2025-10-20T19:00:59.942037] [INFO ] [LOCAL ] GNE: Starting deletion of primary data. [2025-10-20T19:00:59.942233] [DEBUG] [LOCAL ] GNE: Deleted primary data file. [2025-10-20T19:00:59.942339] [INFO ] [MAIN ] GNE: Starting full primary refresh. [2025-10-20T19:01:00.061034] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T19:01:00.256585] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-08. [2025-10-20T19:01:00.340902] [INFO ] [REMOTE ] GNE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:01.051412] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-03. [2025-10-20T19:01:01.383422] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T19:01:01.851387] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-03. [2025-10-20T19:01:02.651427] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-01. [2025-10-20T19:01:03.127600] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-08 (2.87 seconds). [2025-10-20T19:01:03.304264] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-03 (2.25 seconds). [2025-10-20T19:01:03.356316] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T19:01:03.451383] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-09. [2025-10-20T19:01:03.678506] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T19:01:04.251424] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-04. [2025-10-20T19:01:05.051393] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-04. [2025-10-20T19:01:05.851380] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-02. [2025-10-20T19:01:06.003339] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:01:06.134268] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-09 (2.68 seconds). [2025-10-20T19:01:06.385768] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T19:01:06.497692] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T19:01:06.651388] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-05. [2025-10-20T19:01:07.451383] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-03. [2025-10-20T19:01:08.211703] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T19:01:08.251341] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-06. [2025-10-20T19:01:08.320228] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:01:09.051344] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-10. [2025-10-20T19:01:09.572421] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T19:01:09.851429] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-07. [2025-10-20T19:01:10.651372] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-04. [2025-10-20T19:01:11.366670] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T19:01:11.451338] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-08. [2025-10-20T19:01:11.514587] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T19:01:11.641873] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-10 (2.59 seconds). [2025-10-20T19:01:11.656554] [INFO ] [REMOTE ] GME: Full fetch complete (77923 records). [2025-10-20T19:01:11.656797] [DEBUG] [VALIDATOR ] GME: Validating primary data. [2025-10-20T19:01:12.049923] [DEBUG] [VALIDATOR ] GME: All expected trading days are present. [2025-10-20T19:01:12.049998] [DEBUG] [VALIDATOR ] GME: Primary data validation passed. [2025-10-20T19:01:12.050956] [INFO ] [MAIN ] GME: Fetched primary data is valid. [2025-10-20T19:01:12.051020] [DEBUG] [LOCAL ] GME: Saving primary data. [2025-10-20T19:01:12.052476] [DEBUG] [LOCAL ] GME: Trimmed 77923 records to 53781 (keeping 200 days of data) [2025-10-20T19:01:12.256942] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-05. [2025-10-20T19:01:12.371293] [DEBUG] [LOCAL ] GME: Primary data saved (53781 records). [2025-10-20T19:01:12.371418] [INFO ] [MAIN ] GME: Full history saved. [2025-10-20T19:01:12.371493] [INFO ] [MAIN ] Update Progress: 943/2367 symbols completed. GME took 40.4s. Estimated remaining: 3h 2m 55s. Rate: 7.8 symbols/min [2025-10-20T19:01:12.550154] [INFO ] [MAIN ] GNK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:12.586055] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T19:01:12.630295] [DEBUG] [LOCAL ] GNK: Loaded 40529 existing records from primary storage. [2025-10-20T19:01:12.630583] [INFO ] [MAIN ] GNK: Loaded 40529 existing records. [2025-10-20T19:01:12.812257] [WARN ] [MAIN ] GNK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:12.812477] [INFO ] [MAIN ] GNK: Wiping existing primary data before full refresh. [2025-10-20T19:01:12.812525] [INFO ] [LOCAL ] GNK: Starting deletion of primary data. [2025-10-20T19:01:12.812768] [DEBUG] [LOCAL ] GNK: Deleted primary data file. [2025-10-20T19:01:12.812826] [INFO ] [MAIN ] GNK: Starting full primary refresh. [2025-10-20T19:01:13.053302] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-05. [2025-10-20T19:01:13.169615] [INFO ] [REMOTE ] GNK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:13.259272] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:01:13.851435] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-01. [2025-10-20T19:01:14.651427] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-06. [2025-10-20T19:01:14.821532] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T19:01:15.283180] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T19:01:15.451359] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-06. [2025-10-20T19:01:15.772316] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:01:16.251378] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-07. [2025-10-20T19:01:17.062052] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-02. [2025-10-20T19:01:17.067350] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T19:01:17.384593] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T19:01:17.851394] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-07. [2025-10-20T19:01:18.132302] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T19:01:18.651418] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-03. [2025-10-20T19:01:19.418628] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:01:19.451435] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-08. [2025-10-20T19:01:20.232481] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:01:20.251330] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-04. [2025-10-20T19:01:20.523919] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T19:01:21.051383] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-08. [2025-10-20T19:01:21.851452] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-09. [2025-10-20T19:01:21.969440] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T19:01:22.527600] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:01:22.651449] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-05. [2025-10-20T19:01:23.247717] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T19:01:23.451384] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-09. [2025-10-20T19:01:24.213790] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T19:01:24.251363] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-10. [2025-10-20T19:01:24.281595] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T19:01:25.051448] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-09. [2025-10-20T19:01:25.715058] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:01:25.719253] [INFO ] [REMOTE ] GMRE: Full fetch complete (52255 records). [2025-10-20T19:01:25.719341] [DEBUG] [VALIDATOR ] GMRE: Validating primary data. [2025-10-20T19:01:25.861772] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-06. [2025-10-20T19:01:26.109888] [DEBUG] [VALIDATOR ] GMRE: All expected trading days are present. [2025-10-20T19:01:26.109977] [DEBUG] [VALIDATOR ] GMRE: Primary data validation passed. [2025-10-20T19:01:26.110748] [INFO ] [MAIN ] GMRE: Fetched primary data is valid. [2025-10-20T19:01:26.110813] [DEBUG] [LOCAL ] GMRE: Saving primary data. [2025-10-20T19:01:26.111970] [DEBUG] [LOCAL ] GMRE: Trimmed 52255 records to 36823 (keeping 200 days of data) [2025-10-20T19:01:26.333504] [DEBUG] [LOCAL ] GMRE: Primary data saved (36823 records). [2025-10-20T19:01:26.333669] [INFO ] [MAIN ] GMRE: Full history saved. [2025-10-20T19:01:26.333769] [INFO ] [MAIN ] Update Progress: 944/2367 symbols completed. GMRE took 31.5s. Estimated remaining: 3h 2m 57s. Rate: 7.8 symbols/min [2025-10-20T19:01:26.506680] [INFO ] [MAIN ] GNL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:26.548241] [DEBUG] [LOCAL ] GNL: Loaded 47399 existing records from primary storage. [2025-10-20T19:01:26.548571] [INFO ] [MAIN ] GNL: Loaded 47399 existing records. [2025-10-20T19:01:26.656594] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-10. [2025-10-20T19:01:26.767818] [WARN ] [MAIN ] GNL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:26.768426] [INFO ] [MAIN ] GNL: Wiping existing primary data before full refresh. [2025-10-20T19:01:26.770705] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T19:01:26.768837] [INFO ] [LOCAL ] GNL: Starting deletion of primary data. [2025-10-20T19:01:26.781572] [DEBUG] [LOCAL ] GNL: Deleted primary data file. [2025-10-20T19:01:26.782096] [INFO ] [MAIN ] GNL: Starting full primary refresh. [2025-10-20T19:01:27.142533] [INFO ] [REMOTE ] GNL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:27.259674] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T19:01:27.444721] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T19:01:27.447211] [INFO ] [REMOTE ] GNE: Full fetch complete (22560 records). [2025-10-20T19:01:27.447300] [DEBUG] [VALIDATOR ] GNE: Validating primary data. [2025-10-20T19:01:27.458949] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-07. [2025-10-20T19:01:27.816682] [DEBUG] [VALIDATOR ] GNE: All expected trading days are present. [2025-10-20T19:01:27.816798] [DEBUG] [VALIDATOR ] GNE: Primary data validation passed. [2025-10-20T19:01:27.817269] [INFO ] [MAIN ] GNE: Fetched primary data is valid. [2025-10-20T19:01:27.817308] [DEBUG] [LOCAL ] GNE: Saving primary data. [2025-10-20T19:01:27.817942] [DEBUG] [LOCAL ] GNE: Trimmed 22560 records to 18091 (keeping 200 days of data) [2025-10-20T19:01:27.922296] [DEBUG] [LOCAL ] GNE: Primary data saved (18091 records). [2025-10-20T19:01:27.922433] [INFO ] [MAIN ] GNE: Full history saved. [2025-10-20T19:01:27.922513] [INFO ] [MAIN ] Update Progress: 945/2367 symbols completed. GNE took 28.4s. Estimated remaining: 3h 2m 40s. Rate: 7.8 symbols/min [2025-10-20T19:01:28.087385] [INFO ] [MAIN ] GNRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:28.130867] [DEBUG] [LOCAL ] GNRC: Loaded 46194 existing records from primary storage. [2025-10-20T19:01:28.131188] [INFO ] [MAIN ] GNRC: Loaded 46194 existing records. [2025-10-20T19:01:28.256489] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-01. [2025-10-20T19:01:28.306886] [WARN ] [MAIN ] GNRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:28.307193] [INFO ] [MAIN ] GNRC: Wiping existing primary data before full refresh. [2025-10-20T19:01:28.307238] [INFO ] [LOCAL ] GNRC: Starting deletion of primary data. [2025-10-20T19:01:28.307442] [DEBUG] [LOCAL ] GNRC: Deleted primary data file. [2025-10-20T19:01:28.307498] [INFO ] [MAIN ] GNRC: Starting full primary refresh. [2025-10-20T19:01:28.659974] [INFO ] [REMOTE ] GNRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:28.905019] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T19:01:29.051415] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-10. [2025-10-20T19:01:29.851414] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-01. [2025-10-20T19:01:29.939831] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T19:01:30.565707] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T19:01:30.574573] [INFO ] [REMOTE ] GMED: Full fetch complete (70065 records). [2025-10-20T19:01:30.575007] [DEBUG] [VALIDATOR ] GMED: Validating primary data. [2025-10-20T19:01:30.656595] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-08. [2025-10-20T19:01:31.006677] [DEBUG] [VALIDATOR ] GMED: All expected trading days are present. [2025-10-20T19:01:31.006789] [DEBUG] [VALIDATOR ] GMED: Primary data validation passed. [2025-10-20T19:01:31.007734] [INFO ] [MAIN ] GMED: Fetched primary data is valid. [2025-10-20T19:01:31.007800] [DEBUG] [LOCAL ] GMED: Saving primary data. [2025-10-20T19:01:31.009364] [DEBUG] [LOCAL ] GMED: Trimmed 70065 records to 48170 (keeping 200 days of data) [2025-10-20T19:01:31.295797] [DEBUG] [LOCAL ] GMED: Primary data saved (48170 records). [2025-10-20T19:01:31.295949] [INFO ] [MAIN ] GMED: Full history saved. [2025-10-20T19:01:31.296856] [INFO ] [MAIN ] Update Progress: 946/2367 symbols completed. GMED took 38.5s. Estimated remaining: 3h 2m 26s. Rate: 7.8 symbols/min [2025-10-20T19:01:31.454718] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-02. [2025-10-20T19:01:31.475258] [INFO ] [MAIN ] GNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:31.506745] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T19:01:31.520391] [DEBUG] [LOCAL ] GNT: Loaded 8665 existing records from primary storage. [2025-10-20T19:01:31.520703] [INFO ] [MAIN ] GNT: Loaded 8665 existing records. [2025-10-20T19:01:31.711021] [WARN ] [MAIN ] GNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:31.711239] [INFO ] [MAIN ] GNT: Wiping existing primary data before full refresh. [2025-10-20T19:01:31.711287] [INFO ] [LOCAL ] GNT: Starting deletion of primary data. [2025-10-20T19:01:31.711461] [DEBUG] [LOCAL ] GNT: Deleted primary data file. [2025-10-20T19:01:31.711501] [INFO ] [MAIN ] GNT: Starting full primary refresh. [2025-10-20T19:01:32.081733] [INFO ] [REMOTE ] GNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:32.145246] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T19:01:32.251366] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-09. [2025-10-20T19:01:32.681736] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:01:33.051389] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-02. [2025-10-20T19:01:33.218790] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:01:33.851383] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-10. [2025-10-20T19:01:34.651433] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-01. [2025-10-20T19:01:34.687921] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:01:35.255413] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T19:01:35.312294] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:01:35.317982] [INFO ] [REMOTE ] GNK: Full fetch complete (57317 records). [2025-10-20T19:01:35.318076] [DEBUG] [VALIDATOR ] GNK: Validating primary data. [2025-10-20T19:01:35.456556] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-03. [2025-10-20T19:01:35.706724] [DEBUG] [VALIDATOR ] GNK: All expected trading days are present. [2025-10-20T19:01:35.706793] [DEBUG] [VALIDATOR ] GNK: Primary data validation passed. [2025-10-20T19:01:35.707543] [INFO ] [MAIN ] GNK: Fetched primary data is valid. [2025-10-20T19:01:35.707583] [DEBUG] [LOCAL ] GNK: Saving primary data. [2025-10-20T19:01:35.708609] [DEBUG] [LOCAL ] GNK: Trimmed 57317 records to 39773 (keeping 200 days of data) [2025-10-20T19:01:35.942567] [DEBUG] [LOCAL ] GNK: Primary data saved (39773 records). [2025-10-20T19:01:35.942725] [INFO ] [MAIN ] GNK: Full history saved. [2025-10-20T19:01:35.942811] [INFO ] [MAIN ] Update Progress: 947/2367 symbols completed. GNK took 23.6s. Estimated remaining: 3h 2m 13s. Rate: 7.8 symbols/min [2025-10-20T19:01:36.109473] [INFO ] [MAIN ] GNW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:36.155608] [DEBUG] [LOCAL ] GNW: Loaded 52153 existing records from primary storage. [2025-10-20T19:01:36.155927] [INFO ] [MAIN ] GNW: Loaded 52153 existing records. [2025-10-20T19:01:36.256968] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-02. [2025-10-20T19:01:36.337629] [WARN ] [MAIN ] GNW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:36.337829] [INFO ] [MAIN ] GNW: Wiping existing primary data before full refresh. [2025-10-20T19:01:36.337900] [INFO ] [LOCAL ] GNW: Starting deletion of primary data. [2025-10-20T19:01:36.338414] [DEBUG] [LOCAL ] GNW: Deleted primary data file. [2025-10-20T19:01:36.338500] [INFO ] [MAIN ] GNW: Starting full primary refresh. [2025-10-20T19:01:36.693259] [INFO ] [REMOTE ] GNW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:36.908292] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T19:01:37.051368] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-03. [2025-10-20T19:01:37.198542] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T19:01:37.851429] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-01. [2025-10-20T19:01:38.495771] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:01:38.651375] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-04. [2025-10-20T19:01:39.451372] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-04. [2025-10-20T19:01:39.536149] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T19:01:40.164988] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:01:40.251369] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-03. [2025-10-20T19:01:40.937586] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T19:01:41.051355] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-05. [2025-10-20T19:01:41.124249] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T19:01:41.851363] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-02. [2025-10-20T19:01:42.656584] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-05. [2025-10-20T19:01:42.668487] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T19:01:43.312816] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:01:43.451537] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-03. [2025-10-20T19:01:43.876907] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T19:01:44.251378] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-06. [2025-10-20T19:01:44.964429] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T19:01:45.051384] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-04. [2025-10-20T19:01:45.851368] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-06. [2025-10-20T19:01:45.988209] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T19:01:46.651378] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-07. [2025-10-20T19:01:47.126185] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-10-20T19:01:47.388859] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:01:47.451327] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-07. [2025-10-20T19:01:48.251371] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-05. [2025-10-20T19:01:48.306819] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T19:01:48.984251] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T19:01:49.051347] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-08. [2025-10-20T19:01:49.851439] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-08. [2025-10-20T19:01:49.904795] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:01:50.657941] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-06. [2025-10-20T19:01:50.661165] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T19:01:51.096014] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T19:01:51.451401] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-04. [2025-10-20T19:01:52.173193] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T19:01:52.251432] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-05. [2025-10-20T19:01:52.406184] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T19:01:52.662542] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T19:01:53.051428] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-06. [2025-10-20T19:01:53.851377] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-07. [2025-10-20T19:01:53.914793] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T19:01:54.651376] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-09. [2025-10-20T19:01:55.422765] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:01:55.451334] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-08. [2025-10-20T19:01:56.145234] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:01:56.251390] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-07. [2025-10-20T19:01:56.766307] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T19:01:57.024005] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:01:57.051336] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-09. [2025-10-20T19:01:57.851366] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-08. [2025-10-20T19:01:58.506956] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T19:01:58.651372] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-09. [2025-10-20T19:01:58.803750] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:01:59.451420] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-10. [2025-10-20T19:02:00.251448] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-10. [2025-10-20T19:02:00.813401] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-10-20T19:02:01.051428] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-10. [2025-10-20T19:02:01.851369] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-09. [2025-10-20T19:02:02.042840] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-10 (2.59 seconds). [2025-10-20T19:02:02.049623] [INFO ] [REMOTE ] GNRC: Full fetch complete (65644 records). [2025-10-20T19:02:02.049768] [DEBUG] [VALIDATOR ] GNRC: Validating primary data. [2025-10-20T19:02:02.431388] [DEBUG] [VALIDATOR ] GNRC: All expected trading days are present. [2025-10-20T19:02:02.431467] [DEBUG] [VALIDATOR ] GNRC: Primary data validation passed. [2025-10-20T19:02:02.432322] [INFO ] [MAIN ] GNRC: Fetched primary data is valid. [2025-10-20T19:02:02.432384] [DEBUG] [LOCAL ] GNRC: Saving primary data. [2025-10-20T19:02:02.433741] [DEBUG] [LOCAL ] GNRC: Trimmed 65644 records to 45450 (keeping 200 days of data) [2025-10-20T19:02:02.703203] [DEBUG] [LOCAL ] GNRC: Primary data saved (45450 records). [2025-10-20T19:02:02.703340] [INFO ] [MAIN ] GNRC: Full history saved. [2025-10-20T19:02:02.703902] [INFO ] [MAIN ] Update Progress: 948/2367 symbols completed. GNRC took 34.8s. Estimated remaining: 3h 2m 34s. Rate: 7.8 symbols/min [2025-10-20T19:02:02.874481] [INFO ] [MAIN ] GOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:02.918992] [DEBUG] [LOCAL ] GOF: Loaded 43353 existing records from primary storage. [2025-10-20T19:02:02.919348] [INFO ] [MAIN ] GOF: Loaded 43353 existing records. [2025-10-20T19:02:03.119513] [WARN ] [MAIN ] GOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:03.119990] [INFO ] [MAIN ] GOF: Wiping existing primary data before full refresh. [2025-10-20T19:02:03.120282] [INFO ] [LOCAL ] GOF: Starting deletion of primary data. [2025-10-20T19:02:03.139593] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T19:02:03.138901] [DEBUG] [LOCAL ] GOF: Deleted primary data file. [2025-10-20T19:02:03.160249] [INFO ] [MAIN ] GOF: Starting full primary refresh. [2025-10-20T19:02:03.161224] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-10. [2025-10-20T19:02:03.168496] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-10 (2.92 seconds). [2025-10-20T19:02:03.289862] [INFO ] [REMOTE ] GNW: Full fetch complete (73842 records). [2025-10-20T19:02:03.290098] [DEBUG] [VALIDATOR ] GNW: Validating primary data. [2025-10-20T19:02:03.836734] [INFO ] [REMOTE ] GOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:03.858517] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-01. [2025-10-20T19:02:04.028909] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-10 (2.98 seconds). [2025-10-20T19:02:04.058891] [DEBUG] [VALIDATOR ] GNW: All expected trading days are present. [2025-10-20T19:02:04.059536] [DEBUG] [VALIDATOR ] GNW: Primary data validation passed. [2025-10-20T19:02:04.061066] [INFO ] [MAIN ] GNW: Fetched primary data is valid. [2025-10-20T19:02:04.061255] [DEBUG] [LOCAL ] GNW: Saving primary data. [2025-10-20T19:02:04.062274] [INFO ] [REMOTE ] GNL: Full fetch complete (68211 records). [2025-10-20T19:02:04.062723] [DEBUG] [VALIDATOR ] GNL: Validating primary data. [2025-10-20T19:02:04.069186] [DEBUG] [LOCAL ] GNW: Trimmed 73842 records to 51392 (keeping 200 days of data) [2025-10-20T19:02:04.292168] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T19:02:04.301665] [INFO ] [REMOTE ] GNT: Full fetch complete (11345 records). [2025-10-20T19:02:04.319309] [DEBUG] [VALIDATOR ] GNT: Validating primary data. [2025-10-20T19:02:05.062870] [DEBUG] [VALIDATOR ] GNL: All expected trading days are present. [2025-10-20T19:02:05.127890] [DEBUG] [VALIDATOR ] GNL: Primary data validation passed. [2025-10-20T19:02:05.157716] [INFO ] [MAIN ] GNL: Fetched primary data is valid. [2025-10-20T19:02:05.172716] [DEBUG] [LOCAL ] GNL: Saving primary data. [2025-10-20T19:02:05.253630] [DEBUG] [VALIDATOR ] GNT: All expected trading days are present. [2025-10-20T19:02:05.255514] [DEBUG] [VALIDATOR ] GNT: Primary data validation passed. [2025-10-20T19:02:05.256181] [INFO ] [MAIN ] GNT: Fetched primary data is valid. [2025-10-20T19:02:05.256617] [DEBUG] [LOCAL ] GNT: Saving primary data. [2025-10-20T19:02:05.261102] [DEBUG] [LOCAL ] GNW: Primary data saved (51392 records). [2025-10-20T19:02:05.261539] [INFO ] [MAIN ] GNW: Full history saved. [2025-10-20T19:02:05.261307] [DEBUG] [LOCAL ] GNL: Trimmed 68211 records to 46630 (keeping 200 days of data) [2025-10-20T19:02:05.261853] [INFO ] [MAIN ] Update Progress: 949/2367 symbols completed. GNW took 29.3s. Estimated remaining: 3h 2m 19s. Rate: 7.8 symbols/min [2025-10-20T19:02:05.269324] [DEBUG] [LOCAL ] GNT: Trimmed 11345 records to 8645 (keeping 200 days of data) [2025-10-20T19:02:05.652041] [DEBUG] [LOCAL ] GNT: Primary data saved (8645 records). [2025-10-20T19:02:05.661595] [INFO ] [MAIN ] GNT: Full history saved. [2025-10-20T19:02:05.668835] [INFO ] [MAIN ] Update Progress: 950/2367 symbols completed. GNT took 34.4s. Estimated remaining: 3h 2m 0s. Rate: 7.8 symbols/min [2025-10-20T19:02:05.862422] [INFO ] [MAIN ] GOLF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:05.953327] [DEBUG] [LOCAL ] GNL: Primary data saved (46630 records). [2025-10-20T19:02:05.958848] [INFO ] [MAIN ] GNL: Full history saved. [2025-10-20T19:02:05.967382] [INFO ] [MAIN ] Update Progress: 951/2367 symbols completed. GNL took 39.6s. Estimated remaining: 3h 1m 42s. Rate: 7.8 symbols/min [2025-10-20T19:02:06.176778] [INFO ] [MAIN ] GOOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:06.179115] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-01 (2.32 seconds). [2025-10-20T19:02:06.237027] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-02. [2025-10-20T19:02:06.274417] [DEBUG] [LOCAL ] GOLF: Loaded 31259 existing records from primary storage. [2025-10-20T19:02:06.312955] [INFO ] [MAIN ] GOLF: Loaded 31259 existing records. [2025-10-20T19:02:06.312624] [INFO ] [MAIN ] GOTU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:06.472582] [DEBUG] [LOCAL ] GOOS: Loaded 45143 existing records from primary storage. [2025-10-20T19:02:06.518623] [INFO ] [MAIN ] GOOS: Loaded 45143 existing records. [2025-10-20T19:02:06.570199] [DEBUG] [LOCAL ] GOTU: Loaded 37702 existing records from primary storage. [2025-10-20T19:02:06.609859] [INFO ] [MAIN ] GOTU: Loaded 37702 existing records. [2025-10-20T19:02:06.604110] [WARN ] [MAIN ] GOLF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:06.688554] [INFO ] [MAIN ] GOLF: Wiping existing primary data before full refresh. [2025-10-20T19:02:06.694612] [INFO ] [LOCAL ] GOLF: Starting deletion of primary data. [2025-10-20T19:02:06.725354] [DEBUG] [LOCAL ] GOLF: Deleted primary data file. [2025-10-20T19:02:06.736206] [INFO ] [MAIN ] GOLF: Starting full primary refresh. [2025-10-20T19:02:07.046564] [WARN ] [MAIN ] GOTU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:07.079758] [INFO ] [MAIN ] GOTU: Wiping existing primary data before full refresh. [2025-10-20T19:02:07.079427] [WARN ] [MAIN ] GOOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:07.080273] [INFO ] [LOCAL ] GOTU: Starting deletion of primary data. [2025-10-20T19:02:07.080157] [INFO ] [MAIN ] GOOS: Wiping existing primary data before full refresh. [2025-10-20T19:02:07.091526] [INFO ] [LOCAL ] GOOS: Starting deletion of primary data. [2025-10-20T19:02:07.091405] [DEBUG] [LOCAL ] GOTU: Deleted primary data file. [2025-10-20T19:02:07.102051] [INFO ] [MAIN ] GOTU: Starting full primary refresh. [2025-10-20T19:02:07.177812] [DEBUG] [LOCAL ] GOOS: Deleted primary data file. [2025-10-20T19:02:07.240973] [INFO ] [MAIN ] GOOS: Starting full primary refresh. [2025-10-20T19:02:07.875955] [INFO ] [REMOTE ] GOLF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:07.983735] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-01. [2025-10-20T19:02:08.104498] [INFO ] [REMOTE ] GOTU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:08.118203] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-01. [2025-10-20T19:02:08.225485] [INFO ] [REMOTE ] GOOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:08.226559] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-01. [2025-10-20T19:02:08.227422] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-10-20T19:02:08.237253] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-03. [2025-10-20T19:02:09.277699] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T19:02:09.284297] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-02. [2025-10-20T19:02:09.741680] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:02:09.751921] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-02. [2025-10-20T19:02:09.829236] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:02:09.851332] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-02. [2025-10-20T19:02:09.939287] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T19:02:10.241966] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T19:02:10.651393] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-04. [2025-10-20T19:02:10.983940] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:02:11.067315] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T19:02:11.451390] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-03. [2025-10-20T19:02:12.251367] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-03. [2025-10-20T19:02:12.327414] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:02:13.051397] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-05. [2025-10-20T19:02:13.129459] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T19:02:13.562895] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T19:02:13.851378] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-04. [2025-10-20T19:02:14.651384] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-04. [2025-10-20T19:02:14.710271] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T19:02:15.264438] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:02:15.451417] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-03. [2025-10-20T19:02:16.112733] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T19:02:16.251394] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-05. [2025-10-20T19:02:16.707577] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T19:02:17.051432] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-06. [2025-10-20T19:02:17.737930] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T19:02:17.851386] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-05. [2025-10-20T19:02:18.597889] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T19:02:18.651359] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-04. [2025-10-20T19:02:19.451416] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-06. [2025-10-20T19:02:19.504500] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:02:20.137414] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T19:02:20.251368] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-05. [2025-10-20T19:02:20.686769] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T19:02:21.051354] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-07. [2025-10-20T19:02:21.851359] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-07. [2025-10-20T19:02:21.942876] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:02:22.651351] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-06. [2025-10-20T19:02:22.806553] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:02:23.227069] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T19:02:23.451398] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-06. [2025-10-20T19:02:24.180575] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:02:24.251395] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-07. [2025-10-20T19:02:24.860555] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T19:02:25.051413] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-07. [2025-10-20T19:02:25.758907] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T19:02:25.851393] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-08. [2025-10-20T19:02:26.362383] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:02:26.651434] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-08. [2025-10-20T19:02:27.432764] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:02:27.451345] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-09. [2025-10-20T19:02:28.043581] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T19:02:28.251368] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-08. [2025-10-20T19:02:29.041537] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:02:29.051337] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-10. [2025-10-20T19:02:29.267716] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T19:02:29.851383] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-09. [2025-10-20T19:02:30.651469] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-09. [2025-10-20T19:02:30.949062] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T19:02:30.955903] [INFO ] [REMOTE ] GOF: Full fetch complete (62523 records). [2025-10-20T19:02:30.956060] [DEBUG] [VALIDATOR ] GOF: Validating primary data. [2025-10-20T19:02:31.347810] [DEBUG] [VALIDATOR ] GOF: All expected trading days are present. [2025-10-20T19:02:31.348077] [DEBUG] [VALIDATOR ] GOF: Primary data validation passed. [2025-10-20T19:02:31.361619] [INFO ] [MAIN ] GOF: Fetched primary data is valid. [2025-10-20T19:02:31.367732] [DEBUG] [LOCAL ] GOF: Saving primary data. [2025-10-20T19:02:31.374031] [DEBUG] [LOCAL ] GOF: Trimmed 62523 records to 42707 (keeping 200 days of data) [2025-10-20T19:02:31.373879] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:02:31.468285] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-08. [2025-10-20T19:02:31.635967] [DEBUG] [LOCAL ] GOF: Primary data saved (42707 records). [2025-10-20T19:02:31.636088] [INFO ] [MAIN ] GOF: Full history saved. [2025-10-20T19:02:31.636618] [INFO ] [MAIN ] Update Progress: 952/2367 symbols completed. GOF took 28.9s. Estimated remaining: 3h 2m 1s. Rate: 7.8 symbols/min [2025-10-20T19:02:31.806092] [INFO ] [MAIN ] GPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:31.853135] [DEBUG] [LOCAL ] GPC: Loaded 46707 existing records from primary storage. [2025-10-20T19:02:31.853409] [INFO ] [MAIN ] GPC: Loaded 46707 existing records. [2025-10-20T19:02:32.035154] [WARN ] [MAIN ] GPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:32.035485] [INFO ] [MAIN ] GPC: Wiping existing primary data before full refresh. [2025-10-20T19:02:32.035671] [INFO ] [LOCAL ] GPC: Starting deletion of primary data. [2025-10-20T19:02:32.036262] [DEBUG] [LOCAL ] GPC: Deleted primary data file. [2025-10-20T19:02:32.036320] [INFO ] [MAIN ] GPC: Starting full primary refresh. [2025-10-20T19:02:32.256290] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-10. [2025-10-20T19:02:32.397854] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:02:32.427786] [INFO ] [REMOTE ] GPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:32.847816] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T19:02:33.051341] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-01. [2025-10-20T19:02:33.396201] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T19:02:33.401257] [INFO ] [REMOTE ] GOTU: Full fetch complete (58405 records). [2025-10-20T19:02:33.401365] [DEBUG] [VALIDATOR ] GOTU: Validating primary data. [2025-10-20T19:02:33.783792] [DEBUG] [VALIDATOR ] GOTU: All expected trading days are present. [2025-10-20T19:02:33.783867] [DEBUG] [VALIDATOR ] GOTU: Primary data validation passed. [2025-10-20T19:02:33.784690] [INFO ] [MAIN ] GOTU: Fetched primary data is valid. [2025-10-20T19:02:33.784751] [DEBUG] [LOCAL ] GOTU: Saving primary data. [2025-10-20T19:02:33.785845] [DEBUG] [LOCAL ] GOTU: Trimmed 58405 records to 36945 (keeping 200 days of data) [2025-10-20T19:02:33.863461] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-10. [2025-10-20T19:02:34.012295] [DEBUG] [LOCAL ] GOTU: Primary data saved (36945 records). [2025-10-20T19:02:34.012538] [INFO ] [MAIN ] GOTU: Full history saved. [2025-10-20T19:02:34.012685] [INFO ] [MAIN ] Update Progress: 953/2367 symbols completed. GOTU took 28.0s. Estimated remaining: 3h 1m 45s. Rate: 7.8 symbols/min [2025-10-20T19:02:34.183395] [INFO ] [MAIN ] GPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:34.200410] [DEBUG] [LOCAL ] GPI: Loaded 15097 existing records from primary storage. [2025-10-20T19:02:34.200667] [INFO ] [MAIN ] GPI: Loaded 15097 existing records. [2025-10-20T19:02:34.383011] [WARN ] [MAIN ] GPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:34.383197] [INFO ] [MAIN ] GPI: Wiping existing primary data before full refresh. [2025-10-20T19:02:34.383244] [INFO ] [LOCAL ] GPI: Starting deletion of primary data. [2025-10-20T19:02:34.383585] [DEBUG] [LOCAL ] GPI: Deleted primary data file. [2025-10-20T19:02:34.383624] [INFO ] [MAIN ] GPI: Starting full primary refresh. [2025-10-20T19:02:34.656796] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-09. [2025-10-20T19:02:34.775587] [INFO ] [REMOTE ] GPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:34.808284] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:02:35.316097] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T19:02:35.324813] [INFO ] [REMOTE ] GOOS: Full fetch complete (64765 records). [2025-10-20T19:02:35.324995] [DEBUG] [VALIDATOR ] GOOS: Validating primary data. [2025-10-20T19:02:35.453238] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-01. [2025-10-20T19:02:35.752884] [DEBUG] [VALIDATOR ] GOOS: All expected trading days are present. [2025-10-20T19:02:35.753145] [DEBUG] [VALIDATOR ] GOOS: Primary data validation passed. [2025-10-20T19:02:35.754358] [INFO ] [MAIN ] GOOS: Fetched primary data is valid. [2025-10-20T19:02:35.754508] [DEBUG] [LOCAL ] GOOS: Saving primary data. [2025-10-20T19:02:35.755932] [DEBUG] [LOCAL ] GOOS: Trimmed 64765 records to 44284 (keeping 200 days of data) [2025-10-20T19:02:35.871454] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T19:02:36.057475] [DEBUG] [LOCAL ] GOOS: Primary data saved (44284 records). [2025-10-20T19:02:36.057619] [INFO ] [MAIN ] GOOS: Full history saved. [2025-10-20T19:02:36.057739] [INFO ] [MAIN ] Update Progress: 954/2367 symbols completed. GOOS took 30.4s. Estimated remaining: 3h 1m 29s. Rate: 7.8 symbols/min [2025-10-20T19:02:36.253767] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-02. [2025-10-20T19:02:36.281832] [INFO ] [MAIN ] GPJA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:36.289389] [DEBUG] [LOCAL ] GPJA: Loaded 4230 existing records from primary storage. [2025-10-20T19:02:36.289739] [INFO ] [MAIN ] GPJA: Loaded 4230 existing records. [2025-10-20T19:02:36.478107] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T19:02:36.558498] [WARN ] [MAIN ] GPJA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:36.558750] [INFO ] [MAIN ] GPJA: Wiping existing primary data before full refresh. [2025-10-20T19:02:36.558806] [INFO ] [LOCAL ] GPJA: Starting deletion of primary data. [2025-10-20T19:02:36.559028] [DEBUG] [LOCAL ] GPJA: Deleted primary data file. [2025-10-20T19:02:36.559085] [INFO ] [MAIN ] GPJA: Starting full primary refresh. [2025-10-20T19:02:36.944950] [INFO ] [REMOTE ] GPJA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:37.051378] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-02. [2025-10-20T19:02:37.851571] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-10. [2025-10-20T19:02:37.857602] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T19:02:37.983225] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T19:02:38.651390] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-03. [2025-10-20T19:02:39.132921] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T19:02:39.137103] [INFO ] [REMOTE ] GOLF: Full fetch complete (45348 records). [2025-10-20T19:02:39.137181] [DEBUG] [VALIDATOR ] GOLF: Validating primary data. [2025-10-20T19:02:39.456515] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-03. [2025-10-20T19:02:39.520602] [DEBUG] [VALIDATOR ] GOLF: All expected trading days are present. [2025-10-20T19:02:39.520717] [DEBUG] [VALIDATOR ] GOLF: Primary data validation passed. [2025-10-20T19:02:39.521502] [INFO ] [MAIN ] GOLF: Fetched primary data is valid. [2025-10-20T19:02:39.521543] [DEBUG] [LOCAL ] GOLF: Saving primary data. [2025-10-20T19:02:39.522548] [DEBUG] [LOCAL ] GOLF: Trimmed 45348 records to 30669 (keeping 200 days of data) [2025-10-20T19:02:39.709856] [DEBUG] [LOCAL ] GOLF: Primary data saved (30669 records). [2025-10-20T19:02:39.709954] [INFO ] [MAIN ] GOLF: Full history saved. [2025-10-20T19:02:39.710032] [INFO ] [MAIN ] Update Progress: 955/2367 symbols completed. GOLF took 34.4s. Estimated remaining: 3h 1m 15s. Rate: 7.8 symbols/min [2025-10-20T19:02:39.899844] [INFO ] [MAIN ] GPK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:39.901122] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T19:02:39.954846] [DEBUG] [LOCAL ] GPK: Loaded 53548 existing records from primary storage. [2025-10-20T19:02:39.955165] [INFO ] [MAIN ] GPK: Loaded 53548 existing records. [2025-10-20T19:02:40.137895] [WARN ] [MAIN ] GPK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:40.138126] [INFO ] [MAIN ] GPK: Wiping existing primary data before full refresh. [2025-10-20T19:02:40.138181] [INFO ] [LOCAL ] GPK: Starting deletion of primary data. [2025-10-20T19:02:40.138816] [DEBUG] [LOCAL ] GPK: Deleted primary data file. [2025-10-20T19:02:40.138872] [INFO ] [MAIN ] GPK: Starting full primary refresh. [2025-10-20T19:02:40.256463] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-04. [2025-10-20T19:02:40.493873] [INFO ] [REMOTE ] GPK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:41.051355] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-01. [2025-10-20T19:02:41.089828] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:02:41.468784] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T19:02:41.851361] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-05. [2025-10-20T19:02:42.656467] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-01. [2025-10-20T19:02:42.673291] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:02:42.691537] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T19:02:43.451363] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-02. [2025-10-20T19:02:43.493257] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T19:02:44.251371] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-02. [2025-10-20T19:02:44.912038] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T19:02:44.918359] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:02:45.051374] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-03. [2025-10-20T19:02:45.594456] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T19:02:45.851376] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-04. [2025-10-20T19:02:46.419626] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T19:02:46.651364] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-05. [2025-10-20T19:02:47.099595] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T19:02:47.451424] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-04. [2025-10-20T19:02:48.251377] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-06. [2025-10-20T19:02:48.849768] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T19:02:49.051384] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-06. [2025-10-20T19:02:49.147630] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:02:49.851443] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-05. [2025-10-20T19:02:50.042010] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T19:02:50.651432] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-07. [2025-10-20T19:02:51.451352] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-07. [2025-10-20T19:02:51.451256] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T19:02:51.636109] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:02:51.802703] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T19:02:52.251411] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-08. [2025-10-20T19:02:53.051371] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-03. [2025-10-20T19:02:53.193990] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T19:02:53.851463] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-06. [2025-10-20T19:02:54.651395] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-08. [2025-10-20T19:02:54.811088] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T19:02:55.231769] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:02:55.380079] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:02:55.451400] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-09. [2025-10-20T19:02:56.119681] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T19:02:56.251389] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-04. [2025-10-20T19:02:57.051394] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-07. [2025-10-20T19:02:57.851390] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-10. [2025-10-20T19:02:57.915629] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T19:02:58.405391] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T19:02:58.427333] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T19:02:58.428263] [INFO ] [REMOTE ] GPJA: Full fetch complete (6894 records). [2025-10-20T19:02:58.428330] [DEBUG] [VALIDATOR ] GPJA: Validating primary data. [2025-10-20T19:02:58.656546] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-08. [2025-10-20T19:02:58.780349] [DEBUG] [VALIDATOR ] GPJA: All expected trading days are present. [2025-10-20T19:02:58.780421] [DEBUG] [VALIDATOR ] GPJA: Primary data validation passed. [2025-10-20T19:02:58.780591] [INFO ] [MAIN ] GPJA: Fetched primary data is valid. [2025-10-20T19:02:58.780629] [DEBUG] [LOCAL ] GPJA: Saving primary data. [2025-10-20T19:02:58.781027] [DEBUG] [LOCAL ] GPJA: Trimmed 6894 records to 4118 (keeping 200 days of data) [2025-10-20T19:02:58.805689] [DEBUG] [LOCAL ] GPJA: Primary data saved (4118 records). [2025-10-20T19:02:58.805749] [INFO ] [MAIN ] GPJA: Full history saved. [2025-10-20T19:02:58.805813] [INFO ] [MAIN ] Update Progress: 956/2367 symbols completed. GPJA took 22.7s. Estimated remaining: 3h 1m 24s. Rate: 7.8 symbols/min [2025-10-20T19:02:58.973137] [INFO ] [MAIN ] GPMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:58.998265] [DEBUG] [LOCAL ] GPMT: Loaded 26215 existing records from primary storage. [2025-10-20T19:02:58.998509] [INFO ] [MAIN ] GPMT: Loaded 26215 existing records. [2025-10-20T19:02:59.176156] [WARN ] [MAIN ] GPMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:59.176378] [INFO ] [MAIN ] GPMT: Wiping existing primary data before full refresh. [2025-10-20T19:02:59.176424] [INFO ] [LOCAL ] GPMT: Starting deletion of primary data. [2025-10-20T19:02:59.176625] [DEBUG] [LOCAL ] GPMT: Deleted primary data file. [2025-10-20T19:02:59.176708] [INFO ] [MAIN ] GPMT: Starting full primary refresh. [2025-10-20T19:02:59.451485] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-09. [2025-10-20T19:02:59.532576] [INFO ] [REMOTE ] GPMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:00.141451] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:03:00.191051] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:03:00.251360] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-09. [2025-10-20T19:03:01.051409] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-05. [2025-10-20T19:03:01.851371] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-10. [2025-10-20T19:03:02.025446] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:03:02.651407] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-01. [2025-10-20T19:03:02.970988] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T19:03:03.014089] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T19:03:03.017263] [INFO ] [REMOTE ] GPI: Full fetch complete (21779 records). [2025-10-20T19:03:03.017339] [DEBUG] [VALIDATOR ] GPI: Validating primary data. [2025-10-20T19:03:03.396265] [DEBUG] [VALIDATOR ] GPI: All expected trading days are present. [2025-10-20T19:03:03.396338] [DEBUG] [VALIDATOR ] GPI: Primary data validation passed. [2025-10-20T19:03:03.396867] [INFO ] [MAIN ] GPI: Fetched primary data is valid. [2025-10-20T19:03:03.396927] [DEBUG] [LOCAL ] GPI: Saving primary data. [2025-10-20T19:03:03.397673] [DEBUG] [LOCAL ] GPI: Trimmed 21779 records to 14844 (keeping 200 days of data) [2025-10-20T19:03:03.464982] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-06. [2025-10-20T19:03:03.495661] [DEBUG] [LOCAL ] GPI: Primary data saved (14844 records). [2025-10-20T19:03:03.495822] [INFO ] [MAIN ] GPI: Full history saved. [2025-10-20T19:03:03.495915] [INFO ] [MAIN ] Update Progress: 957/2367 symbols completed. GPI took 29.5s. Estimated remaining: 3h 1m 12s. Rate: 7.8 symbols/min [2025-10-20T19:03:03.742715] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T19:03:04.251436] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-10. [2025-10-20T19:03:05.034916] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:03:05.051347] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-07. [2025-10-20T19:03:05.860552] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-02. [2025-10-20T19:03:05.866252] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T19:03:05.874505] [INFO ] [REMOTE ] GPC: Full fetch complete (68672 records). [2025-10-20T19:03:05.874666] [DEBUG] [VALIDATOR ] GPC: Validating primary data. [2025-10-20T19:03:06.275512] [DEBUG] [VALIDATOR ] GPC: All expected trading days are present. [2025-10-20T19:03:06.275605] [DEBUG] [VALIDATOR ] GPC: Primary data validation passed. [2025-10-20T19:03:06.277415] [INFO ] [MAIN ] GPC: Fetched primary data is valid. [2025-10-20T19:03:06.277497] [DEBUG] [LOCAL ] GPC: Saving primary data. [2025-10-20T19:03:06.279287] [DEBUG] [LOCAL ] GPC: Trimmed 68672 records to 45947 (keeping 200 days of data) [2025-10-20T19:03:06.571947] [DEBUG] [LOCAL ] GPC: Primary data saved (45947 records). [2025-10-20T19:03:06.572096] [INFO ] [MAIN ] GPC: Full history saved. [2025-10-20T19:03:06.572551] [INFO ] [MAIN ] Update Progress: 958/2367 symbols completed. GPC took 34.9s. Estimated remaining: 3h 0m 57s. Rate: 7.8 symbols/min [2025-10-20T19:03:06.839843] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:03:06.852832] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-08. [2025-10-20T19:03:06.935408] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T19:03:07.451404] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-03. [2025-10-20T19:03:08.660524] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:03:08.675037] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-09. [2025-10-20T19:03:08.695335] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T19:03:09.051445] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-04. [2025-10-20T19:03:10.101270] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T19:03:10.109517] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-05. [2025-10-20T19:03:10.546907] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T19:03:10.651443] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-10. [2025-10-20T19:03:11.105667] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:03:11.451459] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-06. [2025-10-20T19:03:12.144863] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:03:12.157565] [INFO ] [REMOTE ] GPK: Full fetch complete (76056 records). [2025-10-20T19:03:12.157809] [DEBUG] [VALIDATOR ] GPK: Validating primary data. [2025-10-20T19:03:12.623461] [DEBUG] [VALIDATOR ] GPK: All expected trading days are present. [2025-10-20T19:03:12.623704] [DEBUG] [VALIDATOR ] GPK: Primary data validation passed. [2025-10-20T19:03:12.625170] [INFO ] [MAIN ] GPK: Fetched primary data is valid. [2025-10-20T19:03:12.625254] [DEBUG] [LOCAL ] GPK: Saving primary data. [2025-10-20T19:03:12.627721] [DEBUG] [LOCAL ] GPK: Trimmed 76056 records to 52784 (keeping 200 days of data) [2025-10-20T19:03:13.026720] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:03:13.049946] [DEBUG] [LOCAL ] GPK: Primary data saved (52784 records). [2025-10-20T19:03:13.050322] [INFO ] [MAIN ] GPK: Full history saved. [2025-10-20T19:03:13.050521] [INFO ] [MAIN ] Update Progress: 959/2367 symbols completed. GPK took 33.3s. Estimated remaining: 3h 0m 48s. Rate: 7.8 symbols/min [2025-10-20T19:03:13.051187] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-07. [2025-10-20T19:03:14.132403] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T19:03:14.140325] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-08. [2025-10-20T19:03:15.182840] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T19:03:15.187938] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-09. [2025-10-20T19:03:16.145308] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T19:03:16.149956] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-10. [2025-10-20T19:03:17.261756] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T19:03:17.265854] [INFO ] [REMOTE ] GPMT: Full fetch complete (36538 records). [2025-10-20T19:03:17.266097] [DEBUG] [VALIDATOR ] GPMT: Validating primary data. [2025-10-20T19:03:17.697948] [DEBUG] [VALIDATOR ] GPMT: All expected trading days are present. [2025-10-20T19:03:17.698084] [DEBUG] [VALIDATOR ] GPMT: Primary data validation passed. [2025-10-20T19:03:17.699697] [INFO ] [MAIN ] GPMT: Fetched primary data is valid. [2025-10-20T19:03:17.699769] [DEBUG] [LOCAL ] GPMT: Saving primary data. [2025-10-20T19:03:17.701364] [DEBUG] [LOCAL ] GPMT: Trimmed 36538 records to 25982 (keeping 200 days of data) [2025-10-20T19:03:17.866094] [DEBUG] [LOCAL ] GPMT: Primary data saved (25982 records). [2025-10-20T19:03:17.866261] [INFO ] [MAIN ] GPMT: Full history saved. [2025-10-20T19:03:17.866362] [INFO ] [MAIN ] Update Progress: 960/2367 symbols completed. GPMT took 19.1s. Estimated remaining: 3h 0m 36s. Rate: 7.8 symbols/min [2025-10-20T19:03:18.572590] [INFO ] [MAIN ] GRBK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:18.626295] [INFO ] [MAIN ] GPOR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:18.648990] [INFO ] [MAIN ] GPN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:18.650696] [INFO ] [MAIN ] GPRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:18.692042] [DEBUG] [LOCAL ] GRBK: Loaded 27388 existing records from primary storage. [2025-10-20T19:03:18.693667] [INFO ] [MAIN ] GRBK: Loaded 27388 existing records. [2025-10-20T19:03:18.744005] [DEBUG] [LOCAL ] GPOR: Loaded 23691 existing records from primary storage. [2025-10-20T19:03:18.822030] [INFO ] [MAIN ] GPOR: Loaded 23691 existing records. [2025-10-20T19:03:19.036354] [DEBUG] [LOCAL ] GPRK: Loaded 40348 existing records from primary storage. [2025-10-20T19:03:19.058504] [WARN ] [MAIN ] GRBK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:19.083694] [INFO ] [MAIN ] GPRK: Loaded 40348 existing records. [2025-10-20T19:03:19.093370] [INFO ] [MAIN ] GRBK: Wiping existing primary data before full refresh. [2025-10-20T19:03:19.121235] [DEBUG] [LOCAL ] GPN: Loaded 52716 existing records from primary storage. [2025-10-20T19:03:19.098791] [INFO ] [LOCAL ] GRBK: Starting deletion of primary data. [2025-10-20T19:03:19.173360] [INFO ] [MAIN ] GPN: Loaded 52716 existing records. [2025-10-20T19:03:19.238217] [DEBUG] [LOCAL ] GRBK: Deleted primary data file. [2025-10-20T19:03:19.238464] [INFO ] [MAIN ] GRBK: Starting full primary refresh. [2025-10-20T19:03:19.313492] [WARN ] [MAIN ] GPOR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:19.421493] [INFO ] [MAIN ] GPOR: Wiping existing primary data before full refresh. [2025-10-20T19:03:19.542864] [INFO ] [LOCAL ] GPOR: Starting deletion of primary data. [2025-10-20T19:03:19.609937] [DEBUG] [LOCAL ] GPOR: Deleted primary data file. [2025-10-20T19:03:19.650313] [WARN ] [MAIN ] GPRK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:19.657758] [INFO ] [MAIN ] GPOR: Starting full primary refresh. [2025-10-20T19:03:19.766274] [INFO ] [MAIN ] GPRK: Wiping existing primary data before full refresh. [2025-10-20T19:03:19.760913] [WARN ] [MAIN ] GPN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:19.861214] [INFO ] [LOCAL ] GPRK: Starting deletion of primary data. [2025-10-20T19:03:19.915479] [INFO ] [MAIN ] GPN: Wiping existing primary data before full refresh. [2025-10-20T19:03:19.944008] [INFO ] [LOCAL ] GPN: Starting deletion of primary data. [2025-10-20T19:03:19.963300] [DEBUG] [LOCAL ] GPRK: Deleted primary data file. [2025-10-20T19:03:19.983096] [DEBUG] [LOCAL ] GPN: Deleted primary data file. [2025-10-20T19:03:19.983009] [INFO ] [MAIN ] GPRK: Starting full primary refresh. [2025-10-20T19:03:20.062828] [INFO ] [MAIN ] GPN: Starting full primary refresh. [2025-10-20T19:03:20.712892] [INFO ] [REMOTE ] GRBK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:20.862151] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-01. [2025-10-20T19:03:21.209170] [INFO ] [REMOTE ] GPRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:21.239282] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-01. [2025-10-20T19:03:21.270353] [INFO ] [REMOTE ] GPOR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:21.293572] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-01. [2025-10-20T19:03:21.351436] [INFO ] [REMOTE ] GPN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:21.352506] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-01. [2025-10-20T19:03:22.334753] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T19:03:22.342026] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-02. [2025-10-20T19:03:22.549047] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T19:03:22.560500] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-02. [2025-10-20T19:03:22.669168] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T19:03:22.675164] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-02. [2025-10-20T19:03:22.797279] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:03:22.806923] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-02. [2025-10-20T19:03:23.247620] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T19:03:23.254173] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-03. [2025-10-20T19:03:23.812588] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:03:23.820077] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-03. [2025-10-20T19:03:24.101185] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T19:03:24.128166] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:03:24.137837] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-03. [2025-10-20T19:03:24.217541] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T19:03:24.251568] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-03. [2025-10-20T19:03:24.714971] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T19:03:25.051408] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-04. [2025-10-20T19:03:25.298804] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:03:25.851464] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-04. [2025-10-20T19:03:25.926774] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T19:03:26.182427] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T19:03:26.651410] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-04. [2025-10-20T19:03:27.243848] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T19:03:27.451418] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-05. [2025-10-20T19:03:28.251384] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-04. [2025-10-20T19:03:28.401065] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:03:29.001672] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:03:29.051379] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-06. [2025-10-20T19:03:29.546544] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T19:03:29.851422] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-05. [2025-10-20T19:03:30.532455] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T19:03:30.651349] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-05. [2025-10-20T19:03:31.198489] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T19:03:31.451395] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-05. [2025-10-20T19:03:31.829524] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T19:03:32.251348] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-06. [2025-10-20T19:03:32.303072] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T19:03:33.051380] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-06. [2025-10-20T19:03:33.851606] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-06. [2025-10-20T19:03:33.889761] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:03:34.183984] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T19:03:34.651399] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-07. [2025-10-20T19:03:34.847613] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T19:03:35.451405] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-07. [2025-10-20T19:03:36.086480] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T19:03:36.251371] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-07. [2025-10-20T19:03:36.748120] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T19:03:37.051428] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-07. [2025-10-20T19:03:37.851367] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-08. [2025-10-20T19:03:38.174863] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T19:03:38.207356] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T19:03:38.651372] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-08. [2025-10-20T19:03:38.701186] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T19:03:39.451375] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-08. [2025-10-20T19:03:40.105805] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T19:03:40.251378] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-09. [2025-10-20T19:03:40.635995] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T19:03:41.051413] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-09. [2025-10-20T19:03:41.382357] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T19:03:41.851348] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-08. [2025-10-20T19:03:42.240490] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T19:03:42.651376] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-10. [2025-10-20T19:03:43.451350] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-09. [2025-10-20T19:03:43.507419] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:03:43.881406] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:03:43.885518] [INFO ] [REMOTE ] GPOR: Full fetch complete (34908 records). [2025-10-20T19:03:43.885607] [DEBUG] [VALIDATOR ] GPOR: Validating primary data. [2025-10-20T19:03:44.251637] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-10. [2025-10-20T19:03:44.264961] [DEBUG] [VALIDATOR ] GPOR: All expected trading days are present. [2025-10-20T19:03:44.265023] [DEBUG] [VALIDATOR ] GPOR: Primary data validation passed. [2025-10-20T19:03:44.265767] [INFO ] [MAIN ] GPOR: Fetched primary data is valid. [2025-10-20T19:03:44.265931] [DEBUG] [LOCAL ] GPOR: Saving primary data. [2025-10-20T19:03:44.266877] [DEBUG] [LOCAL ] GPOR: Trimmed 34908 records to 23397 (keeping 200 days of data) [2025-10-20T19:03:44.412850] [DEBUG] [LOCAL ] GPOR: Primary data saved (23397 records). [2025-10-20T19:03:44.412998] [INFO ] [MAIN ] GPOR: Full history saved. [2025-10-20T19:03:44.413082] [INFO ] [MAIN ] Update Progress: 961/2367 symbols completed. GPOR took 26.5s. Estimated remaining: 3h 0m 56s. Rate: 7.8 symbols/min [2025-10-20T19:03:44.598748] [INFO ] [MAIN ] GRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:44.609255] [DEBUG] [LOCAL ] GRC: Loaded 9440 existing records from primary storage. [2025-10-20T19:03:44.609483] [INFO ] [MAIN ] GRC: Loaded 9440 existing records. [2025-10-20T19:03:44.802520] [WARN ] [MAIN ] GRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:44.802850] [INFO ] [MAIN ] GRC: Wiping existing primary data before full refresh. [2025-10-20T19:03:44.802931] [INFO ] [LOCAL ] GRC: Starting deletion of primary data. [2025-10-20T19:03:44.803197] [DEBUG] [LOCAL ] GRC: Deleted primary data file. [2025-10-20T19:03:44.803271] [INFO ] [MAIN ] GRC: Starting full primary refresh. [2025-10-20T19:03:44.977722] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T19:03:45.057249] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-10. [2025-10-20T19:03:45.197260] [INFO ] [REMOTE ] GRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:45.613362] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T19:03:45.617697] [INFO ] [REMOTE ] GRBK: Full fetch complete (40133 records). [2025-10-20T19:03:45.617815] [DEBUG] [VALIDATOR ] GRBK: Validating primary data. [2025-10-20T19:03:45.858093] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-01. [2025-10-20T19:03:45.994692] [DEBUG] [VALIDATOR ] GRBK: All expected trading days are present. [2025-10-20T19:03:45.994782] [DEBUG] [VALIDATOR ] GRBK: Primary data validation passed. [2025-10-20T19:03:45.995429] [INFO ] [MAIN ] GRBK: Fetched primary data is valid. [2025-10-20T19:03:45.995469] [DEBUG] [LOCAL ] GRBK: Saving primary data. [2025-10-20T19:03:45.996408] [DEBUG] [LOCAL ] GRBK: Trimmed 40133 records to 26937 (keeping 200 days of data) [2025-10-20T19:03:46.155322] [DEBUG] [LOCAL ] GRBK: Primary data saved (26937 records). [2025-10-20T19:03:46.155463] [INFO ] [MAIN ] GRBK: Full history saved. [2025-10-20T19:03:46.155546] [INFO ] [MAIN ] Update Progress: 962/2367 symbols completed. GRBK took 28.3s. Estimated remaining: 3h 0m 39s. Rate: 7.8 symbols/min [2025-10-20T19:03:46.327189] [INFO ] [MAIN ] GRDN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:46.346486] [DEBUG] [LOCAL ] GRDN: Loaded 21216 existing records from primary storage. [2025-10-20T19:03:46.346754] [INFO ] [MAIN ] GRDN: Loaded 21216 existing records. [2025-10-20T19:03:46.522770] [WARN ] [MAIN ] GRDN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:46.523008] [INFO ] [MAIN ] GRDN: Wiping existing primary data before full refresh. [2025-10-20T19:03:46.523058] [INFO ] [LOCAL ] GRDN: Starting deletion of primary data. [2025-10-20T19:03:46.523235] [DEBUG] [LOCAL ] GRDN: Deleted primary data file. [2025-10-20T19:03:46.523271] [INFO ] [MAIN ] GRDN: Starting full primary refresh. [2025-10-20T19:03:46.662016] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-09. [2025-10-20T19:03:46.886425] [INFO ] [REMOTE ] GRDN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:46.893132] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:03:46.942709] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T19:03:46.948842] [INFO ] [REMOTE ] GPRK: Full fetch complete (58328 records). [2025-10-20T19:03:46.948947] [DEBUG] [VALIDATOR ] GPRK: Validating primary data. [2025-10-20T19:03:47.326831] [DEBUG] [VALIDATOR ] GPRK: All expected trading days are present. [2025-10-20T19:03:47.326908] [DEBUG] [VALIDATOR ] GPRK: Primary data validation passed. [2025-10-20T19:03:47.327733] [INFO ] [MAIN ] GPRK: Fetched primary data is valid. [2025-10-20T19:03:47.327793] [DEBUG] [LOCAL ] GPRK: Saving primary data. [2025-10-20T19:03:47.329096] [DEBUG] [LOCAL ] GPRK: Trimmed 58328 records to 39821 (keeping 200 days of data) [2025-10-20T19:03:47.484289] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-01. [2025-10-20T19:03:47.550856] [DEBUG] [LOCAL ] GPRK: Primary data saved (39821 records). [2025-10-20T19:03:47.550974] [INFO ] [MAIN ] GPRK: Full history saved. [2025-10-20T19:03:47.551048] [INFO ] [MAIN ] Update Progress: 963/2367 symbols completed. GPRK took 29.7s. Estimated remaining: 3h 0m 23s. Rate: 7.8 symbols/min [2025-10-20T19:03:47.715685] [INFO ] [MAIN ] GRMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:47.753181] [DEBUG] [LOCAL ] GRMN: Loaded 40933 existing records from primary storage. [2025-10-20T19:03:47.753450] [INFO ] [MAIN ] GRMN: Loaded 40933 existing records. [2025-10-20T19:03:47.925602] [WARN ] [MAIN ] GRMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:47.925843] [INFO ] [MAIN ] GRMN: Wiping existing primary data before full refresh. [2025-10-20T19:03:47.925888] [INFO ] [LOCAL ] GRMN: Starting deletion of primary data. [2025-10-20T19:03:47.926069] [DEBUG] [LOCAL ] GRMN: Deleted primary data file. [2025-10-20T19:03:47.926106] [INFO ] [MAIN ] GRMN: Starting full primary refresh. [2025-10-20T19:03:48.253474] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T19:03:48.252714] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-02. [2025-10-20T19:03:48.327382] [INFO ] [REMOTE ] GRMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:48.473920] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T19:03:49.051362] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-02. [2025-10-20T19:03:49.103090] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T19:03:49.790953] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T19:03:49.851442] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-03. [2025-10-20T19:03:50.629210] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T19:03:50.651342] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-04. [2025-10-20T19:03:51.451387] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-10. [2025-10-20T19:03:51.475768] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T19:03:52.251358] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-01. [2025-10-20T19:03:53.051394] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-03. [2025-10-20T19:03:53.226841] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T19:03:53.234365] [INFO ] [REMOTE ] GPN: Full fetch complete (75071 records). [2025-10-20T19:03:53.234511] [DEBUG] [VALIDATOR ] GPN: Validating primary data. [2025-10-20T19:03:53.621372] [DEBUG] [VALIDATOR ] GPN: All expected trading days are present. [2025-10-20T19:03:53.621459] [DEBUG] [VALIDATOR ] GPN: Primary data validation passed. [2025-10-20T19:03:53.622398] [INFO ] [MAIN ] GPN: Fetched primary data is valid. [2025-10-20T19:03:53.622460] [DEBUG] [LOCAL ] GPN: Saving primary data. [2025-10-20T19:03:53.623939] [DEBUG] [LOCAL ] GPN: Trimmed 75071 records to 51943 (keeping 200 days of data) [2025-10-20T19:03:53.855002] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-05. [2025-10-20T19:03:53.926191] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:03:54.001800] [DEBUG] [LOCAL ] GPN: Primary data saved (51943 records). [2025-10-20T19:03:54.003087] [INFO ] [MAIN ] GPN: Full history saved. [2025-10-20T19:03:54.004823] [INFO ] [MAIN ] Update Progress: 964/2367 symbols completed. GPN took 36.1s. Estimated remaining: 3h 0m 13s. Rate: 7.8 symbols/min [2025-10-20T19:03:54.004627] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T19:03:54.184596] [INFO ] [MAIN ] GRND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:54.233776] [DEBUG] [LOCAL ] GRND: Loaded 50893 existing records from primary storage. [2025-10-20T19:03:54.234104] [INFO ] [MAIN ] GRND: Loaded 50893 existing records. [2025-10-20T19:03:54.421906] [WARN ] [MAIN ] GRND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:54.422115] [INFO ] [MAIN ] GRND: Wiping existing primary data before full refresh. [2025-10-20T19:03:54.422161] [INFO ] [LOCAL ] GRND: Starting deletion of primary data. [2025-10-20T19:03:54.422675] [DEBUG] [LOCAL ] GRND: Deleted primary data file. [2025-10-20T19:03:54.422742] [INFO ] [MAIN ] GRND: Starting full primary refresh. [2025-10-20T19:03:54.651796] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-02. [2025-10-20T19:03:54.728687] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T19:03:54.808909] [INFO ] [REMOTE ] GRND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:55.451376] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-01. [2025-10-20T19:03:56.251416] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-04. [2025-10-20T19:03:56.669434] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-10-20T19:03:56.949507] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T19:03:57.051383] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-02. [2025-10-20T19:03:57.270868] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T19:03:57.851378] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-05. [2025-10-20T19:03:58.651401] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-03. [2025-10-20T19:03:58.780360] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T19:03:59.091590] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:03:59.451376] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-03. [2025-10-20T19:04:00.200190] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:04:00.251402] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-04. [2025-10-20T19:04:01.051475] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-06. [2025-10-20T19:04:01.406877] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T19:04:01.481474] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T19:04:01.851344] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-04. [2025-10-20T19:04:02.133287] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T19:04:02.651379] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-06. [2025-10-20T19:04:03.365842] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T19:04:03.451446] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-05. [2025-10-20T19:04:03.647011] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:04:04.251416] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-05. [2025-10-20T19:04:05.051408] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-07. [2025-10-20T19:04:05.133925] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T19:04:05.851398] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-07. [2025-10-20T19:04:06.141418] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T19:04:06.416627] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T19:04:06.651386] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-06. [2025-10-20T19:04:07.091530] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T19:04:07.451392] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-08. [2025-10-20T19:04:08.116403] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:04:08.251351] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-07. [2025-10-20T19:04:08.489703] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T19:04:09.051371] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-09. [2025-10-20T19:04:09.851399] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-08. [2025-10-20T19:04:09.962519] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T19:04:10.416446] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T19:04:10.619126] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T19:04:10.651358] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-10. [2025-10-20T19:04:11.451368] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-08. [2025-10-20T19:04:11.556729] [DEBUG] [REMOTE ] GRMN: Error fetching primary data for 2025-08: 'API call returned an empty response.' [2025-10-20T19:04:11.557011] [WARN ] [REMOTE ] GRMN: No data returned for month 2025-08 [2025-10-20T19:04:11.557100] [FATAL] [REMOTE ] GRMN: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:04:11.557204] [FATAL] [MAIN ] GRMN: Failed to fetch primary data. [2025-10-20T19:04:11.557275] [WARN ] [MAIN ] GRMN: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:04:11.557358] [INFO ] [LOCAL ] GRMN: Starting deletion of primary data. [2025-10-20T19:04:11.557428] [DEBUG] [LOCAL ] GRMN: No primary data file found to delete. [2025-10-20T19:04:11.557467] [INFO ] [LOCAL ] GRMN: Starting deletion of derivative data. [2025-10-20T19:04:11.557728] [DEBUG] [LOCAL ] GRMN: Deleted derivative data file. [2025-10-20T19:04:11.557816] [INFO ] [MAIN ] Update Progress: 965/2367 symbols completed. GRMN took 24.0s. Estimated remaining: 3h 0m 20s. Rate: 7.8 symbols/min [2025-10-20T19:04:11.736594] [INFO ] [MAIN ] GRNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:11.768767] [DEBUG] [LOCAL ] GRNT: Loaded 34443 existing records from primary storage. [2025-10-20T19:04:11.769068] [INFO ] [MAIN ] GRNT: Loaded 34443 existing records. [2025-10-20T19:04:11.953485] [WARN ] [MAIN ] GRNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:11.963085] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:04:11.962785] [INFO ] [MAIN ] GRNT: Wiping existing primary data before full refresh. [2025-10-20T19:04:11.966825] [INFO ] [LOCAL ] GRNT: Starting deletion of primary data. [2025-10-20T19:04:11.966930] [INFO ] [REMOTE ] GRDN: Full fetch complete (26339 records). [2025-10-20T19:04:11.967047] [DEBUG] [VALIDATOR ] GRDN: Validating primary data. [2025-10-20T19:04:11.992029] [DEBUG] [LOCAL ] GRNT: Deleted primary data file. [2025-10-20T19:04:12.012945] [INFO ] [MAIN ] GRNT: Starting full primary refresh. [2025-10-20T19:04:12.251464] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-06. [2025-10-20T19:04:12.738629] [INFO ] [REMOTE ] GRNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:12.738881] [DEBUG] [VALIDATOR ] GRDN: All expected trading days are present. [2025-10-20T19:04:12.739233] [DEBUG] [VALIDATOR ] GRDN: Primary data validation passed. [2025-10-20T19:04:12.739755] [INFO ] [MAIN ] GRDN: Fetched primary data is valid. [2025-10-20T19:04:12.739817] [DEBUG] [LOCAL ] GRDN: Saving primary data. [2025-10-20T19:04:12.740423] [DEBUG] [LOCAL ] GRDN: Trimmed 26339 records to 21063 (keeping 200 days of data) [2025-10-20T19:04:12.860387] [DEBUG] [LOCAL ] GRDN: Primary data saved (21063 records). [2025-10-20T19:04:12.860514] [INFO ] [MAIN ] GRDN: Full history saved. [2025-10-20T19:04:12.860589] [INFO ] [MAIN ] Update Progress: 966/2367 symbols completed. GRDN took 26.7s. Estimated remaining: 3h 0m 3s. Rate: 7.8 symbols/min [2025-10-20T19:04:13.037228] [INFO ] [MAIN ] GROV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:13.047458] [DEBUG] [LOCAL ] GROV: Loaded 8805 existing records from primary storage. [2025-10-20T19:04:13.047697] [INFO ] [MAIN ] GROV: Loaded 8805 existing records. [2025-10-20T19:04:13.058715] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-01. [2025-10-20T19:04:13.235583] [WARN ] [MAIN ] GROV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:13.235837] [INFO ] [MAIN ] GROV: Wiping existing primary data before full refresh. [2025-10-20T19:04:13.235885] [INFO ] [LOCAL ] GROV: Starting deletion of primary data. [2025-10-20T19:04:13.236076] [DEBUG] [LOCAL ] GROV: Deleted primary data file. [2025-10-20T19:04:13.236130] [INFO ] [MAIN ] GROV: Starting full primary refresh. [2025-10-20T19:04:13.585599] [INFO ] [REMOTE ] GROV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:13.851381] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-09. [2025-10-20T19:04:13.951265] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:04:14.465356] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:04:14.606274] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T19:04:14.651409] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-02. [2025-10-20T19:04:15.451401] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-01. [2025-10-20T19:04:15.677339] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:04:16.245391] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:04:16.251335] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-02. [2025-10-20T19:04:17.051402] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-10. [2025-10-20T19:04:17.061018] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T19:04:17.851447] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-03. [2025-10-20T19:04:18.111755] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T19:04:18.115857] [INFO ] [REMOTE ] GRC: Full fetch complete (13431 records). [2025-10-20T19:04:18.116018] [DEBUG] [VALIDATOR ] GRC: Validating primary data. [2025-10-20T19:04:18.657191] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-03. [2025-10-20T19:04:18.691555] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T19:04:18.698904] [DEBUG] [VALIDATOR ] GRC: All expected trading days are present. [2025-10-20T19:04:18.699400] [DEBUG] [VALIDATOR ] GRC: Primary data validation passed. [2025-10-20T19:04:18.701839] [INFO ] [MAIN ] GRC: Fetched primary data is valid. [2025-10-20T19:04:18.702069] [DEBUG] [LOCAL ] GRC: Saving primary data. [2025-10-20T19:04:18.704152] [DEBUG] [LOCAL ] GRC: Trimmed 13431 records to 9306 (keeping 200 days of data) [2025-10-20T19:04:18.772214] [DEBUG] [LOCAL ] GRC: Primary data saved (9306 records). [2025-10-20T19:04:18.772368] [INFO ] [MAIN ] GRC: Full history saved. [2025-10-20T19:04:18.772454] [INFO ] [MAIN ] Update Progress: 967/2367 symbols completed. GRC took 34.4s. Estimated remaining: 2h 59m 52s. Rate: 7.8 symbols/min [2025-10-20T19:04:18.994357] [INFO ] [MAIN ] GRX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:19.002085] [DEBUG] [LOCAL ] GRX: Loaded 4862 existing records from primary storage. [2025-10-20T19:04:19.002347] [INFO ] [MAIN ] GRX: Loaded 4862 existing records. [2025-10-20T19:04:19.211697] [WARN ] [MAIN ] GRX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:19.211943] [INFO ] [MAIN ] GRX: Wiping existing primary data before full refresh. [2025-10-20T19:04:19.211990] [INFO ] [LOCAL ] GRX: Starting deletion of primary data. [2025-10-20T19:04:19.212194] [DEBUG] [LOCAL ] GRX: Deleted primary data file. [2025-10-20T19:04:19.212254] [INFO ] [MAIN ] GRX: Starting full primary refresh. [2025-10-20T19:04:19.459530] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-07. [2025-10-20T19:04:19.640338] [INFO ] [REMOTE ] GRX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:20.065514] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T19:04:20.251377] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-01. [2025-10-20T19:04:20.819913] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T19:04:21.051355] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-04. [2025-10-20T19:04:21.231056] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:04:21.698550] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T19:04:21.851354] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-04. [2025-10-20T19:04:22.651378] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-02. [2025-10-20T19:04:23.214863] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T19:04:23.239909] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T19:04:23.451404] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-03. [2025-10-20T19:04:24.036827] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T19:04:24.251430] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-05. [2025-10-20T19:04:24.825311] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T19:04:25.051385] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-06. [2025-10-20T19:04:25.768752] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T19:04:25.851434] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-05. [2025-10-20T19:04:26.651415] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-07. [2025-10-20T19:04:27.086130] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T19:04:27.451388] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-06. [2025-10-20T19:04:27.707248] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T19:04:28.251404] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-08. [2025-10-20T19:04:28.856203] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T19:04:28.960223] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T19:04:29.051369] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-09. [2025-10-20T19:04:29.426984] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-10-20T19:04:29.851403] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-08. [2025-10-20T19:04:30.651383] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-10. [2025-10-20T19:04:31.451374] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-07. [2025-10-20T19:04:31.477250] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T19:04:31.478856] [INFO ] [REMOTE ] GROV: Full fetch complete (13699 records). [2025-10-20T19:04:31.478925] [DEBUG] [VALIDATOR ] GROV: Validating primary data. [2025-10-20T19:04:31.867369] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-10-20T19:04:31.876319] [DEBUG] [VALIDATOR ] GROV: All expected trading days are present. [2025-10-20T19:04:31.876401] [DEBUG] [VALIDATOR ] GROV: Primary data validation passed. [2025-10-20T19:04:31.886266] [INFO ] [MAIN ] GROV: Fetched primary data is valid. [2025-10-20T19:04:31.886343] [DEBUG] [LOCAL ] GROV: Saving primary data. [2025-10-20T19:04:31.887341] [DEBUG] [LOCAL ] GROV: Trimmed 13699 records to 8747 (keeping 200 days of data) [2025-10-20T19:04:31.936032] [DEBUG] [LOCAL ] GROV: Primary data saved (8747 records). [2025-10-20T19:04:31.936165] [INFO ] [MAIN ] GROV: Full history saved. [2025-10-20T19:04:31.936243] [INFO ] [MAIN ] Update Progress: 968/2367 symbols completed. GROV took 19.1s. Estimated remaining: 2h 59m 52s. Rate: 7.8 symbols/min [2025-10-20T19:04:32.102148] [INFO ] [MAIN ] GS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:32.154171] [DEBUG] [LOCAL ] GS: Loaded 52558 existing records from primary storage. [2025-10-20T19:04:32.154509] [INFO ] [MAIN ] GS: Loaded 52558 existing records. [2025-10-20T19:04:32.253063] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-04. [2025-10-20T19:04:32.336422] [WARN ] [MAIN ] GS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:32.336682] [INFO ] [MAIN ] GS: Wiping existing primary data before full refresh. [2025-10-20T19:04:32.336742] [INFO ] [LOCAL ] GS: Starting deletion of primary data. [2025-10-20T19:04:32.337428] [DEBUG] [LOCAL ] GS: Deleted primary data file. [2025-10-20T19:04:32.337486] [INFO ] [MAIN ] GS: Starting full primary refresh. [2025-10-20T19:04:32.690699] [INFO ] [REMOTE ] GS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:32.800936] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T19:04:33.046830] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:04:33.051334] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-05. [2025-10-20T19:04:33.638182] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T19:04:33.851368] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-09. [2025-10-20T19:04:34.651360] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-06. [2025-10-20T19:04:35.217398] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T19:04:35.451360] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-01. [2025-10-20T19:04:35.492297] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T19:04:36.251385] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-07. [2025-10-20T19:04:36.842406] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T19:04:37.051459] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-08. [2025-10-20T19:04:37.355893] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T19:04:37.633222] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:04:37.851361] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-02. [2025-10-20T19:04:38.651382] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-08. [2025-10-20T19:04:39.451455] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-09. [2025-10-20T19:04:39.955890] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T19:04:39.994078] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-10-20T19:04:40.145328] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T19:04:40.251390] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-10. [2025-10-20T19:04:41.051372] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-03. [2025-10-20T19:04:41.851367] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-09. [2025-10-20T19:04:42.163189] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T19:04:42.172600] [INFO ] [REMOTE ] GRND: Full fetch complete (71704 records). [2025-10-20T19:04:42.172792] [DEBUG] [VALIDATOR ] GRND: Validating primary data. [2025-10-20T19:04:42.564874] [DEBUG] [VALIDATOR ] GRND: All expected trading days are present. [2025-10-20T19:04:42.564981] [DEBUG] [VALIDATOR ] GRND: Primary data validation passed. [2025-10-20T19:04:42.565949] [INFO ] [MAIN ] GRND: Fetched primary data is valid. [2025-10-20T19:04:42.566015] [DEBUG] [LOCAL ] GRND: Saving primary data. [2025-10-20T19:04:42.567439] [DEBUG] [LOCAL ] GRND: Trimmed 71704 records to 50199 (keeping 200 days of data) [2025-10-20T19:04:42.656813] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-10. [2025-10-20T19:04:42.852901] [DEBUG] [LOCAL ] GRND: Primary data saved (50199 records). [2025-10-20T19:04:42.853030] [INFO ] [MAIN ] GRND: Full history saved. [2025-10-20T19:04:42.853108] [INFO ] [MAIN ] Update Progress: 969/2367 symbols completed. GRND took 48.8s. Estimated remaining: 2h 59m 49s. Rate: 7.8 symbols/min [2025-10-20T19:04:43.023028] [INFO ] [MAIN ] GSBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:43.058015] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T19:04:43.095988] [DEBUG] [LOCAL ] GSBD: Loaded 40293 existing records from primary storage. [2025-10-20T19:04:43.096266] [INFO ] [MAIN ] GSBD: Loaded 40293 existing records. [2025-10-20T19:04:43.269482] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T19:04:43.269305] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:04:43.302951] [WARN ] [MAIN ] GSBD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:43.296614] [INFO ] [REMOTE ] GRX: Full fetch complete (6538 records). [2025-10-20T19:04:43.309569] [INFO ] [MAIN ] GSBD: Wiping existing primary data before full refresh. [2025-10-20T19:04:43.309827] [INFO ] [LOCAL ] GSBD: Starting deletion of primary data. [2025-10-20T19:04:43.309725] [DEBUG] [VALIDATOR ] GRX: Validating primary data. [2025-10-20T19:04:43.315707] [DEBUG] [LOCAL ] GSBD: Deleted primary data file. [2025-10-20T19:04:43.316208] [INFO ] [MAIN ] GSBD: Starting full primary refresh. [2025-10-20T19:04:43.452086] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-04. [2025-10-20T19:04:44.253893] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-10. [2025-10-20T19:04:44.254441] [DEBUG] [VALIDATOR ] GRX: All expected trading days are present. [2025-10-20T19:04:44.253794] [INFO ] [REMOTE ] GSBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:44.256207] [DEBUG] [VALIDATOR ] GRX: Primary data validation passed. [2025-10-20T19:04:44.256599] [INFO ] [MAIN ] GRX: Fetched primary data is valid. [2025-10-20T19:04:44.256790] [DEBUG] [LOCAL ] GRX: Saving primary data. [2025-10-20T19:04:44.258048] [DEBUG] [LOCAL ] GRX: Trimmed 6538 records to 4840 (keeping 200 days of data) [2025-10-20T19:04:44.295701] [DEBUG] [LOCAL ] GRX: Primary data saved (4840 records). [2025-10-20T19:04:44.295959] [INFO ] [MAIN ] GRX: Full history saved. [2025-10-20T19:04:44.296078] [INFO ] [MAIN ] Update Progress: 970/2367 symbols completed. GRX took 25.5s. Estimated remaining: 2h 59m 33s. Rate: 7.8 symbols/min [2025-10-20T19:04:44.486718] [INFO ] [MAIN ] GSK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:44.547583] [DEBUG] [LOCAL ] GSK: Loaded 54285 existing records from primary storage. [2025-10-20T19:04:44.548185] [INFO ] [MAIN ] GSK: Loaded 54285 existing records. [2025-10-20T19:04:44.743804] [WARN ] [MAIN ] GSK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:44.744087] [INFO ] [MAIN ] GSK: Wiping existing primary data before full refresh. [2025-10-20T19:04:44.744136] [INFO ] [LOCAL ] GSK: Starting deletion of primary data. [2025-10-20T19:04:44.744366] [DEBUG] [LOCAL ] GSK: Deleted primary data file. [2025-10-20T19:04:44.744406] [INFO ] [MAIN ] GSK: Starting full primary refresh. [2025-10-20T19:04:45.055496] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-01. [2025-10-20T19:04:45.143497] [INFO ] [REMOTE ] GSK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:45.559534] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:04:45.564740] [INFO ] [REMOTE ] GRNT: Full fetch complete (48348 records). [2025-10-20T19:04:45.564829] [DEBUG] [VALIDATOR ] GRNT: Validating primary data. [2025-10-20T19:04:45.852164] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-01. [2025-10-20T19:04:45.956760] [DEBUG] [VALIDATOR ] GRNT: All expected trading days are present. [2025-10-20T19:04:45.956941] [DEBUG] [VALIDATOR ] GRNT: Primary data validation passed. [2025-10-20T19:04:45.957721] [INFO ] [MAIN ] GRNT: Fetched primary data is valid. [2025-10-20T19:04:45.957806] [DEBUG] [LOCAL ] GRNT: Saving primary data. [2025-10-20T19:04:45.959100] [DEBUG] [LOCAL ] GRNT: Trimmed 48348 records to 33992 (keeping 200 days of data) [2025-10-20T19:04:46.079963] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-04 (2.63 seconds). [2025-10-20T19:04:46.196862] [DEBUG] [LOCAL ] GRNT: Primary data saved (33992 records). [2025-10-20T19:04:46.196986] [INFO ] [MAIN ] GRNT: Full history saved. [2025-10-20T19:04:46.197060] [INFO ] [MAIN ] Update Progress: 971/2367 symbols completed. GRNT took 34.6s. Estimated remaining: 2h 59m 17s. Rate: 7.8 symbols/min [2025-10-20T19:04:46.365395] [INFO ] [MAIN ] GSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:46.396466] [DEBUG] [LOCAL ] GSL: Loaded 33725 existing records from primary storage. [2025-10-20T19:04:46.396731] [INFO ] [MAIN ] GSL: Loaded 33725 existing records. [2025-10-20T19:04:46.578885] [WARN ] [MAIN ] GSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:46.579231] [INFO ] [MAIN ] GSL: Wiping existing primary data before full refresh. [2025-10-20T19:04:46.592476] [INFO ] [LOCAL ] GSL: Starting deletion of primary data. [2025-10-20T19:04:46.601030] [DEBUG] [LOCAL ] GSL: Deleted primary data file. [2025-10-20T19:04:46.601822] [INFO ] [MAIN ] GSL: Starting full primary refresh. [2025-10-20T19:04:46.601768] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T19:04:46.652901] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-05. [2025-10-20T19:04:46.958161] [INFO ] [REMOTE ] GSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:47.407089] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T19:04:47.451409] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-01. [2025-10-20T19:04:48.251467] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-02. [2025-10-20T19:04:48.752368] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T19:04:48.918341] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-05 (2.27 seconds). [2025-10-20T19:04:49.051391] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-06. [2025-10-20T19:04:49.551174] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:04:49.851378] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-02. [2025-10-20T19:04:50.651365] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-02. [2025-10-20T19:04:51.013382] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-10-20T19:04:51.105955] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T19:04:51.451386] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-03. [2025-10-20T19:04:52.126824] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T19:04:52.251457] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-03. [2025-10-20T19:04:52.896747] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T19:04:53.051389] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-04. [2025-10-20T19:04:53.851385] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-03. [2025-10-20T19:04:53.902732] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:04:54.525142] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:04:54.651404] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-05. [2025-10-20T19:04:55.271479] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T19:04:55.451440] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-04. [2025-10-20T19:04:56.102820] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:04:56.251359] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-06. [2025-10-20T19:04:57.051389] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-07. [2025-10-20T19:04:57.263414] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T19:04:57.446458] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T19:04:57.851427] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-07. [2025-10-20T19:04:58.651407] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-05. [2025-10-20T19:04:59.123908] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T19:04:59.278903] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-10-20T19:04:59.451363] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-08. [2025-10-20T19:05:00.251346] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-08. [2025-10-20T19:05:00.505376] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T19:05:01.051444] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-06. [2025-10-20T19:05:01.258338] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:05:01.305639] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:05:01.851352] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-09. [2025-10-20T19:05:02.651368] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-09. [2025-10-20T19:05:02.933309] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T19:05:03.003221] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:05:03.451400] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-07. [2025-10-20T19:05:04.251340] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-04. [2025-10-20T19:05:04.760597] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-10-20T19:05:05.051552] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-10. [2025-10-20T19:05:05.051480] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:05:05.853996] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-08. [2025-10-20T19:05:05.859780] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:05:05.944104] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T19:05:05.951753] [INFO ] [REMOTE ] GS: Full fetch complete (75845 records). [2025-10-20T19:05:05.951880] [DEBUG] [VALIDATOR ] GS: Validating primary data. [2025-10-20T19:05:06.359513] [DEBUG] [VALIDATOR ] GS: All expected trading days are present. [2025-10-20T19:05:06.359603] [DEBUG] [VALIDATOR ] GS: Primary data validation passed. [2025-10-20T19:05:06.360538] [INFO ] [MAIN ] GS: Fetched primary data is valid. [2025-10-20T19:05:06.360603] [DEBUG] [LOCAL ] GS: Saving primary data. [2025-10-20T19:05:06.362041] [DEBUG] [LOCAL ] GS: Trimmed 75845 records to 51784 (keeping 200 days of data) [2025-10-20T19:05:06.664413] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-05. [2025-10-20T19:05:06.686436] [DEBUG] [LOCAL ] GS: Primary data saved (51784 records). [2025-10-20T19:05:06.686541] [INFO ] [MAIN ] GS: Full history saved. [2025-10-20T19:05:06.687259] [INFO ] [MAIN ] Update Progress: 972/2367 symbols completed. GS took 34.8s. Estimated remaining: 2h 59m 27s. Rate: 7.8 symbols/min [2025-10-20T19:05:06.860029] [INFO ] [MAIN ] GTES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:06.909858] [DEBUG] [LOCAL ] GTES: Loaded 51134 existing records from primary storage. [2025-10-20T19:05:06.910159] [INFO ] [MAIN ] GTES: Loaded 51134 existing records. [2025-10-20T19:05:07.091563] [WARN ] [MAIN ] GTES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:07.091756] [INFO ] [MAIN ] GTES: Wiping existing primary data before full refresh. [2025-10-20T19:05:07.091807] [INFO ] [LOCAL ] GTES: Starting deletion of primary data. [2025-10-20T19:05:07.092078] [DEBUG] [LOCAL ] GTES: Deleted primary data file. [2025-10-20T19:05:07.092119] [INFO ] [MAIN ] GTES: Starting full primary refresh. [2025-10-20T19:05:07.454450] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-10. [2025-10-20T19:05:07.454392] [INFO ] [REMOTE ] GTES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:07.723764] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T19:05:08.107055] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T19:05:08.251361] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-01. [2025-10-20T19:05:08.778991] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T19:05:08.786482] [INFO ] [REMOTE ] GSL: Full fetch complete (48304 records). [2025-10-20T19:05:08.786634] [DEBUG] [VALIDATOR ] GSL: Validating primary data. [2025-10-20T19:05:09.056556] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-06. [2025-10-20T19:05:09.168967] [DEBUG] [VALIDATOR ] GSL: All expected trading days are present. [2025-10-20T19:05:09.169061] [DEBUG] [VALIDATOR ] GSL: Primary data validation passed. [2025-10-20T19:05:09.169791] [INFO ] [MAIN ] GSL: Fetched primary data is valid. [2025-10-20T19:05:09.169852] [DEBUG] [LOCAL ] GSL: Saving primary data. [2025-10-20T19:05:09.170900] [DEBUG] [LOCAL ] GSL: Trimmed 48304 records to 33197 (keeping 200 days of data) [2025-10-20T19:05:09.366493] [DEBUG] [LOCAL ] GSL: Primary data saved (33197 records). [2025-10-20T19:05:09.366624] [INFO ] [MAIN ] GSL: Full history saved. [2025-10-20T19:05:09.366735] [INFO ] [MAIN ] Update Progress: 973/2367 symbols completed. GSL took 23.2s. Estimated remaining: 2h 59m 12s. Rate: 7.8 symbols/min [2025-10-20T19:05:09.538107] [INFO ] [MAIN ] GTLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:09.579604] [DEBUG] [LOCAL ] GTLS: Loaded 43892 existing records from primary storage. [2025-10-20T19:05:09.579921] [INFO ] [MAIN ] GTLS: Loaded 43892 existing records. [2025-10-20T19:05:09.790935] [WARN ] [MAIN ] GTLS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:09.791213] [INFO ] [MAIN ] GTLS: Wiping existing primary data before full refresh. [2025-10-20T19:05:09.791291] [INFO ] [LOCAL ] GTLS: Starting deletion of primary data. [2025-10-20T19:05:09.792086] [DEBUG] [LOCAL ] GTLS: Deleted primary data file. [2025-10-20T19:05:09.793960] [INFO ] [MAIN ] GTLS: Starting full primary refresh. [2025-10-20T19:05:09.793886] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T19:05:09.857811] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-02. [2025-10-20T19:05:10.163734] [INFO ] [REMOTE ] GTLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:10.197587] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:05:10.651360] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-01. [2025-10-20T19:05:11.451448] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-07. [2025-10-20T19:05:11.709997] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T19:05:12.074548] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:05:12.251409] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-03. [2025-10-20T19:05:13.006904] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T19:05:13.051375] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-08. [2025-10-20T19:05:13.851351] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-02. [2025-10-20T19:05:13.883710] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:05:14.560614] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:05:14.651397] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-09. [2025-10-20T19:05:14.986048] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:05:15.451429] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-03. [2025-10-20T19:05:16.251386] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-09. [2025-10-20T19:05:16.566473] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T19:05:17.033100] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:05:17.051389] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-04. [2025-10-20T19:05:17.852020] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-10. [2025-10-20T19:05:17.854490] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T19:05:18.251290] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:05:18.651368] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-10. [2025-10-20T19:05:19.451359] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-05. [2025-10-20T19:05:19.949064] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T19:05:19.957141] [INFO ] [REMOTE ] GSK: Full fetch complete (77635 records). [2025-10-20T19:05:19.957276] [DEBUG] [VALIDATOR ] GSK: Validating primary data. [2025-10-20T19:05:20.255150] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-04. [2025-10-20T19:05:20.365260] [DEBUG] [VALIDATOR ] GSK: All expected trading days are present. [2025-10-20T19:05:20.365338] [DEBUG] [VALIDATOR ] GSK: Primary data validation passed. [2025-10-20T19:05:20.366293] [INFO ] [MAIN ] GSK: Fetched primary data is valid. [2025-10-20T19:05:20.366355] [DEBUG] [LOCAL ] GSK: Saving primary data. [2025-10-20T19:05:20.367773] [DEBUG] [LOCAL ] GSK: Trimmed 77635 records to 53512 (keeping 200 days of data) [2025-10-20T19:05:20.620761] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T19:05:20.646937] [INFO ] [REMOTE ] GSBD: Full fetch complete (58595 records). [2025-10-20T19:05:20.650164] [DEBUG] [VALIDATOR ] GSBD: Validating primary data. [2025-10-20T19:05:20.833587] [DEBUG] [LOCAL ] GSK: Primary data saved (53512 records). [2025-10-20T19:05:20.833851] [INFO ] [MAIN ] GSK: Full history saved. [2025-10-20T19:05:20.834533] [INFO ] [MAIN ] Update Progress: 974/2367 symbols completed. GSK took 36.5s. Estimated remaining: 2h 59m 10s. Rate: 7.8 symbols/min [2025-10-20T19:05:21.275239] [INFO ] [MAIN ] GTN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:21.309888] [DEBUG] [VALIDATOR ] GSBD: All expected trading days are present. [2025-10-20T19:05:21.310331] [DEBUG] [VALIDATOR ] GSBD: Primary data validation passed. [2025-10-20T19:05:21.311207] [INFO ] [MAIN ] GSBD: Fetched primary data is valid. [2025-10-20T19:05:21.311381] [DEBUG] [LOCAL ] GSBD: Saving primary data. [2025-10-20T19:05:21.335037] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T19:05:21.343444] [DEBUG] [LOCAL ] GSBD: Trimmed 58595 records to 39645 (keeping 200 days of data) [2025-10-20T19:05:21.344961] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-06. [2025-10-20T19:05:21.494963] [DEBUG] [LOCAL ] GTN: Loaded 45765 existing records from primary storage. [2025-10-20T19:05:21.496551] [INFO ] [MAIN ] GTN: Loaded 45765 existing records. [2025-10-20T19:05:21.812358] [WARN ] [MAIN ] GTN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:21.813311] [INFO ] [MAIN ] GTN: Wiping existing primary data before full refresh. [2025-10-20T19:05:21.817364] [INFO ] [LOCAL ] GTN: Starting deletion of primary data. [2025-10-20T19:05:21.825842] [DEBUG] [LOCAL ] GTN: Deleted primary data file. [2025-10-20T19:05:21.836695] [INFO ] [MAIN ] GTN: Starting full primary refresh. [2025-10-20T19:05:21.854693] [DEBUG] [LOCAL ] GSBD: Primary data saved (39645 records). [2025-10-20T19:05:21.870043] [INFO ] [MAIN ] GSBD: Full history saved. [2025-10-20T19:05:21.884516] [INFO ] [MAIN ] Update Progress: 975/2367 symbols completed. GSBD took 39.0s. Estimated remaining: 2h 58m 53s. Rate: 7.8 symbols/min [2025-10-20T19:05:22.233524] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T19:05:22.349956] [INFO ] [MAIN ] GTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:22.376823] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-05. [2025-10-20T19:05:22.512897] [INFO ] [REMOTE ] GTN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:22.529109] [DEBUG] [LOCAL ] GTY: Loaded 30765 existing records from primary storage. [2025-10-20T19:05:22.529559] [INFO ] [MAIN ] GTY: Loaded 30765 existing records. [2025-10-20T19:05:22.653395] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-01. [2025-10-20T19:05:22.722097] [WARN ] [MAIN ] GTY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:22.722382] [INFO ] [MAIN ] GTY: Wiping existing primary data before full refresh. [2025-10-20T19:05:22.722434] [INFO ] [LOCAL ] GTY: Starting deletion of primary data. [2025-10-20T19:05:22.722683] [DEBUG] [LOCAL ] GTY: Deleted primary data file. [2025-10-20T19:05:22.722743] [INFO ] [MAIN ] GTY: Starting full primary refresh. [2025-10-20T19:05:23.119876] [INFO ] [REMOTE ] GTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:23.148585] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T19:05:23.451379] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-07. [2025-10-20T19:05:23.785735] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T19:05:23.841820] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:05:24.251395] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-06. [2025-10-20T19:05:24.990607] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:05:25.051361] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-08. [2025-10-20T19:05:25.770810] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T19:05:25.851374] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-02. [2025-10-20T19:05:26.532735] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:05:26.651382] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-09. [2025-10-20T19:05:27.126891] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T19:05:27.451369] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-07. [2025-10-20T19:05:28.232947] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T19:05:28.251335] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-10. [2025-10-20T19:05:28.804212] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T19:05:29.051375] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-03. [2025-10-20T19:05:29.825159] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T19:05:29.831758] [INFO ] [REMOTE ] GTLS: Full fetch complete (61617 records). [2025-10-20T19:05:29.831909] [DEBUG] [VALIDATOR ] GTLS: Validating primary data. [2025-10-20T19:05:29.863215] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-01. [2025-10-20T19:05:30.220506] [DEBUG] [VALIDATOR ] GTLS: All expected trading days are present. [2025-10-20T19:05:30.220617] [DEBUG] [VALIDATOR ] GTLS: Primary data validation passed. [2025-10-20T19:05:30.221507] [INFO ] [MAIN ] GTLS: Fetched primary data is valid. [2025-10-20T19:05:30.221559] [DEBUG] [LOCAL ] GTLS: Saving primary data. [2025-10-20T19:05:30.222844] [DEBUG] [LOCAL ] GTLS: Trimmed 61617 records to 43326 (keeping 200 days of data) [2025-10-20T19:05:30.474391] [DEBUG] [LOCAL ] GTLS: Primary data saved (43326 records). [2025-10-20T19:05:30.474517] [INFO ] [MAIN ] GTLS: Full history saved. [2025-10-20T19:05:30.474592] [INFO ] [MAIN ] Update Progress: 976/2367 symbols completed. GTLS took 21.1s. Estimated remaining: 2h 58m 46s. Rate: 7.8 symbols/min [2025-10-20T19:05:30.666686] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-08. [2025-10-20T19:05:30.669589] [INFO ] [MAIN ] GUG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:30.684051] [DEBUG] [LOCAL ] GUG: Loaded 12286 existing records from primary storage. [2025-10-20T19:05:30.684361] [INFO ] [MAIN ] GUG: Loaded 12286 existing records. [2025-10-20T19:05:30.897052] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T19:05:30.903280] [WARN ] [MAIN ] GUG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:30.913930] [INFO ] [MAIN ] GUG: Wiping existing primary data before full refresh. [2025-10-20T19:05:30.914370] [INFO ] [LOCAL ] GUG: Starting deletion of primary data. [2025-10-20T19:05:30.914683] [DEBUG] [LOCAL ] GUG: Deleted primary data file. [2025-10-20T19:05:30.914745] [INFO ] [MAIN ] GUG: Starting full primary refresh. [2025-10-20T19:05:31.291202] [INFO ] [REMOTE ] GUG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:31.308625] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T19:05:31.451385] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-01. [2025-10-20T19:05:32.251363] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-02. [2025-10-20T19:05:32.525686] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T19:05:32.555798] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:05:33.051364] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-02. [2025-10-20T19:05:33.431438] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T19:05:33.851387] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-04. [2025-10-20T19:05:33.973416] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T19:05:34.651426] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-03. [2025-10-20T19:05:35.451394] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-09. [2025-10-20T19:05:35.560698] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:05:36.027988] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T19:05:36.251391] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-03. [2025-10-20T19:05:37.051397] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-04. [2025-10-20T19:05:37.098976] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T19:05:37.104278] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T19:05:37.851366] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-04. [2025-10-20T19:05:38.409899] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T19:05:38.651315] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-05. [2025-10-20T19:05:38.752256] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T19:05:39.451420] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-05. [2025-10-20T19:05:39.977368] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T19:05:40.163407] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T19:05:40.251336] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-06. [2025-10-20T19:05:41.051372] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-06. [2025-10-20T19:05:41.106297] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T19:05:41.851363] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-10. [2025-10-20T19:05:42.284530] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T19:05:42.651380] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-07. [2025-10-20T19:05:43.451375] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-05. [2025-10-20T19:05:43.624723] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T19:05:43.632111] [INFO ] [REMOTE ] GTES: Full fetch complete (73921 records). [2025-10-20T19:05:43.632257] [DEBUG] [VALIDATOR ] GTES: Validating primary data. [2025-10-20T19:05:44.046326] [DEBUG] [VALIDATOR ] GTES: All expected trading days are present. [2025-10-20T19:05:44.046497] [DEBUG] [VALIDATOR ] GTES: Primary data validation passed. [2025-10-20T19:05:44.047761] [INFO ] [MAIN ] GTES: Fetched primary data is valid. [2025-10-20T19:05:44.047971] [DEBUG] [LOCAL ] GTES: Saving primary data. [2025-10-20T19:05:44.068538] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:05:44.076396] [DEBUG] [LOCAL ] GTES: Trimmed 73921 records to 50321 (keeping 200 days of data) [2025-10-20T19:05:44.276859] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-07. [2025-10-20T19:05:44.373000] [DEBUG] [LOCAL ] GTES: Primary data saved (50321 records). [2025-10-20T19:05:44.373134] [INFO ] [MAIN ] GTES: Full history saved. [2025-10-20T19:05:44.373854] [INFO ] [MAIN ] Update Progress: 977/2367 symbols completed. GTES took 37.7s. Estimated remaining: 2h 58m 47s. Rate: 7.8 symbols/min [2025-10-20T19:05:44.555887] [INFO ] [MAIN ] GUT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:44.572013] [DEBUG] [LOCAL ] GUT: Loaded 15859 existing records from primary storage. [2025-10-20T19:05:44.572245] [INFO ] [MAIN ] GUT: Loaded 15859 existing records. [2025-10-20T19:05:44.758299] [WARN ] [MAIN ] GUT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:44.758527] [INFO ] [MAIN ] GUT: Wiping existing primary data before full refresh. [2025-10-20T19:05:44.758573] [INFO ] [LOCAL ] GUT: Starting deletion of primary data. [2025-10-20T19:05:44.758844] [DEBUG] [LOCAL ] GUT: Deleted primary data file. [2025-10-20T19:05:44.758902] [INFO ] [MAIN ] GUT: Starting full primary refresh. [2025-10-20T19:05:45.040311] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T19:05:45.056748] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-08. [2025-10-20T19:05:45.161952] [INFO ] [REMOTE ] GUT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:45.344752] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T19:05:45.851436] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-06. [2025-10-20T19:05:46.497113] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T19:05:46.651368] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-08. [2025-10-20T19:05:47.183836] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T19:05:47.408552] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:05:47.451380] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-07. [2025-10-20T19:05:48.251366] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-01. [2025-10-20T19:05:48.897092] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T19:05:49.051382] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-09. [2025-10-20T19:05:49.260994] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T19:05:49.851370] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-08. [2025-10-20T19:05:50.291535] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T19:05:50.651366] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-09. [2025-10-20T19:05:51.439475] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T19:05:51.441063] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:05:51.453312] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-02. [2025-10-20T19:05:52.251392] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-09. [2025-10-20T19:05:52.538764] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T19:05:53.051423] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-03. [2025-10-20T19:05:53.729952] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T19:05:53.851364] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-10. [2025-10-20T19:05:53.997936] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T19:05:54.651379] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-04. [2025-10-20T19:05:55.314014] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:05:55.318876] [INFO ] [REMOTE ] GTY: Full fetch complete (43839 records). [2025-10-20T19:05:55.318988] [DEBUG] [VALIDATOR ] GTY: Validating primary data. [2025-10-20T19:05:55.459951] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-10. [2025-10-20T19:05:55.666463] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T19:05:55.718806] [DEBUG] [VALIDATOR ] GTY: All expected trading days are present. [2025-10-20T19:05:55.718872] [DEBUG] [VALIDATOR ] GTY: Primary data validation passed. [2025-10-20T19:05:55.719600] [INFO ] [MAIN ] GTY: Fetched primary data is valid. [2025-10-20T19:05:55.719684] [DEBUG] [LOCAL ] GTY: Saving primary data. [2025-10-20T19:05:55.720559] [DEBUG] [LOCAL ] GTY: Trimmed 43839 records to 30157 (keeping 200 days of data) [2025-10-20T19:05:55.898883] [DEBUG] [LOCAL ] GTY: Primary data saved (30157 records). [2025-10-20T19:05:55.899037] [INFO ] [MAIN ] GTY: Full history saved. [2025-10-20T19:05:55.899120] [INFO ] [MAIN ] Update Progress: 978/2367 symbols completed. GTY took 34.0s. Estimated remaining: 2h 58m 45s. Rate: 7.8 symbols/min [2025-10-20T19:05:56.067001] [INFO ] [MAIN ] GVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:56.100605] [DEBUG] [LOCAL ] GVA: Loaded 33145 existing records from primary storage. [2025-10-20T19:05:56.100890] [INFO ] [MAIN ] GVA: Loaded 33145 existing records. [2025-10-20T19:05:56.260361] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-10. [2025-10-20T19:05:56.280671] [WARN ] [MAIN ] GVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:56.281224] [INFO ] [MAIN ] GVA: Wiping existing primary data before full refresh. [2025-10-20T19:05:56.281437] [INFO ] [LOCAL ] GVA: Starting deletion of primary data. [2025-10-20T19:05:56.282341] [DEBUG] [LOCAL ] GVA: Deleted primary data file. [2025-10-20T19:05:56.282408] [INFO ] [MAIN ] GVA: Starting full primary refresh. [2025-10-20T19:05:56.399996] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:05:56.418333] [INFO ] [REMOTE ] GUG: Full fetch complete (18904 records). [2025-10-20T19:05:56.441741] [DEBUG] [VALIDATOR ] GUG: Validating primary data. [2025-10-20T19:05:56.929547] [INFO ] [REMOTE ] GVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:57.030537] [DEBUG] [VALIDATOR ] GUG: All expected trading days are present. [2025-10-20T19:05:57.030605] [DEBUG] [VALIDATOR ] GUG: Primary data validation passed. [2025-10-20T19:05:57.031003] [INFO ] [MAIN ] GUG: Fetched primary data is valid. [2025-10-20T19:05:57.031062] [DEBUG] [LOCAL ] GUG: Saving primary data. [2025-10-20T19:05:57.031566] [DEBUG] [LOCAL ] GUG: Trimmed 18904 records to 12194 (keeping 200 days of data) [2025-10-20T19:05:57.062918] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-01. [2025-10-20T19:05:57.108011] [DEBUG] [LOCAL ] GUG: Primary data saved (12194 records). [2025-10-20T19:05:57.108110] [INFO ] [MAIN ] GUG: Full history saved. [2025-10-20T19:05:57.108183] [INFO ] [MAIN ] Update Progress: 979/2367 symbols completed. GUG took 26.6s. Estimated remaining: 2h 58m 28s. Rate: 7.8 symbols/min [2025-10-20T19:05:57.277775] [INFO ] [MAIN ] GWH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:57.296000] [DEBUG] [LOCAL ] GWH: Loaded 19414 existing records from primary storage. [2025-10-20T19:05:57.296231] [INFO ] [MAIN ] GWH: Loaded 19414 existing records. [2025-10-20T19:05:57.478776] [WARN ] [MAIN ] GWH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:57.479020] [INFO ] [MAIN ] GWH: Wiping existing primary data before full refresh. [2025-10-20T19:05:57.479077] [INFO ] [LOCAL ] GWH: Starting deletion of primary data. [2025-10-20T19:05:57.479312] [DEBUG] [LOCAL ] GWH: Deleted primary data file. [2025-10-20T19:05:57.479351] [INFO ] [MAIN ] GWH: Starting full primary refresh. [2025-10-20T19:05:57.847157] [INFO ] [REMOTE ] GWH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:57.851321] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-01. [2025-10-20T19:05:57.940154] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:05:57.947688] [INFO ] [REMOTE ] GTN: Full fetch complete (65985 records). [2025-10-20T19:05:57.947945] [DEBUG] [VALIDATOR ] GTN: Validating primary data. [2025-10-20T19:05:58.349578] [DEBUG] [VALIDATOR ] GTN: All expected trading days are present. [2025-10-20T19:05:58.349745] [DEBUG] [VALIDATOR ] GTN: Primary data validation passed. [2025-10-20T19:05:58.350748] [INFO ] [MAIN ] GTN: Fetched primary data is valid. [2025-10-20T19:05:58.350836] [DEBUG] [LOCAL ] GTN: Saving primary data. [2025-10-20T19:05:58.352466] [DEBUG] [LOCAL ] GTN: Trimmed 65985 records to 44989 (keeping 200 days of data) [2025-10-20T19:05:58.481675] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:05:58.645729] [DEBUG] [LOCAL ] GTN: Primary data saved (44989 records). [2025-10-20T19:05:58.645879] [INFO ] [MAIN ] GTN: Full history saved. [2025-10-20T19:05:58.645967] [INFO ] [MAIN ] Update Progress: 980/2367 symbols completed. GTN took 37.8s. Estimated remaining: 2h 58m 12s. Rate: 7.8 symbols/min [2025-10-20T19:05:58.657349] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-05. [2025-10-20T19:05:58.772097] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T19:05:58.857304] [INFO ] [MAIN ] GWRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:58.901275] [DEBUG] [LOCAL ] GWRE: Loaded 40437 existing records from primary storage. [2025-10-20T19:05:58.901748] [INFO ] [MAIN ] GWRE: Loaded 40437 existing records. [2025-10-20T19:05:59.102799] [WARN ] [MAIN ] GWRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:59.103051] [INFO ] [MAIN ] GWRE: Wiping existing primary data before full refresh. [2025-10-20T19:05:59.103102] [INFO ] [LOCAL ] GWRE: Starting deletion of primary data. [2025-10-20T19:05:59.103302] [DEBUG] [LOCAL ] GWRE: Deleted primary data file. [2025-10-20T19:05:59.103340] [INFO ] [MAIN ] GWRE: Starting full primary refresh. [2025-10-20T19:05:59.453215] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-02. [2025-10-20T19:05:59.453115] [INFO ] [REMOTE ] GWRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:59.472361] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T19:06:00.251381] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-06. [2025-10-20T19:06:00.333304] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T19:06:01.051360] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-03. [2025-10-20T19:06:01.395054] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:06:01.851335] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-07. [2025-10-20T19:06:02.166460] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:06:02.651368] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-04. [2025-10-20T19:06:03.032868] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T19:06:03.451419] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-01. [2025-10-20T19:06:03.537253] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T19:06:04.251419] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-05. [2025-10-20T19:06:05.051346] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-02. [2025-10-20T19:06:05.208942] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:06:05.293703] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T19:06:05.851363] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-08. [2025-10-20T19:06:06.368476] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T19:06:06.651377] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-06. [2025-10-20T19:06:06.899222] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:06:07.451362] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-02. [2025-10-20T19:06:07.902195] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T19:06:08.251351] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-09. [2025-10-20T19:06:08.738541] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:06:09.051377] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-03. [2025-10-20T19:06:09.157450] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T19:06:09.851372] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-10. [2025-10-20T19:06:10.651365] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-03. [2025-10-20T19:06:10.846838] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:06:10.846523] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T19:06:10.859862] [INFO ] [REMOTE ] GUT: Full fetch complete (22524 records). [2025-10-20T19:06:10.860009] [DEBUG] [VALIDATOR ] GUT: Validating primary data. [2025-10-20T19:06:11.234618] [DEBUG] [VALIDATOR ] GUT: All expected trading days are present. [2025-10-20T19:06:11.234757] [DEBUG] [VALIDATOR ] GUT: Primary data validation passed. [2025-10-20T19:06:11.235291] [INFO ] [MAIN ] GUT: Fetched primary data is valid. [2025-10-20T19:06:11.235334] [DEBUG] [LOCAL ] GUT: Saving primary data. [2025-10-20T19:06:11.235964] [DEBUG] [LOCAL ] GUT: Trimmed 22524 records to 15737 (keeping 200 days of data) [2025-10-20T19:06:11.326742] [DEBUG] [LOCAL ] GUT: Primary data saved (15737 records). [2025-10-20T19:06:11.326880] [INFO ] [MAIN ] GUT: Full history saved. [2025-10-20T19:06:11.326958] [INFO ] [MAIN ] Update Progress: 981/2367 symbols completed. GUT took 27.0s. Estimated remaining: 2h 58m 11s. Rate: 7.8 symbols/min [2025-10-20T19:06:11.456541] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-04. [2025-10-20T19:06:11.511008] [INFO ] [MAIN ] GWW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:11.532482] [DEBUG] [LOCAL ] GWW: Loaded 19822 existing records from primary storage. [2025-10-20T19:06:11.532862] [INFO ] [MAIN ] GWW: Loaded 19822 existing records. [2025-10-20T19:06:11.732351] [WARN ] [MAIN ] GWW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:11.732597] [INFO ] [MAIN ] GWW: Wiping existing primary data before full refresh. [2025-10-20T19:06:11.732667] [INFO ] [LOCAL ] GWW: Starting deletion of primary data. [2025-10-20T19:06:11.732904] [DEBUG] [LOCAL ] GWW: Deleted primary data file. [2025-10-20T19:06:11.732963] [INFO ] [MAIN ] GWW: Starting full primary refresh. [2025-10-20T19:06:12.129838] [INFO ] [REMOTE ] GWW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:12.251389] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-01. [2025-10-20T19:06:12.284846] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:06:12.768800] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T19:06:13.051638] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-04. [2025-10-20T19:06:13.155252] [DEBUG] [REMOTE ] GVA: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T19:06:13.155572] [WARN ] [REMOTE ] GVA: No data returned for month 2025-04 [2025-10-20T19:06:13.155700] [FATAL] [REMOTE ] GVA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:06:13.155785] [FATAL] [MAIN ] GVA: Failed to fetch primary data. [2025-10-20T19:06:13.155850] [WARN ] [MAIN ] GVA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:06:13.155924] [INFO ] [LOCAL ] GVA: Starting deletion of primary data. [2025-10-20T19:06:13.155987] [DEBUG] [LOCAL ] GVA: No primary data file found to delete. [2025-10-20T19:06:13.156023] [INFO ] [LOCAL ] GVA: Starting deletion of derivative data. [2025-10-20T19:06:13.156223] [DEBUG] [LOCAL ] GVA: Deleted derivative data file. [2025-10-20T19:06:13.156305] [INFO ] [MAIN ] Update Progress: 982/2367 symbols completed. GVA took 17.3s. Estimated remaining: 2h 57m 55s. Rate: 7.8 symbols/min [2025-10-20T19:06:13.334036] [INFO ] [MAIN ] GXO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:13.380811] [DEBUG] [LOCAL ] GXO: Loaded 47418 existing records from primary storage. [2025-10-20T19:06:13.381152] [INFO ] [MAIN ] GXO: Loaded 47418 existing records. [2025-10-20T19:06:13.579389] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T19:06:13.584902] [WARN ] [MAIN ] GXO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:13.585202] [INFO ] [MAIN ] GXO: Wiping existing primary data before full refresh. [2025-10-20T19:06:13.589991] [INFO ] [LOCAL ] GXO: Starting deletion of primary data. [2025-10-20T19:06:13.590214] [DEBUG] [LOCAL ] GXO: Deleted primary data file. [2025-10-20T19:06:13.590259] [INFO ] [MAIN ] GXO: Starting full primary refresh. [2025-10-20T19:06:13.861186] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-02. [2025-10-20T19:06:13.994203] [INFO ] [REMOTE ] GXO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:14.651402] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-05. [2025-10-20T19:06:15.101859] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T19:06:15.451388] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-03. [2025-10-20T19:06:16.251468] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-07. [2025-10-20T19:06:16.455995] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T19:06:16.667271] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T19:06:17.051401] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-04. [2025-10-20T19:06:17.170670] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T19:06:17.851376] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-06. [2025-10-20T19:06:18.338105] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T19:06:18.651382] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-08. [2025-10-20T19:06:19.337708] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T19:06:19.451379] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-05. [2025-10-20T19:06:19.696041] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T19:06:20.251389] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-07. [2025-10-20T19:06:20.453484] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:06:21.051474] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-01. [2025-10-20T19:06:21.840776] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T19:06:21.851343] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-09. [2025-10-20T19:06:22.572186] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T19:06:22.651343] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-02. [2025-10-20T19:06:22.990830] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T19:06:23.451371] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-10. [2025-10-20T19:06:24.256469] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-06. [2025-10-20T19:06:24.271570] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T19:06:25.051362] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-03. [2025-10-20T19:06:25.100379] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T19:06:25.106548] [INFO ] [REMOTE ] GWH: Full fetch complete (24087 records). [2025-10-20T19:06:25.106702] [DEBUG] [VALIDATOR ] GWH: Validating primary data. [2025-10-20T19:06:25.410859] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T19:06:25.497964] [DEBUG] [VALIDATOR ] GWH: All expected trading days are present. [2025-10-20T19:06:25.498048] [DEBUG] [VALIDATOR ] GWH: Primary data validation passed. [2025-10-20T19:06:25.498565] [INFO ] [MAIN ] GWH: Fetched primary data is valid. [2025-10-20T19:06:25.498604] [DEBUG] [LOCAL ] GWH: Saving primary data. [2025-10-20T19:06:25.499348] [DEBUG] [LOCAL ] GWH: Trimmed 24087 records to 19537 (keeping 200 days of data) [2025-10-20T19:06:25.608193] [DEBUG] [LOCAL ] GWH: Primary data saved (19537 records). [2025-10-20T19:06:25.608344] [INFO ] [MAIN ] GWH: Full history saved. [2025-10-20T19:06:25.608426] [INFO ] [MAIN ] Update Progress: 983/2367 symbols completed. GWH took 28.5s. Estimated remaining: 2h 57m 54s. Rate: 7.8 symbols/min [2025-10-20T19:06:25.776396] [INFO ] [MAIN ] GYLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:25.780312] [DEBUG] [LOCAL ] GYLD: Loaded 1562 existing records from primary storage. [2025-10-20T19:06:25.780442] [INFO ] [MAIN ] GYLD: Loaded 1562 existing records. [2025-10-20T19:06:25.856518] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-07. [2025-10-20T19:06:25.956490] [WARN ] [MAIN ] GYLD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:25.956682] [INFO ] [MAIN ] GYLD: Wiping existing primary data before full refresh. [2025-10-20T19:06:25.956738] [INFO ] [LOCAL ] GYLD: Starting deletion of primary data. [2025-10-20T19:06:25.956908] [DEBUG] [LOCAL ] GYLD: Deleted primary data file. [2025-10-20T19:06:25.956965] [INFO ] [MAIN ] GYLD: Starting full primary refresh. [2025-10-20T19:06:26.328921] [INFO ] [REMOTE ] GYLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:26.651408] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-08. [2025-10-20T19:06:26.779605] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:06:26.825209] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T19:06:27.451382] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-01. [2025-10-20T19:06:27.818474] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-01 (0.37 seconds). [2025-10-20T19:06:28.130526] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:06:28.251377] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-04. [2025-10-20T19:06:29.051400] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-08. [2025-10-20T19:06:29.851416] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-09. [2025-10-20T19:06:29.919606] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T19:06:30.170967] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T19:06:30.651401] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-02. [2025-10-20T19:06:31.026091] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T19:06:31.161040] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T19:06:31.451383] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-03. [2025-10-20T19:06:31.887337] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-10-20T19:06:32.251394] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-04. [2025-10-20T19:06:32.687505] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-10-20T19:06:33.051376] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-05. [2025-10-20T19:06:33.851369] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-05. [2025-10-20T19:06:34.300845] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T19:06:34.657312] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-06. [2025-10-20T19:06:34.677195] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T19:06:35.114731] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T19:06:35.451351] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-10. [2025-10-20T19:06:36.251357] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-09. [2025-10-20T19:06:36.942606] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:06:36.948181] [INFO ] [REMOTE ] GWRE: Full fetch complete (60257 records). [2025-10-20T19:06:36.948304] [DEBUG] [VALIDATOR ] GWRE: Validating primary data. [2025-10-20T19:06:37.056503] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-06. [2025-10-20T19:06:37.336786] [DEBUG] [VALIDATOR ] GWRE: All expected trading days are present. [2025-10-20T19:06:37.336903] [DEBUG] [VALIDATOR ] GWRE: Primary data validation passed. [2025-10-20T19:06:37.337787] [INFO ] [MAIN ] GWRE: Fetched primary data is valid. [2025-10-20T19:06:37.337954] [DEBUG] [LOCAL ] GWRE: Saving primary data. [2025-10-20T19:06:37.339389] [DEBUG] [LOCAL ] GWRE: Trimmed 60257 records to 39801 (keeping 200 days of data) [2025-10-20T19:06:37.454341] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T19:06:37.601345] [DEBUG] [LOCAL ] GWRE: Primary data saved (39801 records). [2025-10-20T19:06:37.601492] [INFO ] [MAIN ] GWRE: Full history saved. [2025-10-20T19:06:37.601570] [INFO ] [MAIN ] Update Progress: 984/2367 symbols completed. GWRE took 39.0s. Estimated remaining: 2h 57m 52s. Rate: 7.8 symbols/min [2025-10-20T19:06:37.767682] [INFO ] [MAIN ] H: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:37.811183] [DEBUG] [LOCAL ] H: Loaded 45047 existing records from primary storage. [2025-10-20T19:06:37.811471] [INFO ] [MAIN ] H: Loaded 45047 existing records. [2025-10-20T19:06:37.856687] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-07. [2025-10-20T19:06:37.990085] [WARN ] [MAIN ] H: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:37.990299] [INFO ] [MAIN ] H: Wiping existing primary data before full refresh. [2025-10-20T19:06:37.990346] [INFO ] [LOCAL ] H: Starting deletion of primary data. [2025-10-20T19:06:37.990560] [DEBUG] [LOCAL ] H: Deleted primary data file. [2025-10-20T19:06:37.990614] [INFO ] [MAIN ] H: Starting full primary refresh. [2025-10-20T19:06:38.395942] [INFO ] [REMOTE ] H: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:38.427615] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T19:06:38.442289] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T19:06:38.651443] [INFO ] [REMOTE ] H: Fetching primary data for 2025-01. [2025-10-20T19:06:39.451416] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-07. [2025-10-20T19:06:40.142446] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:06:40.251363] [INFO ] [REMOTE ] H: Fetching primary data for 2025-02. [2025-10-20T19:06:40.584625] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T19:06:41.051362] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-10. [2025-10-20T19:06:41.821611] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:06:41.851328] [INFO ] [REMOTE ] H: Fetching primary data for 2025-03. [2025-10-20T19:06:42.116065] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T19:06:42.119472] [INFO ] [REMOTE ] GWW: Full fetch complete (29083 records). [2025-10-20T19:06:42.119553] [DEBUG] [VALIDATOR ] GWW: Validating primary data. [2025-10-20T19:06:42.535245] [DEBUG] [VALIDATOR ] GWW: All expected trading days are present. [2025-10-20T19:06:42.535334] [DEBUG] [VALIDATOR ] GWW: Primary data validation passed. [2025-10-20T19:06:42.535913] [INFO ] [MAIN ] GWW: Fetched primary data is valid. [2025-10-20T19:06:42.535975] [DEBUG] [LOCAL ] GWW: Saving primary data. [2025-10-20T19:06:42.536842] [DEBUG] [LOCAL ] GWW: Trimmed 29083 records to 19418 (keeping 200 days of data) [2025-10-20T19:06:42.655606] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-08. [2025-10-20T19:06:42.658700] [DEBUG] [LOCAL ] GWW: Primary data saved (19418 records). [2025-10-20T19:06:42.659693] [INFO ] [MAIN ] GWW: Full history saved. [2025-10-20T19:06:42.659802] [INFO ] [MAIN ] Update Progress: 985/2367 symbols completed. GWW took 31.3s. Estimated remaining: 2h 57m 41s. Rate: 7.8 symbols/min [2025-10-20T19:06:42.836544] [INFO ] [MAIN ] HAE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:42.874929] [DEBUG] [LOCAL ] HAE: Loaded 40001 existing records from primary storage. [2025-10-20T19:06:42.875207] [INFO ] [MAIN ] HAE: Loaded 40001 existing records. [2025-10-20T19:06:43.056279] [WARN ] [MAIN ] HAE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:43.056458] [INFO ] [MAIN ] HAE: Wiping existing primary data before full refresh. [2025-10-20T19:06:43.056505] [INFO ] [LOCAL ] HAE: Starting deletion of primary data. [2025-10-20T19:06:43.056733] [DEBUG] [LOCAL ] HAE: Deleted primary data file. [2025-10-20T19:06:43.056800] [INFO ] [MAIN ] HAE: Starting full primary refresh. [2025-10-20T19:06:43.222343] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-10-20T19:06:43.426536] [INFO ] [REMOTE ] HAE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:43.451331] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-01. [2025-10-20T19:06:43.555450] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T19:06:44.251391] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-09. [2025-10-20T19:06:44.722745] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T19:06:44.860719] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:06:45.051414] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-08. [2025-10-20T19:06:45.851368] [INFO ] [REMOTE ] H: Fetching primary data for 2025-04. [2025-10-20T19:06:46.651364] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-10. [2025-10-20T19:06:46.678033] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T19:06:47.451422] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-09. [2025-10-20T19:06:47.508388] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:06:47.509247] [INFO ] [REMOTE ] GYLD: Full fetch complete (2078 records). [2025-10-20T19:06:47.509315] [DEBUG] [VALIDATOR ] GYLD: Validating primary data. [2025-10-20T19:06:47.820215] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T19:06:47.911815] [DEBUG] [VALIDATOR ] GYLD: All expected trading days are present. [2025-10-20T19:06:47.911895] [DEBUG] [VALIDATOR ] GYLD: Primary data validation passed. [2025-10-20T19:06:47.912005] [INFO ] [MAIN ] GYLD: Fetched primary data is valid. [2025-10-20T19:06:47.912071] [DEBUG] [LOCAL ] GYLD: Saving primary data. [2025-10-20T19:06:47.912406] [DEBUG] [LOCAL ] GYLD: Trimmed 2078 records to 1544 (keeping 200 days of data) [2025-10-20T19:06:47.922863] [DEBUG] [LOCAL ] GYLD: Primary data saved (1544 records). [2025-10-20T19:06:47.922992] [INFO ] [MAIN ] GYLD: Full history saved. [2025-10-20T19:06:47.923073] [INFO ] [MAIN ] Update Progress: 986/2367 symbols completed. GYLD took 22.3s. Estimated remaining: 2h 57m 30s. Rate: 7.8 symbols/min [2025-10-20T19:06:48.088612] [INFO ] [MAIN ] HAFN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:48.125295] [DEBUG] [LOCAL ] HAFN: Loaded 43553 existing records from primary storage. [2025-10-20T19:06:48.125549] [INFO ] [MAIN ] HAFN: Loaded 43553 existing records. [2025-10-20T19:06:48.256623] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-02. [2025-10-20T19:06:48.311138] [WARN ] [MAIN ] HAFN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:48.311589] [INFO ] [MAIN ] HAFN: Wiping existing primary data before full refresh. [2025-10-20T19:06:48.311638] [INFO ] [LOCAL ] HAFN: Starting deletion of primary data. [2025-10-20T19:06:48.311922] [DEBUG] [LOCAL ] HAFN: Deleted primary data file. [2025-10-20T19:06:48.311964] [INFO ] [MAIN ] HAFN: Starting full primary refresh. [2025-10-20T19:06:48.660196] [INFO ] [REMOTE ] HAFN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:49.008054] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T19:06:49.051326] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-10. [2025-10-20T19:06:49.851369] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-01. [2025-10-20T19:06:49.973058] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:06:50.651404] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-03. [2025-10-20T19:06:50.709123] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T19:06:50.715533] [INFO ] [REMOTE ] GXO: Full fetch complete (69503 records). [2025-10-20T19:06:50.715628] [DEBUG] [VALIDATOR ] GXO: Validating primary data. [2025-10-20T19:06:51.096013] [DEBUG] [VALIDATOR ] GXO: All expected trading days are present. [2025-10-20T19:06:51.096112] [DEBUG] [VALIDATOR ] GXO: Primary data validation passed. [2025-10-20T19:06:51.097773] [INFO ] [MAIN ] GXO: Fetched primary data is valid. [2025-10-20T19:06:51.097844] [DEBUG] [LOCAL ] GXO: Saving primary data. [2025-10-20T19:06:51.099510] [DEBUG] [LOCAL ] GXO: Trimmed 69503 records to 46617 (keeping 200 days of data) [2025-10-20T19:06:51.368871] [DEBUG] [LOCAL ] GXO: Primary data saved (46617 records). [2025-10-20T19:06:51.369026] [INFO ] [MAIN ] GXO: Full history saved. [2025-10-20T19:06:51.369111] [INFO ] [MAIN ] Update Progress: 987/2367 symbols completed. GXO took 38.2s. Estimated remaining: 2h 57m 16s. Rate: 7.8 symbols/min [2025-10-20T19:06:51.467505] [INFO ] [REMOTE ] H: Fetching primary data for 2025-05. [2025-10-20T19:06:51.540379] [INFO ] [MAIN ] HAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:51.591497] [DEBUG] [LOCAL ] HAL: Loaded 54498 existing records from primary storage. [2025-10-20T19:06:51.591867] [INFO ] [MAIN ] HAL: Loaded 54498 existing records. [2025-10-20T19:06:51.796996] [WARN ] [MAIN ] HAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:51.797507] [INFO ] [MAIN ] HAL: Wiping existing primary data before full refresh. [2025-10-20T19:06:51.799600] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T19:06:51.799724] [INFO ] [LOCAL ] HAL: Starting deletion of primary data. [2025-10-20T19:06:51.809345] [DEBUG] [LOCAL ] HAL: Deleted primary data file. [2025-10-20T19:06:51.810125] [INFO ] [MAIN ] HAL: Starting full primary refresh. [2025-10-20T19:06:52.192050] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:06:52.196516] [INFO ] [REMOTE ] HAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:52.251340] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-04. [2025-10-20T19:06:52.773099] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:06:53.051355] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-01. [2025-10-20T19:06:53.851356] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-02. [2025-10-20T19:06:53.980798] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T19:06:54.651419] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-05. [2025-10-20T19:06:55.018562] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T19:06:55.451347] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-02. [2025-10-20T19:06:55.669013] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T19:06:55.748883] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T19:06:56.251359] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-03. [2025-10-20T19:06:56.997323] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T19:06:57.051354] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-03. [2025-10-20T19:06:57.851349] [INFO ] [REMOTE ] H: Fetching primary data for 2025-06. [2025-10-20T19:06:58.021207] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T19:06:58.651359] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-06. [2025-10-20T19:06:59.078152] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T19:06:59.120715] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T19:06:59.451362] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-04. [2025-10-20T19:06:59.821793] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T19:07:00.251401] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-04. [2025-10-20T19:07:01.051445] [INFO ] [REMOTE ] H: Fetching primary data for 2025-07. [2025-10-20T19:07:01.482243] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T19:07:01.851402] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-05. [2025-10-20T19:07:02.297402] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T19:07:02.376924] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T19:07:02.651363] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-05. [2025-10-20T19:07:03.451385] [INFO ] [REMOTE ] H: Fetching primary data for 2025-08. [2025-10-20T19:07:03.568788] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T19:07:04.251356] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-06. [2025-10-20T19:07:04.718171] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-10-20T19:07:05.051428] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-06. [2025-10-20T19:07:05.197507] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T19:07:05.786133] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:07:05.851396] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-07. [2025-10-20T19:07:06.651367] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-07. [2025-10-20T19:07:07.016927] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T19:07:07.132469] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T19:07:07.451384] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-07. [2025-10-20T19:07:08.032998] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T19:07:08.251366] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-08. [2025-10-20T19:07:09.051388] [INFO ] [REMOTE ] H: Fetching primary data for 2025-09. [2025-10-20T19:07:09.486275] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T19:07:09.857005] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-08. [2025-10-20T19:07:09.867870] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T19:07:10.655728] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-09. [2025-10-20T19:07:10.657385] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T19:07:11.429416] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:07:11.451345] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-09. [2025-10-20T19:07:12.251383] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-08. [2025-10-20T19:07:12.305191] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T19:07:13.051463] [INFO ] [REMOTE ] H: Fetching primary data for 2025-10. [2025-10-20T19:07:13.134267] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T19:07:13.851497] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-10. [2025-10-20T19:07:14.062495] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:07:14.651376] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-09. [2025-10-20T19:07:14.777179] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T19:07:14.784032] [INFO ] [REMOTE ] H: Full fetch complete (63179 records). [2025-10-20T19:07:14.784221] [DEBUG] [VALIDATOR ] H: Validating primary data. [2025-10-20T19:07:15.176708] [DEBUG] [VALIDATOR ] H: All expected trading days are present. [2025-10-20T19:07:15.176896] [DEBUG] [VALIDATOR ] H: Primary data validation passed. [2025-10-20T19:07:15.177787] [INFO ] [MAIN ] H: Fetched primary data is valid. [2025-10-20T19:07:15.177851] [DEBUG] [LOCAL ] H: Saving primary data. [2025-10-20T19:07:15.179224] [DEBUG] [LOCAL ] H: Trimmed 63179 records to 44459 (keeping 200 days of data) [2025-10-20T19:07:15.451882] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-10. [2025-10-20T19:07:15.480851] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T19:07:15.491263] [DEBUG] [LOCAL ] H: Primary data saved (44459 records). [2025-10-20T19:07:15.491427] [INFO ] [MAIN ] H: Full history saved. [2025-10-20T19:07:15.491982] [INFO ] [MAIN ] Update Progress: 988/2367 symbols completed. H took 37.9s. Estimated remaining: 2h 57m 31s. Rate: 7.8 symbols/min [2025-10-20T19:07:15.492146] [INFO ] [REMOTE ] HAE: Full fetch complete (59160 records). [2025-10-20T19:07:15.539784] [DEBUG] [VALIDATOR ] HAE: Validating primary data. [2025-10-20T19:07:15.817339] [INFO ] [MAIN ] HASI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:16.158722] [DEBUG] [VALIDATOR ] HAE: All expected trading days are present. [2025-10-20T19:07:16.160945] [DEBUG] [VALIDATOR ] HAE: Primary data validation passed. [2025-10-20T19:07:16.162216] [INFO ] [MAIN ] HAE: Fetched primary data is valid. [2025-10-20T19:07:16.162318] [DEBUG] [LOCAL ] HAE: Saving primary data. [2025-10-20T19:07:16.164492] [DEBUG] [LOCAL ] HASI: Loaded 48399 existing records from primary storage. [2025-10-20T19:07:16.165554] [INFO ] [MAIN ] HASI: Loaded 48399 existing records. [2025-10-20T19:07:16.199278] [DEBUG] [LOCAL ] HAE: Trimmed 59160 records to 39456 (keeping 200 days of data) [2025-10-20T19:07:16.600995] [WARN ] [MAIN ] HASI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:16.608699] [INFO ] [MAIN ] HASI: Wiping existing primary data before full refresh. [2025-10-20T19:07:16.612560] [INFO ] [LOCAL ] HASI: Starting deletion of primary data. [2025-10-20T19:07:16.613212] [DEBUG] [LOCAL ] HASI: Deleted primary data file. [2025-10-20T19:07:16.617165] [INFO ] [MAIN ] HASI: Starting full primary refresh. [2025-10-20T19:07:16.666588] [DEBUG] [LOCAL ] HAE: Primary data saved (39456 records). [2025-10-20T19:07:16.666766] [INFO ] [MAIN ] HAE: Full history saved. [2025-10-20T19:07:16.666982] [INFO ] [MAIN ] Update Progress: 989/2367 symbols completed. HAE took 34.0s. Estimated remaining: 2h 57m 14s. Rate: 7.8 symbols/min [2025-10-20T19:07:17.097585] [INFO ] [MAIN ] HAYW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:17.100934] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-09 (2.45 seconds). [2025-10-20T19:07:17.176395] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-10. [2025-10-20T19:07:17.282101] [INFO ] [REMOTE ] HASI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:17.284713] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-01. [2025-10-20T19:07:17.284327] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T19:07:17.292498] [INFO ] [REMOTE ] HAFN: Full fetch complete (64787 records). [2025-10-20T19:07:17.292704] [DEBUG] [VALIDATOR ] HAFN: Validating primary data. [2025-10-20T19:07:17.416106] [DEBUG] [LOCAL ] HAYW: Loaded 49622 existing records from primary storage. [2025-10-20T19:07:17.427808] [INFO ] [MAIN ] HAYW: Loaded 49622 existing records. [2025-10-20T19:07:17.829061] [WARN ] [MAIN ] HAYW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:17.858279] [INFO ] [MAIN ] HAYW: Wiping existing primary data before full refresh. [2025-10-20T19:07:17.858407] [INFO ] [LOCAL ] HAYW: Starting deletion of primary data. [2025-10-20T19:07:17.869249] [DEBUG] [LOCAL ] HAYW: Deleted primary data file. [2025-10-20T19:07:17.886094] [INFO ] [MAIN ] HAYW: Starting full primary refresh. [2025-10-20T19:07:17.947382] [DEBUG] [VALIDATOR ] HAFN: All expected trading days are present. [2025-10-20T19:07:17.968729] [DEBUG] [VALIDATOR ] HAFN: Primary data validation passed. [2025-10-20T19:07:17.981254] [INFO ] [MAIN ] HAFN: Fetched primary data is valid. [2025-10-20T19:07:18.004248] [DEBUG] [LOCAL ] HAFN: Saving primary data. [2025-10-20T19:07:18.058443] [DEBUG] [LOCAL ] HAFN: Trimmed 64787 records to 42843 (keeping 200 days of data) [2025-10-20T19:07:18.412239] [INFO ] [REMOTE ] HAYW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:18.422270] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-01. [2025-10-20T19:07:18.428289] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T19:07:18.551455] [INFO ] [REMOTE ] HAL: Full fetch complete (77897 records). [2025-10-20T19:07:18.580779] [DEBUG] [VALIDATOR ] HAL: Validating primary data. [2025-10-20T19:07:18.585533] [DEBUG] [LOCAL ] HAFN: Primary data saved (42843 records). [2025-10-20T19:07:18.585836] [INFO ] [MAIN ] HAFN: Full history saved. [2025-10-20T19:07:18.585940] [INFO ] [MAIN ] Update Progress: 990/2367 symbols completed. HAFN took 30.7s. Estimated remaining: 2h 56m 58s. Rate: 7.8 symbols/min [2025-10-20T19:07:19.013552] [INFO ] [MAIN ] HBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:19.146520] [DEBUG] [LOCAL ] HBB: Loaded 6540 existing records from primary storage. [2025-10-20T19:07:19.185724] [INFO ] [MAIN ] HBB: Loaded 6540 existing records. [2025-10-20T19:07:19.239631] [DEBUG] [VALIDATOR ] HAL: All expected trading days are present. [2025-10-20T19:07:19.244989] [DEBUG] [VALIDATOR ] HAL: Primary data validation passed. [2025-10-20T19:07:19.273395] [INFO ] [MAIN ] HAL: Fetched primary data is valid. [2025-10-20T19:07:19.305173] [DEBUG] [LOCAL ] HAL: Saving primary data. [2025-10-20T19:07:19.366125] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T19:07:19.371849] [DEBUG] [LOCAL ] HAL: Trimmed 77897 records to 53718 (keeping 200 days of data) [2025-10-20T19:07:19.435428] [WARN ] [MAIN ] HBB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:19.435885] [INFO ] [MAIN ] HBB: Wiping existing primary data before full refresh. [2025-10-20T19:07:19.435969] [INFO ] [LOCAL ] HBB: Starting deletion of primary data. [2025-10-20T19:07:19.431516] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-02. [2025-10-20T19:07:19.437373] [DEBUG] [LOCAL ] HBB: Deleted primary data file. [2025-10-20T19:07:19.489577] [INFO ] [MAIN ] HBB: Starting full primary refresh. [2025-10-20T19:07:20.057148] [INFO ] [REMOTE ] HBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:20.061404] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-01. [2025-10-20T19:07:20.161025] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T19:07:20.191821] [DEBUG] [LOCAL ] HAL: Primary data saved (53718 records). [2025-10-20T19:07:20.191934] [INFO ] [MAIN ] HAL: Full history saved. [2025-10-20T19:07:20.192004] [INFO ] [MAIN ] Update Progress: 991/2367 symbols completed. HAL took 28.8s. Estimated remaining: 2h 56m 42s. Rate: 7.8 symbols/min [2025-10-20T19:07:20.261133] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-02. [2025-10-20T19:07:20.363939] [INFO ] [MAIN ] HBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:20.412590] [DEBUG] [LOCAL ] HBI: Loaded 53738 existing records from primary storage. [2025-10-20T19:07:20.412942] [INFO ] [MAIN ] HBI: Loaded 53738 existing records. [2025-10-20T19:07:20.587283] [WARN ] [MAIN ] HBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:20.587501] [INFO ] [MAIN ] HBI: Wiping existing primary data before full refresh. [2025-10-20T19:07:20.587547] [INFO ] [LOCAL ] HBI: Starting deletion of primary data. [2025-10-20T19:07:20.587758] [DEBUG] [LOCAL ] HBI: Deleted primary data file. [2025-10-20T19:07:20.587813] [INFO ] [MAIN ] HBI: Starting full primary refresh. [2025-10-20T19:07:20.910136] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T19:07:20.976534] [INFO ] [REMOTE ] HBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:21.051372] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-01. [2025-10-20T19:07:21.120439] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T19:07:21.800109] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T19:07:21.851373] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-03. [2025-10-20T19:07:22.651429] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-02. [2025-10-20T19:07:22.786377] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T19:07:22.970445] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:07:23.077950] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-10-20T19:07:23.451400] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-04. [2025-10-20T19:07:24.251440] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-03. [2025-10-20T19:07:25.051857] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-03. [2025-10-20T19:07:25.051752] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-10-20T19:07:25.056110] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T19:07:25.851431] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-02. [2025-10-20T19:07:26.651437] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-04. [2025-10-20T19:07:26.745810] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:07:27.451995] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-05. [2025-10-20T19:07:27.453513] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T19:07:27.498485] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:07:28.251420] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-04. [2025-10-20T19:07:28.979405] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:07:29.051335] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-06. [2025-10-20T19:07:29.851367] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-05. [2025-10-20T19:07:29.910435] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T19:07:30.650385] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:07:30.659160] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-03. [2025-10-20T19:07:30.691424] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T19:07:31.451373] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-05. [2025-10-20T19:07:32.251371] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-06. [2025-10-20T19:07:32.394206] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T19:07:32.582817] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-10-20T19:07:32.982116] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:07:33.051364] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-06. [2025-10-20T19:07:33.851394] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-04. [2025-10-20T19:07:34.651391] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-07. [2025-10-20T19:07:34.715914] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:07:35.282387] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-10-20T19:07:35.451419] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-08. [2025-10-20T19:07:35.525765] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T19:07:36.175481] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-10-20T19:07:36.251346] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-05. [2025-10-20T19:07:37.051371] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-09. [2025-10-20T19:07:37.613765] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T19:07:37.851343] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-07. [2025-10-20T19:07:37.942603] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:07:38.651367] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-06. [2025-10-20T19:07:39.451436] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-07. [2025-10-20T19:07:39.549332] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:07:40.251350] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-10. [2025-10-20T19:07:40.353765] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:07:40.746203] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T19:07:40.747383] [INFO ] [REMOTE ] HBB: Full fetch complete (9281 records). [2025-10-20T19:07:40.747451] [DEBUG] [VALIDATOR ] HBB: Validating primary data. [2025-10-20T19:07:41.051556] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-08. [2025-10-20T19:07:41.130487] [DEBUG] [VALIDATOR ] HBB: All expected trading days are present. [2025-10-20T19:07:41.130635] [DEBUG] [VALIDATOR ] HBB: Primary data validation passed. [2025-10-20T19:07:41.131033] [INFO ] [MAIN ] HBB: Fetched primary data is valid. [2025-10-20T19:07:41.131117] [DEBUG] [LOCAL ] HBB: Saving primary data. [2025-10-20T19:07:41.131673] [DEBUG] [LOCAL ] HBB: Trimmed 9281 records to 6459 (keeping 200 days of data) [2025-10-20T19:07:41.177028] [DEBUG] [LOCAL ] HBB: Primary data saved (6459 records). [2025-10-20T19:07:41.191382] [INFO ] [MAIN ] HBB: Full history saved. [2025-10-20T19:07:41.202955] [INFO ] [MAIN ] Update Progress: 992/2367 symbols completed. HBB took 22.6s. Estimated remaining: 2h 56m 53s. Rate: 7.8 symbols/min [2025-10-20T19:07:41.274323] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T19:07:41.391836] [INFO ] [MAIN ] HBM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:41.439754] [DEBUG] [LOCAL ] HBM: Loaded 53738 existing records from primary storage. [2025-10-20T19:07:41.440122] [INFO ] [MAIN ] HBM: Loaded 53738 existing records. [2025-10-20T19:07:41.614536] [WARN ] [MAIN ] HBM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:41.614775] [INFO ] [MAIN ] HBM: Wiping existing primary data before full refresh. [2025-10-20T19:07:41.614825] [INFO ] [LOCAL ] HBM: Starting deletion of primary data. [2025-10-20T19:07:41.615121] [DEBUG] [LOCAL ] HBM: Deleted primary data file. [2025-10-20T19:07:41.615177] [INFO ] [MAIN ] HBM: Starting full primary refresh. [2025-10-20T19:07:41.856506] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-08. [2025-10-20T19:07:41.966606] [INFO ] [REMOTE ] HBM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:42.651377] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-07. [2025-10-20T19:07:42.766968] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:07:43.451429] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-01. [2025-10-20T19:07:43.662622] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:07:44.251346] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-09. [2025-10-20T19:07:44.335039] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T19:07:44.831505] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T19:07:45.051378] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-02. [2025-10-20T19:07:45.565605] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T19:07:45.851384] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-10. [2025-10-20T19:07:46.575534] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T19:07:46.651361] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-03. [2025-10-20T19:07:47.451395] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-08. [2025-10-20T19:07:47.877439] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-10 (2.03 seconds). [2025-10-20T19:07:47.884901] [INFO ] [REMOTE ] HAYW: Full fetch complete (70924 records). [2025-10-20T19:07:47.885036] [DEBUG] [VALIDATOR ] HAYW: Validating primary data. [2025-10-20T19:07:48.263823] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-09. [2025-10-20T19:07:48.289610] [DEBUG] [VALIDATOR ] HAYW: All expected trading days are present. [2025-10-20T19:07:48.289746] [DEBUG] [VALIDATOR ] HAYW: Primary data validation passed. [2025-10-20T19:07:48.290995] [INFO ] [MAIN ] HAYW: Fetched primary data is valid. [2025-10-20T19:07:48.291121] [DEBUG] [LOCAL ] HAYW: Saving primary data. [2025-10-20T19:07:48.292757] [DEBUG] [LOCAL ] HAYW: Trimmed 70924 records to 48833 (keeping 200 days of data) [2025-10-20T19:07:48.538809] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T19:07:48.627821] [DEBUG] [LOCAL ] HAYW: Primary data saved (48833 records). [2025-10-20T19:07:48.627965] [INFO ] [MAIN ] HAYW: Full history saved. [2025-10-20T19:07:48.628048] [INFO ] [MAIN ] Update Progress: 993/2367 symbols completed. HAYW took 32.0s. Estimated remaining: 2h 56m 45s. Rate: 7.8 symbols/min [2025-10-20T19:07:48.799225] [INFO ] [MAIN ] HCA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:48.842967] [DEBUG] [LOCAL ] HCA: Loaded 47654 existing records from primary storage. [2025-10-20T19:07:48.843289] [INFO ] [MAIN ] HCA: Loaded 47654 existing records. [2025-10-20T19:07:49.027856] [WARN ] [MAIN ] HCA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:49.028131] [INFO ] [MAIN ] HCA: Wiping existing primary data before full refresh. [2025-10-20T19:07:49.028277] [INFO ] [LOCAL ] HCA: Starting deletion of primary data. [2025-10-20T19:07:49.028560] [DEBUG] [LOCAL ] HCA: Deleted primary data file. [2025-10-20T19:07:49.028711] [INFO ] [MAIN ] HCA: Starting full primary refresh. [2025-10-20T19:07:49.054248] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-04. [2025-10-20T19:07:49.336760] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T19:07:49.456114] [INFO ] [REMOTE ] HCA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:49.785344] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:07:49.851334] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-10. [2025-10-20T19:07:50.651359] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-09. [2025-10-20T19:07:50.908757] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:07:51.451439] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-01. [2025-10-20T19:07:51.906508] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T19:07:51.913823] [INFO ] [REMOTE ] HASI: Full fetch complete (68394 records). [2025-10-20T19:07:51.913936] [DEBUG] [VALIDATOR ] HASI: Validating primary data. [2025-10-20T19:07:52.258841] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-05. [2025-10-20T19:07:52.315733] [DEBUG] [VALIDATOR ] HASI: All expected trading days are present. [2025-10-20T19:07:52.315826] [DEBUG] [VALIDATOR ] HASI: Primary data validation passed. [2025-10-20T19:07:52.316792] [INFO ] [MAIN ] HASI: Fetched primary data is valid. [2025-10-20T19:07:52.316857] [DEBUG] [LOCAL ] HASI: Saving primary data. [2025-10-20T19:07:52.318239] [DEBUG] [LOCAL ] HASI: Trimmed 68394 records to 47810 (keeping 200 days of data) [2025-10-20T19:07:52.662291] [DEBUG] [LOCAL ] HASI: Primary data saved (47810 records). [2025-10-20T19:07:52.664823] [INFO ] [MAIN ] HASI: Full history saved. [2025-10-20T19:07:52.664989] [INFO ] [MAIN ] Update Progress: 994/2367 symbols completed. HASI took 37.2s. Estimated remaining: 2h 56m 32s. Rate: 7.8 symbols/min [2025-10-20T19:07:52.664549] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T19:07:52.851893] [INFO ] [MAIN ] HCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:52.884843] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T19:07:52.932366] [DEBUG] [LOCAL ] HCC: Loaded 45791 existing records from primary storage. [2025-10-20T19:07:52.932684] [INFO ] [MAIN ] HCC: Loaded 45791 existing records. [2025-10-20T19:07:53.056525] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-10. [2025-10-20T19:07:53.134304] [WARN ] [MAIN ] HCC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:53.134497] [INFO ] [MAIN ] HCC: Wiping existing primary data before full refresh. [2025-10-20T19:07:53.134545] [INFO ] [LOCAL ] HCC: Starting deletion of primary data. [2025-10-20T19:07:53.134766] [DEBUG] [LOCAL ] HCC: Deleted primary data file. [2025-10-20T19:07:53.134822] [INFO ] [MAIN ] HCC: Starting full primary refresh. [2025-10-20T19:07:53.491217] [INFO ] [REMOTE ] HCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:53.851418] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-02. [2025-10-20T19:07:54.141177] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T19:07:54.651344] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-06. [2025-10-20T19:07:54.902208] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:07:54.911520] [INFO ] [REMOTE ] HBI: Full fetch complete (77005 records). [2025-10-20T19:07:54.911717] [DEBUG] [VALIDATOR ] HBI: Validating primary data. [2025-10-20T19:07:55.314435] [DEBUG] [VALIDATOR ] HBI: All expected trading days are present. [2025-10-20T19:07:55.314525] [DEBUG] [VALIDATOR ] HBI: Primary data validation passed. [2025-10-20T19:07:55.315481] [INFO ] [MAIN ] HBI: Fetched primary data is valid. [2025-10-20T19:07:55.315545] [DEBUG] [LOCAL ] HBI: Saving primary data. [2025-10-20T19:07:55.317205] [DEBUG] [LOCAL ] HBI: Trimmed 77005 records to 52948 (keeping 200 days of data) [2025-10-20T19:07:55.477989] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-01. [2025-10-20T19:07:55.643180] [DEBUG] [LOCAL ] HBI: Primary data saved (52948 records). [2025-10-20T19:07:55.659800] [INFO ] [MAIN ] HBI: Full history saved. [2025-10-20T19:07:55.659962] [INFO ] [MAIN ] Update Progress: 995/2367 symbols completed. HBI took 35.5s. Estimated remaining: 2h 56m 18s. Rate: 7.8 symbols/min [2025-10-20T19:07:55.749724] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T19:07:55.867454] [INFO ] [MAIN ] HCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:55.886552] [DEBUG] [LOCAL ] HCI: Loaded 14354 existing records from primary storage. [2025-10-20T19:07:55.886856] [INFO ] [MAIN ] HCI: Loaded 14354 existing records. [2025-10-20T19:07:56.092888] [WARN ] [MAIN ] HCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:56.093099] [INFO ] [MAIN ] HCI: Wiping existing primary data before full refresh. [2025-10-20T19:07:56.093172] [INFO ] [LOCAL ] HCI: Starting deletion of primary data. [2025-10-20T19:07:56.093398] [DEBUG] [LOCAL ] HCI: Deleted primary data file. [2025-10-20T19:07:56.093455] [INFO ] [MAIN ] HCI: Starting full primary refresh. [2025-10-20T19:07:56.256494] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-03. [2025-10-20T19:07:56.497150] [INFO ] [REMOTE ] HCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:56.528099] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T19:07:57.039474] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:07:57.051341] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-07. [2025-10-20T19:07:57.851378] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-02. [2025-10-20T19:07:58.272357] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T19:07:58.651375] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-04. [2025-10-20T19:07:58.906934] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T19:07:59.121253] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:07:59.451378] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-08. [2025-10-20T19:08:00.251396] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-03. [2025-10-20T19:08:00.425445] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:08:01.051371] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-05. [2025-10-20T19:08:01.263069] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:08:01.338886] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T19:08:01.851357] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-01. [2025-10-20T19:08:02.308113] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T19:08:02.355300] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T19:08:02.651354] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-02. [2025-10-20T19:08:03.451404] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-04. [2025-10-20T19:08:03.485932] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T19:08:04.251441] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-09. [2025-10-20T19:08:05.051375] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-03. [2025-10-20T19:08:05.170027] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T19:08:05.851391] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-05. [2025-10-20T19:08:05.981016] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T19:08:06.223366] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:08:06.651445] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-06. [2025-10-20T19:08:07.052595] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T19:08:07.451442] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-06. [2025-10-20T19:08:08.251389] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-10. [2025-10-20T19:08:08.303898] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:08:08.912984] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:08:09.051365] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-07. [2025-10-20T19:08:09.851405] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-04. [2025-10-20T19:08:10.240411] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T19:08:10.248986] [INFO ] [REMOTE ] HBM: Full fetch complete (76730 records). [2025-10-20T19:08:10.249166] [DEBUG] [VALIDATOR ] HBM: Validating primary data. [2025-10-20T19:08:10.694464] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-07. [2025-10-20T19:08:10.703811] [DEBUG] [VALIDATOR ] HBM: All expected trading days are present. [2025-10-20T19:08:10.705301] [DEBUG] [VALIDATOR ] HBM: Primary data validation passed. [2025-10-20T19:08:10.706800] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T19:08:10.706346] [INFO ] [MAIN ] HBM: Fetched primary data is valid. [2025-10-20T19:08:10.707151] [DEBUG] [LOCAL ] HBM: Saving primary data. [2025-10-20T19:08:10.740229] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T19:08:10.738980] [DEBUG] [LOCAL ] HBM: Trimmed 76730 records to 52976 (keeping 200 days of data) [2025-10-20T19:08:11.053418] [DEBUG] [LOCAL ] HBM: Primary data saved (52976 records). [2025-10-20T19:08:11.053545] [INFO ] [MAIN ] HBM: Full history saved. [2025-10-20T19:08:11.053633] [INFO ] [MAIN ] Update Progress: 996/2367 symbols completed. HBM took 29.9s. Estimated remaining: 2h 56m 21s. Rate: 7.8 symbols/min [2025-10-20T19:08:11.219026] [INFO ] [MAIN ] HCXY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:11.219155] [DEBUG] [LOCAL ] HCXY: No existing data file found in primary storage. [2025-10-20T19:08:11.219192] [WARN ] [MAIN ] HCXY: No existing data found, performing full refresh instead. [2025-10-20T19:08:11.219305] [INFO ] [MAIN ] HCXY: Wiping existing primary data before full refresh. [2025-10-20T19:08:11.219345] [INFO ] [LOCAL ] HCXY: Starting deletion of primary data. [2025-10-20T19:08:11.219384] [DEBUG] [LOCAL ] HCXY: No primary data file found to delete. [2025-10-20T19:08:11.219415] [INFO ] [MAIN ] HCXY: Starting full primary refresh. [2025-10-20T19:08:11.458504] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-05. [2025-10-20T19:08:11.578255] [INFO ] [REMOTE ] HCXY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:12.251399] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-08. [2025-10-20T19:08:12.316721] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T19:08:12.488730] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:08:13.051379] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-08. [2025-10-20T19:08:13.759326] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:08:13.851382] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-09. [2025-10-20T19:08:14.638248] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:08:14.651349] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-01. [2025-10-20T19:08:15.318725] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T19:08:15.350285] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T19:08:15.451376] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-06. [2025-10-20T19:08:16.251375] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-09. [2025-10-20T19:08:16.421944] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T19:08:17.051385] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-07. [2025-10-20T19:08:17.837556] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:08:17.851557] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-10. [2025-10-20T19:08:18.017929] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T19:08:18.651370] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-10. [2025-10-20T19:08:19.451443] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-02. [2025-10-20T19:08:19.646034] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T19:08:19.653319] [INFO ] [REMOTE ] HCA: Full fetch complete (68946 records). [2025-10-20T19:08:19.653436] [DEBUG] [VALIDATOR ] HCA: Validating primary data. [2025-10-20T19:08:19.728626] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-02 (0.28 seconds). [2025-10-20T19:08:20.046437] [DEBUG] [VALIDATOR ] HCA: All expected trading days are present. [2025-10-20T19:08:20.046570] [DEBUG] [VALIDATOR ] HCA: Primary data validation passed. [2025-10-20T19:08:20.047504] [INFO ] [MAIN ] HCA: Fetched primary data is valid. [2025-10-20T19:08:20.047569] [DEBUG] [LOCAL ] HCA: Saving primary data. [2025-10-20T19:08:20.048979] [DEBUG] [LOCAL ] HCA: Trimmed 68946 records to 46957 (keeping 200 days of data) [2025-10-20T19:08:20.259743] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-03. [2025-10-20T19:08:20.339404] [DEBUG] [LOCAL ] HCA: Primary data saved (46957 records). [2025-10-20T19:08:20.339725] [INFO ] [MAIN ] HCA: Full history saved. [2025-10-20T19:08:20.340624] [INFO ] [MAIN ] Update Progress: 997/2367 symbols completed. HCA took 31.7s. Estimated remaining: 2h 56m 15s. Rate: 7.8 symbols/min [2025-10-20T19:08:20.419760] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T19:08:20.426249] [INFO ] [REMOTE ] HCC: Full fetch complete (64916 records). [2025-10-20T19:08:20.426404] [DEBUG] [VALIDATOR ] HCC: Validating primary data. [2025-10-20T19:08:20.608366] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T19:08:20.842266] [DEBUG] [VALIDATOR ] HCC: All expected trading days are present. [2025-10-20T19:08:20.842347] [DEBUG] [VALIDATOR ] HCC: Primary data validation passed. [2025-10-20T19:08:20.843194] [INFO ] [MAIN ] HCC: Fetched primary data is valid. [2025-10-20T19:08:20.843258] [DEBUG] [LOCAL ] HCC: Saving primary data. [2025-10-20T19:08:20.844528] [DEBUG] [LOCAL ] HCC: Trimmed 64916 records to 45162 (keeping 200 days of data) [2025-10-20T19:08:21.063238] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-04. [2025-10-20T19:08:21.130835] [DEBUG] [LOCAL ] HCC: Primary data saved (45162 records). [2025-10-20T19:08:21.130954] [INFO ] [MAIN ] HCC: Full history saved. [2025-10-20T19:08:21.131414] [INFO ] [MAIN ] Update Progress: 998/2367 symbols completed. HCC took 28.5s. Estimated remaining: 2h 55m 58s. Rate: 7.8 symbols/min [2025-10-20T19:08:21.306282] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-04 (0.24 seconds). [2025-10-20T19:08:21.851422] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-05. [2025-10-20T19:08:22.093188] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-10-20T19:08:22.651353] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-06. [2025-10-20T19:08:23.041595] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-10-20T19:08:23.451388] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-08. [2025-10-20T19:08:24.251373] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-07. [2025-10-20T19:08:24.393056] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T19:08:24.569131] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-10-20T19:08:25.051370] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-09. [2025-10-20T19:08:25.851402] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-08. [2025-10-20T19:08:26.025808] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:08:26.166138] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-10-20T19:08:26.651421] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-09. [2025-10-20T19:08:26.883210] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-09 (0.23 seconds). [2025-10-20T19:08:27.451367] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-10. [2025-10-20T19:08:27.808177] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T19:08:27.808865] [INFO ] [REMOTE ] HCXY: Full fetch complete (510 records). [2025-10-20T19:08:27.808966] [DEBUG] [VALIDATOR ] HCXY: Validating primary data. [2025-10-20T19:08:27.809363] [FATAL] [VALIDATOR ] HCXY: Data is stale - most recent date (2025-10-16) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T19:08:27.809451] [FATAL] [MAIN ] HCXY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:08:27.809497] [WARN ] [MAIN ] HCXY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:08:27.809537] [INFO ] [LOCAL ] HCXY: Starting deletion of primary data. [2025-10-20T19:08:27.809595] [DEBUG] [LOCAL ] HCXY: No primary data file found to delete. [2025-10-20T19:08:27.809633] [INFO ] [LOCAL ] HCXY: Starting deletion of derivative data. [2025-10-20T19:08:27.809748] [DEBUG] [LOCAL ] HCXY: No derivative data file found to delete. [2025-10-20T19:08:27.809829] [INFO ] [MAIN ] Update Progress: 999/2367 symbols completed. HCXY took 16.8s. Estimated remaining: 2h 55m 49s. Rate: 7.8 symbols/min [2025-10-20T19:08:28.251376] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-10. [2025-10-20T19:08:29.288593] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T19:08:29.291281] [INFO ] [REMOTE ] HCI: Full fetch complete (19655 records). [2025-10-20T19:08:29.291360] [DEBUG] [VALIDATOR ] HCI: Validating primary data. [2025-10-20T19:08:29.663092] [DEBUG] [VALIDATOR ] HCI: All expected trading days are present. [2025-10-20T19:08:29.663200] [DEBUG] [VALIDATOR ] HCI: Primary data validation passed. [2025-10-20T19:08:29.663722] [INFO ] [MAIN ] HCI: Fetched primary data is valid. [2025-10-20T19:08:29.663788] [DEBUG] [LOCAL ] HCI: Saving primary data. [2025-10-20T19:08:29.664929] [DEBUG] [LOCAL ] HCI: Trimmed 19655 records to 14113 (keeping 200 days of data) [2025-10-20T19:08:29.757018] [DEBUG] [LOCAL ] HCI: Primary data saved (14113 records). [2025-10-20T19:08:29.757166] [INFO ] [MAIN ] HCI: Full history saved. [2025-10-20T19:08:29.757258] [INFO ] [MAIN ] Update Progress: 1000/2367 symbols completed. HCI took 34.1s. Estimated remaining: 2h 55m 33s. Rate: 7.8 symbols/min [2025-10-20T19:08:30.351524] [INFO ] [MAIN ] HDB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:30.354952] [INFO ] [MAIN ] HE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:30.482994] [INFO ] [MAIN ] HEI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:30.503078] [INFO ] [MAIN ] HD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:30.503773] [DEBUG] [LOCAL ] HEI: No existing data file found in primary storage. [2025-10-20T19:08:30.503973] [WARN ] [MAIN ] HEI: No existing data found, performing full refresh instead. [2025-10-20T19:08:30.504202] [INFO ] [MAIN ] HEI: Wiping existing primary data before full refresh. [2025-10-20T19:08:30.504270] [INFO ] [LOCAL ] HEI: Starting deletion of primary data. [2025-10-20T19:08:30.504431] [DEBUG] [LOCAL ] HEI: No primary data file found to delete. [2025-10-20T19:08:30.504497] [INFO ] [MAIN ] HEI: Starting full primary refresh. [2025-10-20T19:08:30.869924] [DEBUG] [LOCAL ] HE: Loaded 49612 existing records from primary storage. [2025-10-20T19:08:30.930251] [DEBUG] [LOCAL ] HDB: Loaded 52710 existing records from primary storage. [2025-10-20T19:08:30.953211] [DEBUG] [LOCAL ] HD: Loaded 54264 existing records from primary storage. [2025-10-20T19:08:30.930381] [INFO ] [MAIN ] HE: Loaded 49612 existing records. [2025-10-20T19:08:30.978342] [INFO ] [MAIN ] HD: Loaded 54264 existing records. [2025-10-20T19:08:30.986702] [INFO ] [REMOTE ] HEI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:30.996351] [INFO ] [MAIN ] HDB: Loaded 52710 existing records. [2025-10-20T19:08:31.077361] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-01. [2025-10-20T19:08:31.442034] [WARN ] [MAIN ] HE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:31.486471] [INFO ] [MAIN ] HE: Wiping existing primary data before full refresh. [2025-10-20T19:08:31.537107] [INFO ] [LOCAL ] HE: Starting deletion of primary data. [2025-10-20T19:08:31.519469] [WARN ] [MAIN ] HD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:31.541149] [WARN ] [MAIN ] HDB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:31.541806] [INFO ] [MAIN ] HD: Wiping existing primary data before full refresh. [2025-10-20T19:08:31.541972] [INFO ] [LOCAL ] HD: Starting deletion of primary data. [2025-10-20T19:08:31.542063] [DEBUG] [LOCAL ] HE: Deleted primary data file. [2025-10-20T19:08:31.542326] [INFO ] [MAIN ] HE: Starting full primary refresh. [2025-10-20T19:08:31.542392] [INFO ] [MAIN ] HDB: Wiping existing primary data before full refresh. [2025-10-20T19:08:31.542635] [INFO ] [LOCAL ] HDB: Starting deletion of primary data. [2025-10-20T19:08:31.542590] [DEBUG] [LOCAL ] HD: Deleted primary data file. [2025-10-20T19:08:31.542897] [INFO ] [MAIN ] HD: Starting full primary refresh. [2025-10-20T19:08:31.548192] [DEBUG] [LOCAL ] HDB: Deleted primary data file. [2025-10-20T19:08:31.548291] [INFO ] [MAIN ] HDB: Starting full primary refresh. [2025-10-20T19:08:32.520864] [INFO ] [REMOTE ] HE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:32.594181] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-01. [2025-10-20T19:08:32.597051] [INFO ] [REMOTE ] HDB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:32.598044] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-01. [2025-10-20T19:08:32.660853] [INFO ] [REMOTE ] HD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:32.661308] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-01. [2025-10-20T19:08:32.704383] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:08:32.712016] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-02. [2025-10-20T19:08:34.224783] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T19:08:34.232261] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-03. [2025-10-20T19:08:34.468013] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T19:08:34.478750] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-02. [2025-10-20T19:08:34.511327] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T19:08:34.550143] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T19:08:34.651363] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-02. [2025-10-20T19:08:35.451421] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-02. [2025-10-20T19:08:35.793248] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:08:35.881510] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:08:36.173936] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T19:08:36.251348] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-03. [2025-10-20T19:08:37.051374] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-03. [2025-10-20T19:08:37.278634] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T19:08:37.858683] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-04. [2025-10-20T19:08:37.879406] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:08:38.507803] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:08:38.651380] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-04. [2025-10-20T19:08:39.456491] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-04. [2025-10-20T19:08:39.470778] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:08:40.251400] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-05. [2025-10-20T19:08:40.399793] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:08:40.666943] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T19:08:41.051411] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-05. [2025-10-20T19:08:41.738608] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T19:08:41.851353] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-03. [2025-10-20T19:08:42.651433] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-06. [2025-10-20T19:08:42.761462] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T19:08:43.451355] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-05. [2025-10-20T19:08:43.729677] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T19:08:43.758074] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T19:08:44.251366] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-04. [2025-10-20T19:08:44.913022] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T19:08:45.051381] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-06. [2025-10-20T19:08:45.851464] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-06. [2025-10-20T19:08:46.450520] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-10-20T19:08:46.508408] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:08:46.651402] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-07. [2025-10-20T19:08:47.388805] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:08:47.451402] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-07. [2025-10-20T19:08:47.957167] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:08:48.251437] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-05. [2025-10-20T19:08:48.826246] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T19:08:49.051387] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-08. [2025-10-20T19:08:49.851373] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-07. [2025-10-20T19:08:50.435334] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-05 (2.18 seconds). [2025-10-20T19:08:50.651340] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-06. [2025-10-20T19:08:50.730549] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T19:08:51.275025] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:08:51.451441] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-08. [2025-10-20T19:08:52.251427] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-09. [2025-10-20T19:08:52.438465] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T19:08:52.885847] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T19:08:53.051386] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-09. [2025-10-20T19:08:53.489135] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T19:08:53.846801] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-10-20T19:08:53.852928] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-08. [2025-10-20T19:08:54.651439] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-10. [2025-10-20T19:08:55.406055] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T19:08:55.451347] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-09. [2025-10-20T19:08:55.577334] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T19:08:55.581075] [INFO ] [REMOTE ] HEI: Full fetch complete (47512 records). [2025-10-20T19:08:55.581164] [DEBUG] [VALIDATOR ] HEI: Validating primary data. [2025-10-20T19:08:55.952957] [DEBUG] [VALIDATOR ] HEI: All expected trading days are present. [2025-10-20T19:08:55.953059] [DEBUG] [VALIDATOR ] HEI: Primary data validation passed. [2025-10-20T19:08:55.953786] [INFO ] [MAIN ] HEI: Fetched primary data is valid. [2025-10-20T19:08:55.953848] [DEBUG] [LOCAL ] HEI: Saving primary data. [2025-10-20T19:08:55.954888] [DEBUG] [LOCAL ] HEI: Trimmed 47512 records to 30890 (keeping 200 days of data) [2025-10-20T19:08:56.141585] [DEBUG] [LOCAL ] HEI: Primary data saved (30890 records). [2025-10-20T19:08:56.141742] [INFO ] [MAIN ] HEI: Full history saved. [2025-10-20T19:08:56.141821] [INFO ] [MAIN ] Update Progress: 1001/2367 symbols completed. HEI took 26.4s. Estimated remaining: 2h 55m 51s. Rate: 7.8 symbols/min [2025-10-20T19:08:56.256567] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-10. [2025-10-20T19:08:56.308550] [INFO ] [MAIN ] HEQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:56.318148] [DEBUG] [LOCAL ] HEQ: Loaded 6898 existing records from primary storage. [2025-10-20T19:08:56.318374] [INFO ] [MAIN ] HEQ: Loaded 6898 existing records. [2025-10-20T19:08:56.492477] [WARN ] [MAIN ] HEQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:56.492696] [INFO ] [MAIN ] HEQ: Wiping existing primary data before full refresh. [2025-10-20T19:08:56.492753] [INFO ] [LOCAL ] HEQ: Starting deletion of primary data. [2025-10-20T19:08:56.492901] [DEBUG] [LOCAL ] HEQ: Deleted primary data file. [2025-10-20T19:08:56.492937] [INFO ] [MAIN ] HEQ: Starting full primary refresh. [2025-10-20T19:08:56.838292] [INFO ] [REMOTE ] HEQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:57.051359] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-07. [2025-10-20T19:08:57.160768] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:08:57.851471] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-10. [2025-10-20T19:08:58.040239] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T19:08:58.047437] [INFO ] [REMOTE ] HE: Full fetch complete (72162 records). [2025-10-20T19:08:58.047580] [DEBUG] [VALIDATOR ] HE: Validating primary data. [2025-10-20T19:08:58.451621] [DEBUG] [VALIDATOR ] HE: All expected trading days are present. [2025-10-20T19:08:58.451832] [DEBUG] [VALIDATOR ] HE: Primary data validation passed. [2025-10-20T19:08:58.452795] [INFO ] [MAIN ] HE: Fetched primary data is valid. [2025-10-20T19:08:58.452858] [DEBUG] [LOCAL ] HE: Saving primary data. [2025-10-20T19:08:58.454250] [DEBUG] [LOCAL ] HE: Trimmed 72162 records to 48865 (keeping 200 days of data) [2025-10-20T19:08:58.652931] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-01. [2025-10-20T19:08:58.739847] [DEBUG] [LOCAL ] HE: Primary data saved (48865 records). [2025-10-20T19:08:58.739968] [INFO ] [MAIN ] HE: Full history saved. [2025-10-20T19:08:58.740045] [INFO ] [MAIN ] Update Progress: 1002/2367 symbols completed. HE took 29.0s. Estimated remaining: 2h 55m 36s. Rate: 7.8 symbols/min [2025-10-20T19:08:58.916080] [INFO ] [MAIN ] HESM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:58.959751] [DEBUG] [LOCAL ] HESM: Loaded 49235 existing records from primary storage. [2025-10-20T19:08:58.960201] [INFO ] [MAIN ] HESM: Loaded 49235 existing records. [2025-10-20T19:08:59.146589] [WARN ] [MAIN ] HESM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:59.147070] [INFO ] [MAIN ] HESM: Wiping existing primary data before full refresh. [2025-10-20T19:08:59.147214] [INFO ] [LOCAL ] HESM: Starting deletion of primary data. [2025-10-20T19:08:59.147748] [DEBUG] [LOCAL ] HESM: Deleted primary data file. [2025-10-20T19:08:59.147805] [INFO ] [MAIN ] HESM: Starting full primary refresh. [2025-10-20T19:08:59.257311] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T19:08:59.456962] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-02. [2025-10-20T19:08:59.562093] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-07 (2.51 seconds). [2025-10-20T19:08:59.617723] [INFO ] [REMOTE ] HESM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:59.702965] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:08:59.712576] [INFO ] [REMOTE ] HDB: Full fetch complete (75662 records). [2025-10-20T19:08:59.712839] [DEBUG] [VALIDATOR ] HDB: Validating primary data. [2025-10-20T19:09:00.232517] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:09:00.257027] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-01. [2025-10-20T19:09:00.261482] [DEBUG] [VALIDATOR ] HDB: All expected trading days are present. [2025-10-20T19:09:00.262018] [DEBUG] [VALIDATOR ] HDB: Primary data validation passed. [2025-10-20T19:09:00.267620] [INFO ] [MAIN ] HDB: Fetched primary data is valid. [2025-10-20T19:09:00.267895] [DEBUG] [LOCAL ] HDB: Saving primary data. [2025-10-20T19:09:00.269684] [DEBUG] [LOCAL ] HDB: Trimmed 75662 records to 51936 (keeping 200 days of data) [2025-10-20T19:09:00.625474] [DEBUG] [LOCAL ] HDB: Primary data saved (51936 records). [2025-10-20T19:09:00.625631] [INFO ] [MAIN ] HDB: Full history saved. [2025-10-20T19:09:00.626193] [INFO ] [MAIN ] Update Progress: 1003/2367 symbols completed. HDB took 30.9s. Estimated remaining: 2h 55m 21s. Rate: 7.8 symbols/min [2025-10-20T19:09:00.824235] [INFO ] [MAIN ] HF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:00.824448] [DEBUG] [LOCAL ] HF: No existing data file found in primary storage. [2025-10-20T19:09:00.824493] [WARN ] [MAIN ] HF: No existing data found, performing full refresh instead. [2025-10-20T19:09:00.824601] [INFO ] [MAIN ] HF: Wiping existing primary data before full refresh. [2025-10-20T19:09:00.824664] [INFO ] [LOCAL ] HF: Starting deletion of primary data. [2025-10-20T19:09:00.824759] [DEBUG] [LOCAL ] HF: No primary data file found to delete. [2025-10-20T19:09:00.824797] [INFO ] [MAIN ] HF: Starting full primary refresh. [2025-10-20T19:09:01.056592] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-03. [2025-10-20T19:09:01.247881] [INFO ] [REMOTE ] HF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:01.816795] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T19:09:01.854506] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-08. [2025-10-20T19:09:01.859209] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:09:02.651437] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-04. [2025-10-20T19:09:03.368157] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T19:09:03.451341] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-05. [2025-10-20T19:09:03.941869] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-10-20T19:09:04.111119] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T19:09:04.251385] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-06. [2025-10-20T19:09:04.833993] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T19:09:05.051435] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-09. [2025-10-20T19:09:05.851413] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-02. [2025-10-20T19:09:06.651377] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-07. [2025-10-20T19:09:07.381851] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-09 (2.33 seconds). [2025-10-20T19:09:07.411075] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T19:09:07.430293] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T19:09:07.451330] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-08. [2025-10-20T19:09:08.100060] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T19:09:08.251360] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-09. [2025-10-20T19:09:08.960672] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T19:09:09.051403] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-01. [2025-10-20T19:09:09.387659] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-10-20T19:09:09.851368] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-10. [2025-10-20T19:09:10.651455] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-03. [2025-10-20T19:09:10.665167] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:09:10.667147] [INFO ] [REMOTE ] HEQ: Full fetch complete (9527 records). [2025-10-20T19:09:10.667230] [DEBUG] [VALIDATOR ] HEQ: Validating primary data. [2025-10-20T19:09:11.020046] [DEBUG] [VALIDATOR ] HEQ: All expected trading days are present. [2025-10-20T19:09:11.020123] [DEBUG] [VALIDATOR ] HEQ: Primary data validation passed. [2025-10-20T19:09:11.020345] [INFO ] [MAIN ] HEQ: Fetched primary data is valid. [2025-10-20T19:09:11.020382] [DEBUG] [LOCAL ] HEQ: Saving primary data. [2025-10-20T19:09:11.020819] [DEBUG] [LOCAL ] HEQ: Trimmed 9527 records to 6822 (keeping 200 days of data) [2025-10-20T19:09:11.060539] [DEBUG] [LOCAL ] HEQ: Primary data saved (6822 records). [2025-10-20T19:09:11.060675] [INFO ] [MAIN ] HEQ: Full history saved. [2025-10-20T19:09:11.060770] [INFO ] [MAIN ] Update Progress: 1004/2367 symbols completed. HEQ took 14.9s. Estimated remaining: 2h 55m 17s. Rate: 7.8 symbols/min [2025-10-20T19:09:11.227025] [INFO ] [MAIN ] HFND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:11.227211] [DEBUG] [LOCAL ] HFND: No existing data file found in primary storage. [2025-10-20T19:09:11.227247] [WARN ] [MAIN ] HFND: No existing data found, performing full refresh instead. [2025-10-20T19:09:11.227356] [INFO ] [MAIN ] HFND: Wiping existing primary data before full refresh. [2025-10-20T19:09:11.227398] [INFO ] [LOCAL ] HFND: Starting deletion of primary data. [2025-10-20T19:09:11.227437] [DEBUG] [LOCAL ] HFND: No primary data file found to delete. [2025-10-20T19:09:11.227468] [INFO ] [MAIN ] HFND: Starting full primary refresh. [2025-10-20T19:09:11.463744] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-02. [2025-10-20T19:09:11.578104] [INFO ] [REMOTE ] HFND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:12.132736] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T19:09:12.175596] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T19:09:12.251381] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-01. [2025-10-20T19:09:12.600205] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-01 (0.35 seconds). [2025-10-20T19:09:13.051388] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-03. [2025-10-20T19:09:13.298202] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-03 (0.25 seconds). [2025-10-20T19:09:13.851384] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-02. [2025-10-20T19:09:14.265779] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T19:09:14.651383] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-04. [2025-10-20T19:09:14.901308] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-04 (0.25 seconds). [2025-10-20T19:09:15.451413] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-05. [2025-10-20T19:09:15.694274] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-10-20T19:09:16.251396] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-10. [2025-10-20T19:09:17.051396] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-04. [2025-10-20T19:09:17.186518] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:09:17.195023] [INFO ] [REMOTE ] HD: Full fetch complete (77564 records). [2025-10-20T19:09:17.195183] [DEBUG] [VALIDATOR ] HD: Validating primary data. [2025-10-20T19:09:17.605132] [DEBUG] [VALIDATOR ] HD: All expected trading days are present. [2025-10-20T19:09:17.605229] [DEBUG] [VALIDATOR ] HD: Primary data validation passed. [2025-10-20T19:09:17.606411] [INFO ] [MAIN ] HD: Fetched primary data is valid. [2025-10-20T19:09:17.606500] [DEBUG] [LOCAL ] HD: Saving primary data. [2025-10-20T19:09:17.608178] [DEBUG] [LOCAL ] HD: Trimmed 77564 records to 53482 (keeping 200 days of data) [2025-10-20T19:09:17.856868] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-03. [2025-10-20T19:09:17.963476] [DEBUG] [LOCAL ] HD: Primary data saved (53482 records). [2025-10-20T19:09:17.963619] [INFO ] [MAIN ] HD: Full history saved. [2025-10-20T19:09:17.963732] [INFO ] [MAIN ] Update Progress: 1005/2367 symbols completed. HD took 48.2s. Estimated remaining: 2h 55m 8s. Rate: 7.8 symbols/min [2025-10-20T19:09:18.161166] [INFO ] [MAIN ] HFRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:18.181813] [DEBUG] [LOCAL ] HFRO: Loaded 17015 existing records from primary storage. [2025-10-20T19:09:18.182093] [INFO ] [MAIN ] HFRO: Loaded 17015 existing records. [2025-10-20T19:09:18.352613] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T19:09:18.396574] [WARN ] [MAIN ] HFRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:18.397029] [INFO ] [MAIN ] HFRO: Wiping existing primary data before full refresh. [2025-10-20T19:09:18.397173] [INFO ] [LOCAL ] HFRO: Starting deletion of primary data. [2025-10-20T19:09:18.397426] [DEBUG] [LOCAL ] HFRO: Deleted primary data file. [2025-10-20T19:09:18.397604] [INFO ] [MAIN ] HFRO: Starting full primary refresh. [2025-10-20T19:09:18.662011] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-06. [2025-10-20T19:09:18.664005] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:09:18.815088] [INFO ] [REMOTE ] HFRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:18.971515] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-10-20T19:09:19.451377] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-01. [2025-10-20T19:09:20.251392] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-04. [2025-10-20T19:09:20.614770] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:09:20.666100] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T19:09:21.051384] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-05. [2025-10-20T19:09:21.402306] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T19:09:21.851379] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-05. [2025-10-20T19:09:22.651396] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-06. [2025-10-20T19:09:22.979667] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-10-20T19:09:23.283102] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T19:09:23.451358] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-06. [2025-10-20T19:09:24.251382] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-07. [2025-10-20T19:09:24.557612] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-10-20T19:09:24.987867] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:09:25.051363] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-07. [2025-10-20T19:09:25.851366] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-07. [2025-10-20T19:09:26.270391] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-10-20T19:09:26.621320] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:09:26.651333] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-08. [2025-10-20T19:09:27.451366] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-08. [2025-10-20T19:09:27.706759] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-08 (0.26 seconds). [2025-10-20T19:09:28.151189] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:09:28.251338] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-09. [2025-10-20T19:09:29.051370] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-02. [2025-10-20T19:09:29.858270] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-09. [2025-10-20T19:09:29.863547] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T19:09:30.008403] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T19:09:30.136141] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-10-20T19:09:30.651346] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-10. [2025-10-20T19:09:30.995702] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-10 (0.34 seconds). [2025-10-20T19:09:30.996547] [INFO ] [REMOTE ] HF: Full fetch complete (434 records). [2025-10-20T19:09:30.996626] [DEBUG] [VALIDATOR ] HF: Validating primary data. [2025-10-20T19:09:31.187338] [WARN ] [VALIDATOR ] HF: Validation failed - insufficient historical data. Have 143 unique days, need 200 days [2025-10-20T19:09:31.187531] [FATAL] [MAIN ] HF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:09:31.187601] [WARN ] [MAIN ] HF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:09:31.187696] [INFO ] [LOCAL ] HF: Starting deletion of primary data. [2025-10-20T19:09:31.187764] [DEBUG] [LOCAL ] HF: No primary data file found to delete. [2025-10-20T19:09:31.187801] [INFO ] [LOCAL ] HF: Starting deletion of derivative data. [2025-10-20T19:09:31.187841] [DEBUG] [LOCAL ] HF: No derivative data file found to delete. [2025-10-20T19:09:31.187896] [INFO ] [MAIN ] Update Progress: 1006/2367 symbols completed. HF took 30.6s. Estimated remaining: 2h 55m 8s. Rate: 7.8 symbols/min [2025-10-20T19:09:31.363783] [INFO ] [MAIN ] HGER: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:31.381378] [DEBUG] [LOCAL ] HGER: Loaded 15286 existing records from primary storage. [2025-10-20T19:09:31.381603] [INFO ] [MAIN ] HGER: Loaded 15286 existing records. [2025-10-20T19:09:31.456560] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-10. [2025-10-20T19:09:31.565266] [WARN ] [MAIN ] HGER: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:31.565480] [INFO ] [MAIN ] HGER: Wiping existing primary data before full refresh. [2025-10-20T19:09:31.565526] [INFO ] [LOCAL ] HGER: Starting deletion of primary data. [2025-10-20T19:09:31.565711] [DEBUG] [LOCAL ] HGER: Deleted primary data file. [2025-10-20T19:09:31.565766] [INFO ] [MAIN ] HGER: Starting full primary refresh. [2025-10-20T19:09:31.922247] [INFO ] [REMOTE ] HGER: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:32.251365] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-08. [2025-10-20T19:09:32.673577] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T19:09:33.051413] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-03. [2025-10-20T19:09:33.410580] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T19:09:33.417861] [INFO ] [REMOTE ] HESM: Full fetch complete (69699 records). [2025-10-20T19:09:33.417980] [DEBUG] [VALIDATOR ] HESM: Validating primary data. [2025-10-20T19:09:33.829077] [DEBUG] [VALIDATOR ] HESM: All expected trading days are present. [2025-10-20T19:09:33.829194] [DEBUG] [VALIDATOR ] HESM: Primary data validation passed. [2025-10-20T19:09:33.830288] [INFO ] [MAIN ] HESM: Fetched primary data is valid. [2025-10-20T19:09:33.830365] [DEBUG] [LOCAL ] HESM: Saving primary data. [2025-10-20T19:09:33.831842] [DEBUG] [LOCAL ] HESM: Trimmed 69699 records to 48651 (keeping 200 days of data) [2025-10-20T19:09:33.887894] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-01. [2025-10-20T19:09:34.138376] [DEBUG] [LOCAL ] HESM: Primary data saved (48651 records). [2025-10-20T19:09:34.138631] [INFO ] [MAIN ] HESM: Full history saved. [2025-10-20T19:09:34.138806] [INFO ] [MAIN ] Update Progress: 1007/2367 symbols completed. HESM took 35.4s. Estimated remaining: 2h 54m 53s. Rate: 7.8 symbols/min [2025-10-20T19:09:34.221982] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T19:09:34.326821] [INFO ] [MAIN ] HGLB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:34.337638] [DEBUG] [LOCAL ] HGLB: Loaded 9785 existing records from primary storage. [2025-10-20T19:09:34.337883] [INFO ] [MAIN ] HGLB: Loaded 9785 existing records. [2025-10-20T19:09:34.513988] [WARN ] [MAIN ] HGLB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:34.514215] [INFO ] [MAIN ] HGLB: Wiping existing primary data before full refresh. [2025-10-20T19:09:34.514281] [INFO ] [LOCAL ] HGLB: Starting deletion of primary data. [2025-10-20T19:09:34.514455] [DEBUG] [LOCAL ] HGLB: Deleted primary data file. [2025-10-20T19:09:34.514517] [INFO ] [MAIN ] HGLB: Starting full primary refresh. [2025-10-20T19:09:34.620539] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:09:34.663989] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-02. [2025-10-20T19:09:34.882460] [INFO ] [REMOTE ] HGLB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:35.099200] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T19:09:35.451415] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-03. [2025-10-20T19:09:36.152247] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T19:09:36.251424] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-04. [2025-10-20T19:09:37.051351] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-09. [2025-10-20T19:09:37.313667] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T19:09:37.472583] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-10-20T19:09:37.851403] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-10. [2025-10-20T19:09:38.345979] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T19:09:38.346956] [INFO ] [REMOTE ] HFND: Full fetch complete (1757 records). [2025-10-20T19:09:38.347022] [DEBUG] [VALIDATOR ] HFND: Validating primary data. [2025-10-20T19:09:38.521473] [WARN ] [VALIDATOR ] HFND: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-10-20T19:09:38.521672] [FATAL] [MAIN ] HFND: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:09:38.521731] [WARN ] [MAIN ] HFND: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:09:38.521789] [INFO ] [LOCAL ] HFND: Starting deletion of primary data. [2025-10-20T19:09:38.521841] [DEBUG] [LOCAL ] HFND: No primary data file found to delete. [2025-10-20T19:09:38.521876] [INFO ] [LOCAL ] HFND: Starting deletion of derivative data. [2025-10-20T19:09:38.521915] [DEBUG] [LOCAL ] HFND: No derivative data file found to delete. [2025-10-20T19:09:38.521966] [INFO ] [MAIN ] Update Progress: 1008/2367 symbols completed. HFND took 27.5s. Estimated remaining: 2h 54m 41s. Rate: 7.8 symbols/min [2025-10-20T19:09:38.657937] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-04. [2025-10-20T19:09:38.689458] [INFO ] [MAIN ] HGTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:38.705842] [DEBUG] [LOCAL ] HGTY: Loaded 13543 existing records from primary storage. [2025-10-20T19:09:38.706182] [INFO ] [MAIN ] HGTY: Loaded 13543 existing records. [2025-10-20T19:09:38.884921] [WARN ] [MAIN ] HGTY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:38.885118] [INFO ] [MAIN ] HGTY: Wiping existing primary data before full refresh. [2025-10-20T19:09:38.885164] [INFO ] [LOCAL ] HGTY: Starting deletion of primary data. [2025-10-20T19:09:38.885359] [DEBUG] [LOCAL ] HGTY: Deleted primary data file. [2025-10-20T19:09:38.885413] [INFO ] [MAIN ] HGTY: Starting full primary refresh. [2025-10-20T19:09:39.230303] [INFO ] [REMOTE ] HGTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:39.451354] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-05. [2025-10-20T19:09:39.502407] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T19:09:40.224020] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T19:09:40.251381] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-01. [2025-10-20T19:09:40.963331] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T19:09:41.051363] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-05. [2025-10-20T19:09:41.851402] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-01. [2025-10-20T19:09:41.863862] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T19:09:42.603107] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T19:09:42.651354] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-06. [2025-10-20T19:09:43.451375] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-06. [2025-10-20T19:09:43.479402] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T19:09:44.236699] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T19:09:44.251336] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-07. [2025-10-20T19:09:45.051353] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-02. [2025-10-20T19:09:45.178272] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T19:09:45.677896] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T19:09:45.851372] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-03. [2025-10-20T19:09:46.585243] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T19:09:46.651359] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-02. [2025-10-20T19:09:47.415560] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T19:09:47.451382] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-07. [2025-10-20T19:09:48.251382] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-04. [2025-10-20T19:09:48.363038] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T19:09:49.030169] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T19:09:49.051354] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-05. [2025-10-20T19:09:49.851363] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-03. [2025-10-20T19:09:49.958806] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T19:09:50.422484] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T19:09:50.651386] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-08. [2025-10-20T19:09:51.451355] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-08. [2025-10-20T19:09:51.813357] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T19:09:52.251435] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-09. [2025-10-20T19:09:52.427029] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T19:09:52.947600] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-10-20T19:09:53.051401] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-06. [2025-10-20T19:09:53.816134] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T19:09:53.851349] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-07. [2025-10-20T19:09:54.607859] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T19:09:54.651408] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-08. [2025-10-20T19:09:55.451393] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-09. [2025-10-20T19:09:55.583611] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T19:09:56.251368] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-10. [2025-10-20T19:09:56.418426] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:09:57.051378] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-09. [2025-10-20T19:09:57.125101] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T19:09:57.127392] [INFO ] [REMOTE ] HFRO: Full fetch complete (27267 records). [2025-10-20T19:09:57.127465] [DEBUG] [VALIDATOR ] HFRO: Validating primary data. [2025-10-20T19:09:57.496363] [DEBUG] [VALIDATOR ] HFRO: All expected trading days are present. [2025-10-20T19:09:57.496462] [DEBUG] [VALIDATOR ] HFRO: Primary data validation passed. [2025-10-20T19:09:57.497255] [INFO ] [MAIN ] HFRO: Fetched primary data is valid. [2025-10-20T19:09:57.497321] [DEBUG] [LOCAL ] HFRO: Saving primary data. [2025-10-20T19:09:57.498065] [DEBUG] [LOCAL ] HFRO: Trimmed 27267 records to 16506 (keeping 200 days of data) [2025-10-20T19:09:57.598446] [DEBUG] [LOCAL ] HFRO: Primary data saved (16506 records). [2025-10-20T19:09:57.598593] [INFO ] [MAIN ] HFRO: Full history saved. [2025-10-20T19:09:57.598698] [INFO ] [MAIN ] Update Progress: 1009/2367 symbols completed. HFRO took 39.6s. Estimated remaining: 2h 54m 49s. Rate: 7.8 symbols/min [2025-10-20T19:09:57.787598] [INFO ] [MAIN ] HGV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:57.833948] [DEBUG] [LOCAL ] HGV: Loaded 46933 existing records from primary storage. [2025-10-20T19:09:57.834295] [INFO ] [MAIN ] HGV: Loaded 46933 existing records. [2025-10-20T19:09:57.856773] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-10. [2025-10-20T19:09:58.051596] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:09:58.055738] [WARN ] [MAIN ] HGV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:58.056155] [INFO ] [MAIN ] HGV: Wiping existing primary data before full refresh. [2025-10-20T19:09:58.060819] [INFO ] [LOCAL ] HGV: Starting deletion of primary data. [2025-10-20T19:09:58.061856] [DEBUG] [LOCAL ] HGV: Deleted primary data file. [2025-10-20T19:09:58.061927] [INFO ] [MAIN ] HGV: Starting full primary refresh. [2025-10-20T19:09:58.417111] [INFO ] [REMOTE ] HGV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:58.651390] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-01. [2025-10-20T19:09:59.184152] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T19:09:59.189001] [INFO ] [REMOTE ] HGER: Full fetch complete (18593 records). [2025-10-20T19:09:59.189120] [DEBUG] [VALIDATOR ] HGER: Validating primary data. [2025-10-20T19:09:59.456502] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-04. [2025-10-20T19:09:59.563932] [DEBUG] [VALIDATOR ] HGER: All expected trading days are present. [2025-10-20T19:09:59.564016] [DEBUG] [VALIDATOR ] HGER: Primary data validation passed. [2025-10-20T19:09:59.564398] [INFO ] [MAIN ] HGER: Fetched primary data is valid. [2025-10-20T19:09:59.564435] [DEBUG] [LOCAL ] HGER: Saving primary data. [2025-10-20T19:09:59.565100] [DEBUG] [LOCAL ] HGER: Trimmed 18593 records to 15250 (keeping 200 days of data) [2025-10-20T19:09:59.653451] [DEBUG] [LOCAL ] HGER: Primary data saved (15250 records). [2025-10-20T19:09:59.653595] [INFO ] [MAIN ] HGER: Full history saved. [2025-10-20T19:09:59.653701] [INFO ] [MAIN ] Update Progress: 1010/2367 symbols completed. HGER took 28.5s. Estimated remaining: 2h 54m 33s. Rate: 7.8 symbols/min [2025-10-20T19:09:59.821097] [INFO ] [MAIN ] HHH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:59.852973] [DEBUG] [LOCAL ] HHH: Loaded 33589 existing records from primary storage. [2025-10-20T19:09:59.853401] [INFO ] [MAIN ] HHH: Loaded 33589 existing records. [2025-10-20T19:10:00.031615] [WARN ] [MAIN ] HHH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:00.032173] [INFO ] [MAIN ] HHH: Wiping existing primary data before full refresh. [2025-10-20T19:10:00.032224] [INFO ] [LOCAL ] HHH: Starting deletion of primary data. [2025-10-20T19:10:00.032432] [DEBUG] [LOCAL ] HHH: Deleted primary data file. [2025-10-20T19:10:00.032488] [INFO ] [MAIN ] HHH: Starting full primary refresh. [2025-10-20T19:10:00.277975] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-10. [2025-10-20T19:10:00.429572] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:10:00.432717] [INFO ] [REMOTE ] HHH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:00.449473] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T19:10:01.051379] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-02. [2025-10-20T19:10:01.256111] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:10:01.258557] [INFO ] [REMOTE ] HGTY: Full fetch complete (17575 records). [2025-10-20T19:10:01.258627] [DEBUG] [VALIDATOR ] HGTY: Validating primary data. [2025-10-20T19:10:01.646594] [DEBUG] [VALIDATOR ] HGTY: All expected trading days are present. [2025-10-20T19:10:01.646698] [DEBUG] [VALIDATOR ] HGTY: Primary data validation passed. [2025-10-20T19:10:01.647089] [INFO ] [MAIN ] HGTY: Fetched primary data is valid. [2025-10-20T19:10:01.647129] [DEBUG] [LOCAL ] HGTY: Saving primary data. [2025-10-20T19:10:01.647722] [DEBUG] [LOCAL ] HGTY: Trimmed 17575 records to 13382 (keeping 200 days of data) [2025-10-20T19:10:01.733629] [DEBUG] [LOCAL ] HGTY: Primary data saved (13382 records). [2025-10-20T19:10:01.733841] [INFO ] [MAIN ] HGTY: Full history saved. [2025-10-20T19:10:01.733943] [INFO ] [MAIN ] Update Progress: 1011/2367 symbols completed. HGTY took 23.2s. Estimated remaining: 2h 54m 18s. Rate: 7.8 symbols/min [2025-10-20T19:10:01.860746] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-01. [2025-10-20T19:10:01.941377] [INFO ] [MAIN ] HI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:01.982098] [DEBUG] [LOCAL ] HI: Loaded 36524 existing records from primary storage. [2025-10-20T19:10:01.982432] [INFO ] [MAIN ] HI: Loaded 36524 existing records. [2025-10-20T19:10:02.168242] [WARN ] [MAIN ] HI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:02.168707] [INFO ] [MAIN ] HI: Wiping existing primary data before full refresh. [2025-10-20T19:10:02.168802] [INFO ] [LOCAL ] HI: Starting deletion of primary data. [2025-10-20T19:10:02.169206] [DEBUG] [LOCAL ] HI: Deleted primary data file. [2025-10-20T19:10:02.169271] [INFO ] [MAIN ] HI: Starting full primary refresh. [2025-10-20T19:10:02.522837] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:10:02.634480] [INFO ] [REMOTE ] HI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:02.651439] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-01. [2025-10-20T19:10:03.176587] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T19:10:03.451416] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-02. [2025-10-20T19:10:04.143311] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:10:04.251388] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-02. [2025-10-20T19:10:04.835214] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T19:10:05.051426] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-05. [2025-10-20T19:10:05.588715] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T19:10:05.629638] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T19:10:05.851379] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-06. [2025-10-20T19:10:06.547634] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T19:10:06.651384] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-03. [2025-10-20T19:10:07.451434] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-03. [2025-10-20T19:10:07.949505] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:10:08.251418] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-03. [2025-10-20T19:10:09.052699] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-07. [2025-10-20T19:10:09.060419] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T19:10:09.672571] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T19:10:09.837725] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T19:10:09.851391] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-08. [2025-10-20T19:10:10.651475] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-04. [2025-10-20T19:10:10.699058] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T19:10:11.451474] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-04. [2025-10-20T19:10:12.256587] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-04. [2025-10-20T19:10:12.299522] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T19:10:13.051381] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-05. [2025-10-20T19:10:13.098588] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T19:10:13.450271] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T19:10:13.851414] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-05. [2025-10-20T19:10:14.547459] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T19:10:14.651452] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-05. [2025-10-20T19:10:15.455437] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-09. [2025-10-20T19:10:15.462733] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:10:16.022896] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:10:16.218193] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T19:10:16.251348] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-06. [2025-10-20T19:10:17.051406] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-06. [2025-10-20T19:10:17.820066] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:10:17.851459] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-07. [2025-10-20T19:10:18.461717] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T19:10:18.651410] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-06. [2025-10-20T19:10:19.451431] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-07. [2025-10-20T19:10:19.563610] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T19:10:20.010614] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T19:10:20.251376] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-07. [2025-10-20T19:10:20.943860] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T19:10:21.051407] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-08. [2025-10-20T19:10:21.452936] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T19:10:21.851391] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-08. [2025-10-20T19:10:22.238837] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T19:10:22.651448] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-09. [2025-10-20T19:10:23.316162] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T19:10:23.451397] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-10. [2025-10-20T19:10:23.756447] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T19:10:24.251405] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-09. [2025-10-20T19:10:24.383975] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T19:10:24.386256] [INFO ] [REMOTE ] HGLB: Full fetch complete (14413 records). [2025-10-20T19:10:24.386325] [DEBUG] [VALIDATOR ] HGLB: Validating primary data. [2025-10-20T19:10:24.757064] [DEBUG] [VALIDATOR ] HGLB: All expected trading days are present. [2025-10-20T19:10:24.757186] [DEBUG] [VALIDATOR ] HGLB: Primary data validation passed. [2025-10-20T19:10:24.757553] [INFO ] [MAIN ] HGLB: Fetched primary data is valid. [2025-10-20T19:10:24.757600] [DEBUG] [LOCAL ] HGLB: Saving primary data. [2025-10-20T19:10:24.758286] [DEBUG] [LOCAL ] HGLB: Trimmed 14413 records to 9696 (keeping 200 days of data) [2025-10-20T19:10:24.816031] [DEBUG] [LOCAL ] HGLB: Primary data saved (9696 records). [2025-10-20T19:10:24.816154] [INFO ] [MAIN ] HGLB: Full history saved. [2025-10-20T19:10:24.816232] [INFO ] [MAIN ] Update Progress: 1012/2367 symbols completed. HGLB took 50.7s. Estimated remaining: 2h 54m 31s. Rate: 7.8 symbols/min [2025-10-20T19:10:24.988368] [INFO ] [MAIN ] HIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:25.040091] [DEBUG] [LOCAL ] HIG: Loaded 51943 existing records from primary storage. [2025-10-20T19:10:25.040444] [INFO ] [MAIN ] HIG: Loaded 51943 existing records. [2025-10-20T19:10:25.061014] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-10. [2025-10-20T19:10:25.242895] [WARN ] [MAIN ] HIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:25.243172] [INFO ] [MAIN ] HIG: Wiping existing primary data before full refresh. [2025-10-20T19:10:25.243268] [INFO ] [LOCAL ] HIG: Starting deletion of primary data. [2025-10-20T19:10:25.243542] [DEBUG] [LOCAL ] HIG: Deleted primary data file. [2025-10-20T19:10:25.243599] [INFO ] [MAIN ] HIG: Starting full primary refresh. [2025-10-20T19:10:25.614204] [INFO ] [REMOTE ] HIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:25.632400] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T19:10:25.851339] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-10. [2025-10-20T19:10:26.651394] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-01. [2025-10-20T19:10:26.687252] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T19:10:26.693888] [INFO ] [REMOTE ] HGV: Full fetch complete (65908 records). [2025-10-20T19:10:26.694000] [DEBUG] [VALIDATOR ] HGV: Validating primary data. [2025-10-20T19:10:27.080994] [DEBUG] [VALIDATOR ] HGV: All expected trading days are present. [2025-10-20T19:10:27.081151] [DEBUG] [VALIDATOR ] HGV: Primary data validation passed. [2025-10-20T19:10:27.082035] [INFO ] [MAIN ] HGV: Fetched primary data is valid. [2025-10-20T19:10:27.082096] [DEBUG] [LOCAL ] HGV: Saving primary data. [2025-10-20T19:10:27.083441] [DEBUG] [LOCAL ] HGV: Trimmed 65908 records to 46172 (keeping 200 days of data) [2025-10-20T19:10:27.343746] [DEBUG] [LOCAL ] HGV: Primary data saved (46172 records). [2025-10-20T19:10:27.343871] [INFO ] [MAIN ] HGV: Full history saved. [2025-10-20T19:10:27.344721] [INFO ] [MAIN ] Update Progress: 1013/2367 symbols completed. HGV took 29.7s. Estimated remaining: 2h 54m 16s. Rate: 7.8 symbols/min [2025-10-20T19:10:27.461174] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-08. [2025-10-20T19:10:27.514256] [INFO ] [MAIN ] HII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:27.533441] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:10:27.539846] [INFO ] [REMOTE ] HI: Full fetch complete (50955 records). [2025-10-20T19:10:27.539975] [DEBUG] [VALIDATOR ] HI: Validating primary data. [2025-10-20T19:10:27.748164] [DEBUG] [LOCAL ] HII: Loaded 34269 existing records from primary storage. [2025-10-20T19:10:27.785750] [INFO ] [MAIN ] HII: Loaded 34269 existing records. [2025-10-20T19:10:28.124330] [WARN ] [MAIN ] HII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:28.147039] [INFO ] [MAIN ] HII: Wiping existing primary data before full refresh. [2025-10-20T19:10:28.147172] [DEBUG] [VALIDATOR ] HI: All expected trading days are present. [2025-10-20T19:10:28.147280] [INFO ] [LOCAL ] HII: Starting deletion of primary data. [2025-10-20T19:10:28.147341] [DEBUG] [VALIDATOR ] HI: Primary data validation passed. [2025-10-20T19:10:28.148173] [INFO ] [MAIN ] HI: Fetched primary data is valid. [2025-10-20T19:10:28.148283] [DEBUG] [LOCAL ] HI: Saving primary data. [2025-10-20T19:10:28.148478] [DEBUG] [LOCAL ] HII: Deleted primary data file. [2025-10-20T19:10:28.148540] [INFO ] [MAIN ] HII: Starting full primary refresh. [2025-10-20T19:10:28.241507] [DEBUG] [LOCAL ] HI: Trimmed 50955 records to 36061 (keeping 200 days of data) [2025-10-20T19:10:28.766443] [INFO ] [REMOTE ] HII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:28.773577] [DEBUG] [LOCAL ] HI: Primary data saved (36061 records). [2025-10-20T19:10:28.773870] [INFO ] [MAIN ] HI: Full history saved. [2025-10-20T19:10:28.773453] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-01. [2025-10-20T19:10:28.774266] [INFO ] [MAIN ] Update Progress: 1014/2367 symbols completed. HI took 27.0s. Estimated remaining: 2h 54m 0s. Rate: 7.8 symbols/min [2025-10-20T19:10:28.875506] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-01 (2.22 seconds). [2025-10-20T19:10:29.029072] [INFO ] [MAIN ] HIMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:29.031673] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:10:29.064692] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-09. [2025-10-20T19:10:29.089105] [DEBUG] [LOCAL ] HIMS: Loaded 54597 existing records from primary storage. [2025-10-20T19:10:29.089477] [INFO ] [MAIN ] HIMS: Loaded 54597 existing records. [2025-10-20T19:10:29.261159] [WARN ] [MAIN ] HIMS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:29.261360] [INFO ] [MAIN ] HIMS: Wiping existing primary data before full refresh. [2025-10-20T19:10:29.261406] [INFO ] [LOCAL ] HIMS: Starting deletion of primary data. [2025-10-20T19:10:29.261628] [DEBUG] [LOCAL ] HIMS: Deleted primary data file. [2025-10-20T19:10:29.261715] [INFO ] [MAIN ] HIMS: Starting full primary refresh. [2025-10-20T19:10:29.609479] [INFO ] [REMOTE ] HIMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:29.851361] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-01. [2025-10-20T19:10:30.118839] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T19:10:30.189689] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T19:10:30.651434] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-10. [2025-10-20T19:10:31.451407] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-02. [2025-10-20T19:10:32.106910] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:10:32.111398] [INFO ] [REMOTE ] HHH: Full fetch complete (46048 records). [2025-10-20T19:10:32.111480] [DEBUG] [VALIDATOR ] HHH: Validating primary data. [2025-10-20T19:10:32.255319] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-02. [2025-10-20T19:10:32.519383] [DEBUG] [VALIDATOR ] HHH: All expected trading days are present. [2025-10-20T19:10:32.534279] [DEBUG] [VALIDATOR ] HHH: Primary data validation passed. [2025-10-20T19:10:32.535282] [INFO ] [MAIN ] HHH: Fetched primary data is valid. [2025-10-20T19:10:32.535570] [DEBUG] [LOCAL ] HHH: Saving primary data. [2025-10-20T19:10:32.537329] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-01 (2.69 seconds). [2025-10-20T19:10:32.549000] [DEBUG] [LOCAL ] HHH: Trimmed 46048 records to 33185 (keeping 200 days of data) [2025-10-20T19:10:32.754812] [DEBUG] [LOCAL ] HHH: Primary data saved (33185 records). [2025-10-20T19:10:32.755070] [INFO ] [MAIN ] HHH: Full history saved. [2025-10-20T19:10:32.755760] [INFO ] [MAIN ] Update Progress: 1015/2367 symbols completed. HHH took 33.1s. Estimated remaining: 2h 53m 48s. Rate: 7.8 symbols/min [2025-10-20T19:10:32.936044] [INFO ] [MAIN ] HIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:32.961426] [DEBUG] [LOCAL ] HIO: Loaded 24778 existing records from primary storage. [2025-10-20T19:10:32.961703] [INFO ] [MAIN ] HIO: Loaded 24778 existing records. [2025-10-20T19:10:33.056031] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-02. [2025-10-20T19:10:33.173621] [WARN ] [MAIN ] HIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:33.174188] [INFO ] [MAIN ] HIO: Wiping existing primary data before full refresh. [2025-10-20T19:10:33.174780] [INFO ] [LOCAL ] HIO: Starting deletion of primary data. [2025-10-20T19:10:33.176200] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:10:33.186344] [DEBUG] [LOCAL ] HIO: Deleted primary data file. [2025-10-20T19:10:33.186462] [INFO ] [MAIN ] HIO: Starting full primary refresh. [2025-10-20T19:10:33.539108] [INFO ] [REMOTE ] HIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:33.851351] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-01. [2025-10-20T19:10:34.014520] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:10:34.651424] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-03. [2025-10-20T19:10:35.039502] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:10:35.451395] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-02. [2025-10-20T19:10:35.509107] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-02 (2.45 seconds). [2025-10-20T19:10:36.065448] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T19:10:36.251413] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-04. [2025-10-20T19:10:36.735800] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T19:10:37.051432] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-03. [2025-10-20T19:10:37.851407] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-03. [2025-10-20T19:10:37.916565] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T19:10:38.651412] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-03. [2025-10-20T19:10:39.451404] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-05. [2025-10-20T19:10:39.555518] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T19:10:39.662210] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-03 (2.61 seconds). [2025-10-20T19:10:39.691222] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T19:10:40.251376] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-04. [2025-10-20T19:10:41.051382] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-04. [2025-10-20T19:10:41.191933] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T19:10:41.851376] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-04. [2025-10-20T19:10:42.651353] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-06. [2025-10-20T19:10:42.715749] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T19:10:42.876600] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-04 (2.63 seconds). [2025-10-20T19:10:42.970185] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T19:10:43.451363] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-05. [2025-10-20T19:10:44.192415] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:10:44.251349] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-07. [2025-10-20T19:10:45.051360] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-05. [2025-10-20T19:10:45.851365] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-05. [2025-10-20T19:10:45.923908] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T19:10:46.211936] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-05 (2.76 seconds). [2025-10-20T19:10:46.556384] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:10:46.651348] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-06. [2025-10-20T19:10:46.878716] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T19:10:47.451376] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-06. [2025-10-20T19:10:48.251398] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-08. [2025-10-20T19:10:48.524003] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T19:10:48.994348] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-06 (2.34 seconds). [2025-10-20T19:10:49.051333] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-07. [2025-10-20T19:10:49.685415] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T19:10:49.851358] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-09. [2025-10-20T19:10:50.651440] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-07. [2025-10-20T19:10:51.460952] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-06. [2025-10-20T19:10:51.466758] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T19:10:51.812724] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-07 (2.76 seconds). [2025-10-20T19:10:51.845477] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T19:10:52.187049] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:10:52.251362] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-08. [2025-10-20T19:10:53.051440] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-08. [2025-10-20T19:10:53.851408] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-07. [2025-10-20T19:10:54.592897] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T19:10:54.651381] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-09. [2025-10-20T19:10:55.024874] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T19:10:55.080024] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-08 (2.83 seconds). [2025-10-20T19:10:55.451372] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-10. [2025-10-20T19:10:55.975609] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T19:10:55.983070] [INFO ] [REMOTE ] HIG: Full fetch complete (73639 records). [2025-10-20T19:10:55.983177] [DEBUG] [VALIDATOR ] HIG: Validating primary data. [2025-10-20T19:10:56.126220] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:10:56.261549] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-08. [2025-10-20T19:10:56.404747] [DEBUG] [VALIDATOR ] HIG: All expected trading days are present. [2025-10-20T19:10:56.404834] [DEBUG] [VALIDATOR ] HIG: Primary data validation passed. [2025-10-20T19:10:56.405767] [INFO ] [MAIN ] HIG: Fetched primary data is valid. [2025-10-20T19:10:56.405829] [DEBUG] [LOCAL ] HIG: Saving primary data. [2025-10-20T19:10:56.407173] [DEBUG] [LOCAL ] HIG: Trimmed 73639 records to 51188 (keeping 200 days of data) [2025-10-20T19:10:56.727602] [DEBUG] [LOCAL ] HIG: Primary data saved (51188 records). [2025-10-20T19:10:56.727772] [INFO ] [MAIN ] HIG: Full history saved. [2025-10-20T19:10:56.727857] [INFO ] [MAIN ] Update Progress: 1016/2367 symbols completed. HIG took 31.9s. Estimated remaining: 2h 54m 1s. Rate: 7.8 symbols/min [2025-10-20T19:10:56.897974] [INFO ] [MAIN ] HIPO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:56.918606] [DEBUG] [LOCAL ] HIPO: Loaded 22131 existing records from primary storage. [2025-10-20T19:10:56.918948] [INFO ] [MAIN ] HIPO: Loaded 22131 existing records. [2025-10-20T19:10:57.058085] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-09. [2025-10-20T19:10:57.100994] [WARN ] [MAIN ] HIPO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:57.101193] [INFO ] [MAIN ] HIPO: Wiping existing primary data before full refresh. [2025-10-20T19:10:57.101241] [INFO ] [LOCAL ] HIPO: Starting deletion of primary data. [2025-10-20T19:10:57.101461] [DEBUG] [LOCAL ] HIPO: Deleted primary data file. [2025-10-20T19:10:57.101500] [INFO ] [MAIN ] HIPO: Starting full primary refresh. [2025-10-20T19:10:57.457046] [INFO ] [REMOTE ] HIPO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:57.745884] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:10:57.851356] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-01. [2025-10-20T19:10:58.651358] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-09. [2025-10-20T19:10:58.955234] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:10:59.451379] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-02. [2025-10-20T19:10:59.546610] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-09 (2.49 seconds). [2025-10-20T19:10:59.911184] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T19:11:00.251374] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-10. [2025-10-20T19:11:00.428787] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T19:11:01.051379] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-10. [2025-10-20T19:11:01.851440] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-03. [2025-10-20T19:11:02.005163] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T19:11:02.013920] [INFO ] [REMOTE ] HIO: Full fetch complete (37954 records). [2025-10-20T19:11:02.014131] [DEBUG] [VALIDATOR ] HIO: Validating primary data. [2025-10-20T19:11:02.559690] [DEBUG] [VALIDATOR ] HIO: All expected trading days are present. [2025-10-20T19:11:02.559814] [DEBUG] [VALIDATOR ] HIO: Primary data validation passed. [2025-10-20T19:11:02.560809] [INFO ] [MAIN ] HIO: Fetched primary data is valid. [2025-10-20T19:11:02.560888] [DEBUG] [LOCAL ] HIO: Saving primary data. [2025-10-20T19:11:02.562239] [DEBUG] [LOCAL ] HIO: Trimmed 37954 records to 24124 (keeping 200 days of data) [2025-10-20T19:11:02.690261] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-10. [2025-10-20T19:11:02.790678] [DEBUG] [LOCAL ] HIO: Primary data saved (24124 records). [2025-10-20T19:11:02.790882] [INFO ] [MAIN ] HIO: Full history saved. [2025-10-20T19:11:02.791010] [INFO ] [MAIN ] Update Progress: 1017/2367 symbols completed. HIO took 30.0s. Estimated remaining: 2h 53m 51s. Rate: 7.8 symbols/min [2025-10-20T19:11:03.004440] [INFO ] [MAIN ] HIW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:03.020408] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T19:11:03.075197] [DEBUG] [LOCAL ] HIW: Loaded 45201 existing records from primary storage. [2025-10-20T19:11:03.075534] [INFO ] [MAIN ] HIW: Loaded 45201 existing records. [2025-10-20T19:11:03.251774] [WARN ] [MAIN ] HIW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:03.252005] [INFO ] [MAIN ] HIW: Wiping existing primary data before full refresh. [2025-10-20T19:11:03.252052] [INFO ] [LOCAL ] HIW: Starting deletion of primary data. [2025-10-20T19:11:03.252504] [DEBUG] [LOCAL ] HIW: Deleted primary data file. [2025-10-20T19:11:03.252558] [INFO ] [MAIN ] HIW: Starting full primary refresh. [2025-10-20T19:11:03.473718] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-04. [2025-10-20T19:11:03.613805] [INFO ] [REMOTE ] HIW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:03.706050] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-10 (2.65 seconds). [2025-10-20T19:11:03.716492] [INFO ] [REMOTE ] HIMS: Full fetch complete (77996 records). [2025-10-20T19:11:03.716795] [DEBUG] [VALIDATOR ] HIMS: Validating primary data. [2025-10-20T19:11:04.130856] [DEBUG] [VALIDATOR ] HIMS: All expected trading days are present. [2025-10-20T19:11:04.131002] [DEBUG] [VALIDATOR ] HIMS: Primary data validation passed. [2025-10-20T19:11:04.132154] [INFO ] [MAIN ] HIMS: Fetched primary data is valid. [2025-10-20T19:11:04.132218] [DEBUG] [LOCAL ] HIMS: Saving primary data. [2025-10-20T19:11:04.133718] [DEBUG] [LOCAL ] HIMS: Trimmed 77996 records to 53817 (keeping 200 days of data) [2025-10-20T19:11:04.251356] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-01. [2025-10-20T19:11:04.360895] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:11:04.381273] [INFO ] [REMOTE ] HII: Full fetch complete (52464 records). [2025-10-20T19:11:04.386074] [DEBUG] [VALIDATOR ] HII: Validating primary data. [2025-10-20T19:11:04.767058] [DEBUG] [LOCAL ] HIMS: Primary data saved (53817 records). [2025-10-20T19:11:04.788346] [INFO ] [MAIN ] HIMS: Full history saved. [2025-10-20T19:11:04.790182] [INFO ] [MAIN ] Update Progress: 1018/2367 symbols completed. HIMS took 36.0s. Estimated remaining: 2h 53m 36s. Rate: 7.8 symbols/min [2025-10-20T19:11:04.824706] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T19:11:05.029722] [DEBUG] [VALIDATOR ] HII: All expected trading days are present. [2025-10-20T19:11:05.041728] [DEBUG] [VALIDATOR ] HII: Primary data validation passed. [2025-10-20T19:11:05.071569] [INFO ] [MAIN ] HIX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:05.069174] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-05. [2025-10-20T19:11:05.072761] [INFO ] [MAIN ] HII: Fetched primary data is valid. [2025-10-20T19:11:05.072869] [DEBUG] [LOCAL ] HII: Saving primary data. [2025-10-20T19:11:05.074667] [DEBUG] [LOCAL ] HII: Trimmed 52464 records to 33610 (keeping 200 days of data) [2025-10-20T19:11:05.198933] [DEBUG] [LOCAL ] HIX: Loaded 20696 existing records from primary storage. [2025-10-20T19:11:05.213196] [INFO ] [MAIN ] HIX: Loaded 20696 existing records. [2025-10-20T19:11:05.456628] [DEBUG] [LOCAL ] HII: Primary data saved (33610 records). [2025-10-20T19:11:05.472963] [INFO ] [MAIN ] HII: Full history saved. [2025-10-20T19:11:05.473256] [INFO ] [MAIN ] Update Progress: 1019/2367 symbols completed. HII took 38.1s. Estimated remaining: 2h 53m 19s. Rate: 7.8 symbols/min [2025-10-20T19:11:05.477756] [WARN ] [MAIN ] HIX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:05.477918] [INFO ] [MAIN ] HIX: Wiping existing primary data before full refresh. [2025-10-20T19:11:05.477986] [INFO ] [LOCAL ] HIX: Starting deletion of primary data. [2025-10-20T19:11:05.509945] [DEBUG] [LOCAL ] HIX: Deleted primary data file. [2025-10-20T19:11:05.521451] [INFO ] [MAIN ] HIX: Starting full primary refresh. [2025-10-20T19:11:05.776086] [INFO ] [MAIN ] HKD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:05.959483] [DEBUG] [LOCAL ] HKD: Loaded 18952 existing records from primary storage. [2025-10-20T19:11:06.004238] [INFO ] [MAIN ] HKD: Loaded 18952 existing records. [2025-10-20T19:11:06.033071] [INFO ] [REMOTE ] HIX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:06.033433] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-01. [2025-10-20T19:11:06.183237] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:11:06.190194] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-01 (1.94 seconds). [2025-10-20T19:11:06.260097] [WARN ] [MAIN ] HKD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:06.260273] [INFO ] [MAIN ] HKD: Wiping existing primary data before full refresh. [2025-10-20T19:11:06.260320] [INFO ] [LOCAL ] HKD: Starting deletion of primary data. [2025-10-20T19:11:06.260477] [DEBUG] [LOCAL ] HKD: Deleted primary data file. [2025-10-20T19:11:06.260513] [INFO ] [MAIN ] HKD: Starting full primary refresh. [2025-10-20T19:11:06.607323] [INFO ] [REMOTE ] HKD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:06.651364] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-02. [2025-10-20T19:11:07.224870] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:11:07.451374] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-01. [2025-10-20T19:11:08.218224] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:11:08.251328] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-03. [2025-10-20T19:11:08.659108] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T19:11:09.051392] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-02. [2025-10-20T19:11:09.749203] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:11:09.851377] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-04. [2025-10-20T19:11:09.991977] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T19:11:10.651384] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-06. [2025-10-20T19:11:11.460263] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-02. [2025-10-20T19:11:11.463056] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:11:11.848094] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T19:11:12.251342] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-03. [2025-10-20T19:11:12.344078] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T19:11:13.051391] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-05. [2025-10-20T19:11:13.276171] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T19:11:13.851387] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-04. [2025-10-20T19:11:14.490557] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T19:11:14.651351] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-07. [2025-10-20T19:11:15.050953] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:11:15.451384] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-05. [2025-10-20T19:11:16.154869] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:11:16.251385] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-03. [2025-10-20T19:11:17.051414] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-08. [2025-10-20T19:11:17.851430] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-06. [2025-10-20T19:11:18.023126] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-05 (2.57 seconds). [2025-10-20T19:11:18.172180] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T19:11:18.452545] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-03 (2.20 seconds). [2025-10-20T19:11:18.651377] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-04. [2025-10-20T19:11:19.321899] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:11:19.374676] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T19:11:19.451379] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-06. [2025-10-20T19:11:20.251382] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-07. [2025-10-20T19:11:20.429123] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:11:21.051397] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-05. [2025-10-20T19:11:21.772954] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T19:11:21.809702] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T19:11:21.851343] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-06. [2025-10-20T19:11:22.651458] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-09. [2025-10-20T19:11:23.140291] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T19:11:23.451366] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-07. [2025-10-20T19:11:23.670628] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:11:24.251402] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-10. [2025-10-20T19:11:24.314451] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T19:11:25.051369] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-08. [2025-10-20T19:11:25.291716] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T19:11:25.294487] [INFO ] [REMOTE ] HIPO: Full fetch complete (31089 records). [2025-10-20T19:11:25.294560] [DEBUG] [VALIDATOR ] HIPO: Validating primary data. [2025-10-20T19:11:25.689882] [DEBUG] [VALIDATOR ] HIPO: All expected trading days are present. [2025-10-20T19:11:25.689986] [DEBUG] [VALIDATOR ] HIPO: Primary data validation passed. [2025-10-20T19:11:25.690570] [INFO ] [MAIN ] HIPO: Fetched primary data is valid. [2025-10-20T19:11:25.690610] [DEBUG] [LOCAL ] HIPO: Saving primary data. [2025-10-20T19:11:25.691299] [DEBUG] [LOCAL ] HIPO: Trimmed 31089 records to 21623 (keeping 200 days of data) [2025-10-20T19:11:25.811969] [DEBUG] [LOCAL ] HIPO: Primary data saved (21623 records). [2025-10-20T19:11:25.812090] [INFO ] [MAIN ] HIPO: Full history saved. [2025-10-20T19:11:25.812164] [INFO ] [MAIN ] Update Progress: 1020/2367 symbols completed. HIPO took 29.1s. Estimated remaining: 2h 53m 28s. Rate: 7.8 symbols/min [2025-10-20T19:11:25.856612] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-08. [2025-10-20T19:11:26.002788] [INFO ] [MAIN ] HL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:26.051986] [DEBUG] [LOCAL ] HL: Loaded 54455 existing records from primary storage. [2025-10-20T19:11:26.052349] [INFO ] [MAIN ] HL: Loaded 54455 existing records. [2025-10-20T19:11:26.228902] [WARN ] [MAIN ] HL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:26.229110] [INFO ] [MAIN ] HL: Wiping existing primary data before full refresh. [2025-10-20T19:11:26.229156] [INFO ] [LOCAL ] HL: Starting deletion of primary data. [2025-10-20T19:11:26.229376] [DEBUG] [LOCAL ] HL: Deleted primary data file. [2025-10-20T19:11:26.229431] [INFO ] [MAIN ] HL: Starting full primary refresh. [2025-10-20T19:11:26.592308] [INFO ] [REMOTE ] HL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:26.651372] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-07. [2025-10-20T19:11:26.725685] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T19:11:26.876098] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T19:11:27.356149] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T19:11:27.451370] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-09. [2025-10-20T19:11:28.251342] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-01. [2025-10-20T19:11:28.778636] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T19:11:29.051364] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-09. [2025-10-20T19:11:29.851370] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-10. [2025-10-20T19:11:30.001810] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T19:11:30.651403] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-02. [2025-10-20T19:11:30.856140] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:11:30.954686] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T19:11:30.958187] [INFO ] [REMOTE ] HKD: Full fetch complete (28019 records). [2025-10-20T19:11:30.958269] [DEBUG] [VALIDATOR ] HKD: Validating primary data. [2025-10-20T19:11:31.344348] [DEBUG] [VALIDATOR ] HKD: All expected trading days are present. [2025-10-20T19:11:31.344447] [DEBUG] [VALIDATOR ] HKD: Primary data validation passed. [2025-10-20T19:11:31.345096] [INFO ] [MAIN ] HKD: Fetched primary data is valid. [2025-10-20T19:11:31.345158] [DEBUG] [LOCAL ] HKD: Saving primary data. [2025-10-20T19:11:31.345922] [DEBUG] [LOCAL ] HKD: Trimmed 28019 records to 18743 (keeping 200 days of data) [2025-10-20T19:11:31.448780] [DEBUG] [LOCAL ] HKD: Primary data saved (18743 records). [2025-10-20T19:11:31.448940] [INFO ] [MAIN ] HKD: Full history saved. [2025-10-20T19:11:31.449028] [INFO ] [MAIN ] Update Progress: 1021/2367 symbols completed. HKD took 26.0s. Estimated remaining: 2h 53m 18s. Rate: 7.8 symbols/min [2025-10-20T19:11:31.460452] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-08. [2025-10-20T19:11:31.631821] [INFO ] [MAIN ] HLF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:31.676522] [DEBUG] [LOCAL ] HLF: Loaded 49226 existing records from primary storage. [2025-10-20T19:11:31.676853] [INFO ] [MAIN ] HLF: Loaded 49226 existing records. [2025-10-20T19:11:31.855129] [WARN ] [MAIN ] HLF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:31.855368] [INFO ] [MAIN ] HLF: Wiping existing primary data before full refresh. [2025-10-20T19:11:31.855415] [INFO ] [LOCAL ] HLF: Starting deletion of primary data. [2025-10-20T19:11:31.855610] [DEBUG] [LOCAL ] HLF: Deleted primary data file. [2025-10-20T19:11:31.855696] [INFO ] [MAIN ] HLF: Starting full primary refresh. [2025-10-20T19:11:32.231501] [INFO ] [REMOTE ] HLF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:32.251341] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-01. [2025-10-20T19:11:33.051387] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-10. [2025-10-20T19:11:33.444406] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-02 (2.79 seconds). [2025-10-20T19:11:33.669241] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T19:11:33.851372] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-03. [2025-10-20T19:11:34.033048] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:11:34.651349] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-09. [2025-10-20T19:11:34.734788] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:11:34.742834] [INFO ] [REMOTE ] HIW: Full fetch complete (64864 records). [2025-10-20T19:11:34.743014] [DEBUG] [VALIDATOR ] HIW: Validating primary data. [2025-10-20T19:11:35.158799] [DEBUG] [VALIDATOR ] HIW: All expected trading days are present. [2025-10-20T19:11:35.158900] [DEBUG] [VALIDATOR ] HIW: Primary data validation passed. [2025-10-20T19:11:35.159769] [INFO ] [MAIN ] HIW: Fetched primary data is valid. [2025-10-20T19:11:35.159831] [DEBUG] [LOCAL ] HIW: Saving primary data. [2025-10-20T19:11:35.161061] [DEBUG] [LOCAL ] HIW: Trimmed 64864 records to 44503 (keeping 200 days of data) [2025-10-20T19:11:35.426153] [DEBUG] [LOCAL ] HIW: Primary data saved (44503 records). [2025-10-20T19:11:35.426321] [INFO ] [MAIN ] HIW: Full history saved. [2025-10-20T19:11:35.426800] [INFO ] [MAIN ] Update Progress: 1022/2367 symbols completed. HIW took 32.6s. Estimated remaining: 2h 53m 5s. Rate: 7.8 symbols/min [2025-10-20T19:11:35.458595] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-02. [2025-10-20T19:11:35.606748] [INFO ] [MAIN ] HLI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:35.641846] [DEBUG] [LOCAL ] HLI: Loaded 31316 existing records from primary storage. [2025-10-20T19:11:35.642111] [INFO ] [MAIN ] HLI: Loaded 31316 existing records. [2025-10-20T19:11:35.848172] [WARN ] [MAIN ] HLI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:35.834581] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T19:11:35.853022] [INFO ] [MAIN ] HLI: Wiping existing primary data before full refresh. [2025-10-20T19:11:35.853247] [INFO ] [LOCAL ] HLI: Starting deletion of primary data. [2025-10-20T19:11:35.853899] [DEBUG] [LOCAL ] HLI: Deleted primary data file. [2025-10-20T19:11:35.854074] [INFO ] [MAIN ] HLI: Starting full primary refresh. [2025-10-20T19:11:36.186362] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-03 (2.33 seconds). [2025-10-20T19:11:36.252252] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-04. [2025-10-20T19:11:36.266160] [INFO ] [REMOTE ] HLI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:37.051365] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-10. [2025-10-20T19:11:37.162339] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T19:11:37.851376] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-03. [2025-10-20T19:11:37.956921] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:11:38.211449] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T19:11:38.214500] [INFO ] [REMOTE ] HIX: Full fetch complete (29871 records). [2025-10-20T19:11:38.214577] [DEBUG] [VALIDATOR ] HIX: Validating primary data. [2025-10-20T19:11:38.582692] [DEBUG] [VALIDATOR ] HIX: All expected trading days are present. [2025-10-20T19:11:38.582772] [DEBUG] [VALIDATOR ] HIX: Primary data validation passed. [2025-10-20T19:11:38.583341] [INFO ] [MAIN ] HIX: Fetched primary data is valid. [2025-10-20T19:11:38.583382] [DEBUG] [LOCAL ] HIX: Saving primary data. [2025-10-20T19:11:38.584144] [DEBUG] [LOCAL ] HIX: Trimmed 29871 records to 20398 (keeping 200 days of data) [2025-10-20T19:11:38.664395] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-01. [2025-10-20T19:11:38.703542] [DEBUG] [LOCAL ] HIX: Primary data saved (20398 records). [2025-10-20T19:11:38.703716] [INFO ] [MAIN ] HIX: Full history saved. [2025-10-20T19:11:38.703805] [INFO ] [MAIN ] Update Progress: 1023/2367 symbols completed. HIX took 33.9s. Estimated remaining: 2h 52m 51s. Rate: 7.8 symbols/min [2025-10-20T19:11:38.876281] [INFO ] [MAIN ] HLIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:38.904700] [DEBUG] [LOCAL ] HLIO: Loaded 28680 existing records from primary storage. [2025-10-20T19:11:38.904954] [INFO ] [MAIN ] HLIO: Loaded 28680 existing records. [2025-10-20T19:11:39.083578] [WARN ] [MAIN ] HLIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:39.083925] [INFO ] [MAIN ] HLIO: Wiping existing primary data before full refresh. [2025-10-20T19:11:39.083984] [INFO ] [LOCAL ] HLIO: Starting deletion of primary data. [2025-10-20T19:11:39.084151] [DEBUG] [LOCAL ] HLIO: Deleted primary data file. [2025-10-20T19:11:39.084190] [INFO ] [MAIN ] HLIO: Starting full primary refresh. [2025-10-20T19:11:39.450801] [INFO ] [REMOTE ] HLIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:39.451338] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-01. [2025-10-20T19:11:39.482157] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:11:40.034745] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:11:40.251425] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-05. [2025-10-20T19:11:40.454899] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T19:11:41.051385] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-04. [2025-10-20T19:11:41.851383] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-02. [2025-10-20T19:11:42.162663] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:11:42.651435] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-02. [2025-10-20T19:11:42.739099] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T19:11:42.804545] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T19:11:43.451360] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-05. [2025-10-20T19:11:43.624854] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T19:11:44.251424] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-03. [2025-10-20T19:11:45.051404] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-03. [2025-10-20T19:11:45.144628] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:11:45.662778] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T19:11:45.784686] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T19:11:45.851418] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-04. [2025-10-20T19:11:46.651395] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-06. [2025-10-20T19:11:47.079130] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T19:11:47.451379] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-05. [2025-10-20T19:11:48.251407] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-06. [2025-10-20T19:11:48.296621] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:11:48.790805] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T19:11:49.051409] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-04. [2025-10-20T19:11:49.851352] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-06. [2025-10-20T19:11:49.966853] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T19:11:50.354710] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T19:11:50.651427] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-07. [2025-10-20T19:11:50.710062] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T19:11:51.451510] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-07. [2025-10-20T19:11:52.251422] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-07. [2025-10-20T19:11:52.656339] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T19:11:52.868033] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:11:53.051369] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-08. [2025-10-20T19:11:53.851352] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-05. [2025-10-20T19:11:53.948745] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:11:54.651453] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-08. [2025-10-20T19:11:55.209905] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-08 (2.16 seconds). [2025-10-20T19:11:55.211609] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T19:11:55.451384] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-06. [2025-10-20T19:11:56.005148] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:11:56.251406] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-09. [2025-10-20T19:11:56.792680] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T19:11:57.051486] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-09. [2025-10-20T19:11:57.851381] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-08. [2025-10-20T19:11:58.514435] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-09 (2.26 seconds). [2025-10-20T19:11:58.651361] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-07. [2025-10-20T19:11:58.817096] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:11:59.202304] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:11:59.451425] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-10. [2025-10-20T19:11:59.993313] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T19:12:00.251422] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-08. [2025-10-20T19:12:01.051359] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-10. [2025-10-20T19:12:01.394411] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:12:01.401592] [INFO ] [REMOTE ] HLF: Full fetch complete (70919 records). [2025-10-20T19:12:01.401873] [DEBUG] [VALIDATOR ] HLF: Validating primary data. [2025-10-20T19:12:01.552187] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T19:12:01.848260] [DEBUG] [VALIDATOR ] HLF: All expected trading days are present. [2025-10-20T19:12:01.848347] [DEBUG] [VALIDATOR ] HLF: Primary data validation passed. [2025-10-20T19:12:01.849345] [INFO ] [MAIN ] HLF: Fetched primary data is valid. [2025-10-20T19:12:01.849411] [DEBUG] [LOCAL ] HLF: Saving primary data. [2025-10-20T19:12:01.850895] [DEBUG] [LOCAL ] HLF: Trimmed 70919 records to 48540 (keeping 200 days of data) [2025-10-20T19:12:01.851465] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-09. [2025-10-20T19:12:02.163250] [DEBUG] [LOCAL ] HLF: Primary data saved (48540 records). [2025-10-20T19:12:02.163484] [INFO ] [MAIN ] HLF: Full history saved. [2025-10-20T19:12:02.164303] [INFO ] [MAIN ] Update Progress: 1024/2367 symbols completed. HLF took 30.7s. Estimated remaining: 2h 53m 4s. Rate: 7.8 symbols/min [2025-10-20T19:12:02.331634] [INFO ] [MAIN ] HLLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:02.364879] [DEBUG] [LOCAL ] HLLY: Loaded 33825 existing records from primary storage. [2025-10-20T19:12:02.365143] [INFO ] [MAIN ] HLLY: Loaded 33825 existing records. [2025-10-20T19:12:02.539386] [WARN ] [MAIN ] HLLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:02.539639] [INFO ] [MAIN ] HLLY: Wiping existing primary data before full refresh. [2025-10-20T19:12:02.539720] [INFO ] [LOCAL ] HLLY: Starting deletion of primary data. [2025-10-20T19:12:02.539929] [DEBUG] [LOCAL ] HLLY: Deleted primary data file. [2025-10-20T19:12:02.539986] [INFO ] [MAIN ] HLLY: Starting full primary refresh. [2025-10-20T19:12:02.656519] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-09. [2025-10-20T19:12:02.908860] [INFO ] [REMOTE ] HLLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:03.001504] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:12:03.451411] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-10. [2025-10-20T19:12:03.476164] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-10 (2.42 seconds). [2025-10-20T19:12:03.484342] [INFO ] [REMOTE ] HL: Full fetch complete (77825 records). [2025-10-20T19:12:03.484473] [DEBUG] [VALIDATOR ] HL: Validating primary data. [2025-10-20T19:12:03.878575] [DEBUG] [VALIDATOR ] HL: All expected trading days are present. [2025-10-20T19:12:03.878709] [DEBUG] [VALIDATOR ] HL: Primary data validation passed. [2025-10-20T19:12:03.879932] [INFO ] [MAIN ] HL: Fetched primary data is valid. [2025-10-20T19:12:03.879997] [DEBUG] [LOCAL ] HL: Saving primary data. [2025-10-20T19:12:03.881443] [DEBUG] [LOCAL ] HL: Trimmed 77825 records to 53676 (keeping 200 days of data) [2025-10-20T19:12:04.103598] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T19:12:04.230411] [DEBUG] [LOCAL ] HL: Primary data saved (53676 records). [2025-10-20T19:12:04.230566] [INFO ] [MAIN ] HL: Full history saved. [2025-10-20T19:12:04.230671] [INFO ] [MAIN ] Update Progress: 1025/2367 symbols completed. HL took 38.4s. Estimated remaining: 2h 52m 49s. Rate: 7.8 symbols/min [2025-10-20T19:12:04.259965] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-01. [2025-10-20T19:12:04.401713] [INFO ] [MAIN ] HLN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:04.451677] [DEBUG] [LOCAL ] HLN: Loaded 53902 existing records from primary storage. [2025-10-20T19:12:04.451994] [INFO ] [MAIN ] HLN: Loaded 53902 existing records. [2025-10-20T19:12:04.636334] [WARN ] [MAIN ] HLN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:04.636589] [INFO ] [MAIN ] HLN: Wiping existing primary data before full refresh. [2025-10-20T19:12:04.636768] [INFO ] [LOCAL ] HLN: Starting deletion of primary data. [2025-10-20T19:12:04.637374] [DEBUG] [LOCAL ] HLN: Deleted primary data file. [2025-10-20T19:12:04.637427] [INFO ] [MAIN ] HLN: Starting full primary refresh. [2025-10-20T19:12:04.762000] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:12:04.794373] [INFO ] [REMOTE ] HLIO: Full fetch complete (37110 records). [2025-10-20T19:12:04.805227] [DEBUG] [VALIDATOR ] HLIO: Validating primary data. [2025-10-20T19:12:05.052079] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-10. [2025-10-20T19:12:05.260767] [INFO ] [REMOTE ] HLN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:05.420805] [DEBUG] [VALIDATOR ] HLIO: All expected trading days are present. [2025-10-20T19:12:05.420925] [DEBUG] [VALIDATOR ] HLIO: Primary data validation passed. [2025-10-20T19:12:05.421769] [INFO ] [MAIN ] HLIO: Fetched primary data is valid. [2025-10-20T19:12:05.421841] [DEBUG] [LOCAL ] HLIO: Saving primary data. [2025-10-20T19:12:05.423022] [DEBUG] [LOCAL ] HLIO: Trimmed 37110 records to 28259 (keeping 200 days of data) [2025-10-20T19:12:05.605604] [DEBUG] [LOCAL ] HLIO: Primary data saved (28259 records). [2025-10-20T19:12:05.605800] [INFO ] [MAIN ] HLIO: Full history saved. [2025-10-20T19:12:05.605987] [INFO ] [MAIN ] Update Progress: 1026/2367 symbols completed. HLIO took 26.9s. Estimated remaining: 2h 52m 33s. Rate: 7.8 symbols/min [2025-10-20T19:12:05.769667] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T19:12:05.819723] [INFO ] [MAIN ] HLT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:05.851340] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-02. [2025-10-20T19:12:05.874992] [DEBUG] [LOCAL ] HLT: Loaded 51246 existing records from primary storage. [2025-10-20T19:12:05.875356] [INFO ] [MAIN ] HLT: Loaded 51246 existing records. [2025-10-20T19:12:06.061367] [WARN ] [MAIN ] HLT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:06.061576] [INFO ] [MAIN ] HLT: Wiping existing primary data before full refresh. [2025-10-20T19:12:06.061623] [INFO ] [LOCAL ] HLT: Starting deletion of primary data. [2025-10-20T19:12:06.061887] [DEBUG] [LOCAL ] HLT: Deleted primary data file. [2025-10-20T19:12:06.061941] [INFO ] [MAIN ] HLT: Starting full primary refresh. [2025-10-20T19:12:06.407074] [INFO ] [REMOTE ] HLT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:06.651360] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-01. [2025-10-20T19:12:06.697241] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T19:12:06.701839] [INFO ] [REMOTE ] HLI: Full fetch complete (45378 records). [2025-10-20T19:12:06.701927] [DEBUG] [VALIDATOR ] HLI: Validating primary data. [2025-10-20T19:12:07.104693] [DEBUG] [VALIDATOR ] HLI: All expected trading days are present. [2025-10-20T19:12:07.112191] [DEBUG] [VALIDATOR ] HLI: Primary data validation passed. [2025-10-20T19:12:07.113033] [INFO ] [MAIN ] HLI: Fetched primary data is valid. [2025-10-20T19:12:07.113114] [DEBUG] [LOCAL ] HLI: Saving primary data. [2025-10-20T19:12:07.114307] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T19:12:07.120372] [DEBUG] [LOCAL ] HLI: Trimmed 45378 records to 30785 (keeping 200 days of data) [2025-10-20T19:12:07.307367] [DEBUG] [LOCAL ] HLI: Primary data saved (30785 records). [2025-10-20T19:12:07.307492] [INFO ] [MAIN ] HLI: Full history saved. [2025-10-20T19:12:07.307829] [INFO ] [MAIN ] Update Progress: 1027/2367 symbols completed. HLI took 31.9s. Estimated remaining: 2h 52m 18s. Rate: 7.8 symbols/min [2025-10-20T19:12:07.459758] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-03. [2025-10-20T19:12:07.484283] [INFO ] [MAIN ] HLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:07.528978] [DEBUG] [LOCAL ] HLX: Loaded 48172 existing records from primary storage. [2025-10-20T19:12:07.529312] [INFO ] [MAIN ] HLX: Loaded 48172 existing records. [2025-10-20T19:12:07.710269] [WARN ] [MAIN ] HLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:07.710496] [INFO ] [MAIN ] HLX: Wiping existing primary data before full refresh. [2025-10-20T19:12:07.710546] [INFO ] [LOCAL ] HLX: Starting deletion of primary data. [2025-10-20T19:12:07.710776] [DEBUG] [LOCAL ] HLX: Deleted primary data file. [2025-10-20T19:12:07.710832] [INFO ] [MAIN ] HLX: Starting full primary refresh. [2025-10-20T19:12:08.069005] [INFO ] [REMOTE ] HLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:08.251346] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-01. [2025-10-20T19:12:08.431334] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:12:08.796932] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T19:12:09.051416] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-02. [2025-10-20T19:12:09.637464] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T19:12:09.851377] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-02. [2025-10-20T19:12:10.651382] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-04. [2025-10-20T19:12:10.707622] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:12:11.374104] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T19:12:11.451380] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-03. [2025-10-20T19:12:12.138964] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T19:12:12.251387] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-03. [2025-10-20T19:12:13.051436] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-05. [2025-10-20T19:12:13.190556] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T19:12:13.851378] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-01. [2025-10-20T19:12:14.187133] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T19:12:14.438951] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:12:14.651358] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-04. [2025-10-20T19:12:15.451420] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-04. [2025-10-20T19:12:15.559468] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T19:12:16.251415] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-02. [2025-10-20T19:12:16.539930] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T19:12:17.051361] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-05. [2025-10-20T19:12:17.168932] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T19:12:17.851375] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-05. [2025-10-20T19:12:18.040119] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:12:18.651346] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-06. [2025-10-20T19:12:18.765222] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T19:12:19.409474] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T19:12:19.451346] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-06. [2025-10-20T19:12:19.517124] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T19:12:20.251389] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-06. [2025-10-20T19:12:21.019300] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:12:21.051341] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-07. [2025-10-20T19:12:21.851368] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-07. [2025-10-20T19:12:21.956380] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:12:22.651349] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-07. [2025-10-20T19:12:22.808935] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:12:23.193717] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T19:12:23.451389] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-08. [2025-10-20T19:12:24.251350] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-08. [2025-10-20T19:12:24.437598] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:12:25.051418] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-08. [2025-10-20T19:12:25.182780] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T19:12:25.682227] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T19:12:25.851363] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-09. [2025-10-20T19:12:26.644632] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:12:26.656435] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-09. [2025-10-20T19:12:27.008167] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T19:12:27.451427] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-09. [2025-10-20T19:12:28.111309] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T19:12:28.251388] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-10. [2025-10-20T19:12:29.051460] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-10. [2025-10-20T19:12:29.261874] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T19:12:29.606246] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T19:12:29.611356] [INFO ] [REMOTE ] HLLY: Full fetch complete (46082 records). [2025-10-20T19:12:29.611515] [DEBUG] [VALIDATOR ] HLLY: Validating primary data. [2025-10-20T19:12:29.861218] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-03. [2025-10-20T19:12:30.011413] [DEBUG] [VALIDATOR ] HLLY: All expected trading days are present. [2025-10-20T19:12:30.011525] [DEBUG] [VALIDATOR ] HLLY: Primary data validation passed. [2025-10-20T19:12:30.012249] [INFO ] [MAIN ] HLLY: Fetched primary data is valid. [2025-10-20T19:12:30.012310] [DEBUG] [LOCAL ] HLLY: Saving primary data. [2025-10-20T19:12:30.013268] [DEBUG] [LOCAL ] HLLY: Trimmed 46082 records to 33339 (keeping 200 days of data) [2025-10-20T19:12:30.190729] [DEBUG] [LOCAL ] HLLY: Primary data saved (33339 records). [2025-10-20T19:12:30.190858] [INFO ] [MAIN ] HLLY: Full history saved. [2025-10-20T19:12:30.190934] [INFO ] [MAIN ] Update Progress: 1028/2367 symbols completed. HLLY took 28.0s. Estimated remaining: 2h 52m 30s. Rate: 7.8 symbols/min [2025-10-20T19:12:30.362981] [INFO ] [MAIN ] HMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:30.405978] [DEBUG] [LOCAL ] HMC: Loaded 47137 existing records from primary storage. [2025-10-20T19:12:30.406304] [INFO ] [MAIN ] HMC: Loaded 47137 existing records. [2025-10-20T19:12:30.583681] [WARN ] [MAIN ] HMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:30.584014] [INFO ] [MAIN ] HMC: Wiping existing primary data before full refresh. [2025-10-20T19:12:30.584191] [INFO ] [LOCAL ] HMC: Starting deletion of primary data. [2025-10-20T19:12:30.584523] [DEBUG] [LOCAL ] HMC: Deleted primary data file. [2025-10-20T19:12:30.584582] [INFO ] [MAIN ] HMC: Starting full primary refresh. [2025-10-20T19:12:30.657775] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-10. [2025-10-20T19:12:30.852456] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T19:12:30.898557] [INFO ] [REMOTE ] HLX: Full fetch complete (69053 records). [2025-10-20T19:12:30.904299] [DEBUG] [VALIDATOR ] HLX: Validating primary data. [2025-10-20T19:12:31.099849] [INFO ] [REMOTE ] HMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:31.438432] [DEBUG] [VALIDATOR ] HLX: All expected trading days are present. [2025-10-20T19:12:31.438533] [DEBUG] [VALIDATOR ] HLX: Primary data validation passed. [2025-10-20T19:12:31.439554] [INFO ] [MAIN ] HLX: Fetched primary data is valid. [2025-10-20T19:12:31.439625] [DEBUG] [LOCAL ] HLX: Saving primary data. [2025-10-20T19:12:31.441127] [DEBUG] [LOCAL ] HLX: Trimmed 69053 records to 47458 (keeping 200 days of data) [2025-10-20T19:12:31.485760] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-01. [2025-10-20T19:12:31.735329] [DEBUG] [LOCAL ] HLX: Primary data saved (47458 records). [2025-10-20T19:12:31.735518] [INFO ] [MAIN ] HLX: Full history saved. [2025-10-20T19:12:31.736186] [INFO ] [MAIN ] Update Progress: 1029/2367 symbols completed. HLX took 24.4s. Estimated remaining: 2h 52m 14s. Rate: 7.8 symbols/min [2025-10-20T19:12:31.987260] [INFO ] [MAIN ] HMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:31.985627] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-03 (2.12 seconds). [2025-10-20T19:12:32.022220] [DEBUG] [LOCAL ] HMN: Loaded 23749 existing records from primary storage. [2025-10-20T19:12:32.022496] [INFO ] [MAIN ] HMN: Loaded 23749 existing records. [2025-10-20T19:12:32.196255] [WARN ] [MAIN ] HMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:32.196481] [INFO ] [MAIN ] HMN: Wiping existing primary data before full refresh. [2025-10-20T19:12:32.196527] [INFO ] [LOCAL ] HMN: Starting deletion of primary data. [2025-10-20T19:12:32.196787] [DEBUG] [LOCAL ] HMN: Deleted primary data file. [2025-10-20T19:12:32.196844] [INFO ] [MAIN ] HMN: Starting full primary refresh. [2025-10-20T19:12:32.258967] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-04. [2025-10-20T19:12:32.600293] [INFO ] [REMOTE ] HMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:32.601157] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:12:32.610039] [INFO ] [REMOTE ] HLN: Full fetch complete (77035 records). [2025-10-20T19:12:32.610192] [DEBUG] [VALIDATOR ] HLN: Validating primary data. [2025-10-20T19:12:33.012343] [DEBUG] [VALIDATOR ] HLN: All expected trading days are present. [2025-10-20T19:12:33.012477] [DEBUG] [VALIDATOR ] HLN: Primary data validation passed. [2025-10-20T19:12:33.013438] [INFO ] [MAIN ] HLN: Fetched primary data is valid. [2025-10-20T19:12:33.013585] [DEBUG] [LOCAL ] HLN: Saving primary data. [2025-10-20T19:12:33.015302] [DEBUG] [LOCAL ] HLN: Trimmed 77035 records to 53136 (keeping 200 days of data) [2025-10-20T19:12:33.066739] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-01. [2025-10-20T19:12:33.165198] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T19:12:33.365795] [DEBUG] [LOCAL ] HLN: Primary data saved (53136 records). [2025-10-20T19:12:33.365923] [INFO ] [MAIN ] HLN: Full history saved. [2025-10-20T19:12:33.366000] [INFO ] [MAIN ] Update Progress: 1030/2367 symbols completed. HLN took 29.1s. Estimated remaining: 2h 51m 58s. Rate: 7.8 symbols/min [2025-10-20T19:12:33.531017] [INFO ] [MAIN ] HMY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:33.580127] [DEBUG] [LOCAL ] HMY: Loaded 53946 existing records from primary storage. [2025-10-20T19:12:33.580432] [INFO ] [MAIN ] HMY: Loaded 53946 existing records. [2025-10-20T19:12:33.753088] [WARN ] [MAIN ] HMY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:33.753302] [INFO ] [MAIN ] HMY: Wiping existing primary data before full refresh. [2025-10-20T19:12:33.753347] [INFO ] [LOCAL ] HMY: Starting deletion of primary data. [2025-10-20T19:12:33.753742] [DEBUG] [LOCAL ] HMY: Deleted primary data file. [2025-10-20T19:12:33.753797] [INFO ] [MAIN ] HMY: Starting full primary refresh. [2025-10-20T19:12:33.853119] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-02. [2025-10-20T19:12:34.114243] [INFO ] [REMOTE ] HMY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:34.245727] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T19:12:34.267629] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T19:12:34.651346] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-05. [2025-10-20T19:12:35.451357] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-01. [2025-10-20T19:12:35.593595] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:12:36.084822] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T19:12:36.251383] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-02. [2025-10-20T19:12:37.051382] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-06. [2025-10-20T19:12:37.277223] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:12:37.453690] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T19:12:37.851425] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-03. [2025-10-20T19:12:38.651430] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-03. [2025-10-20T19:12:38.723917] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:12:38.886729] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T19:12:39.451377] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-04. [2025-10-20T19:12:40.202328] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:12:40.251344] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-07. [2025-10-20T19:12:40.653125] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:12:41.051370] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-05. [2025-10-20T19:12:41.851389] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-04. [2025-10-20T19:12:42.080890] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T19:12:42.109923] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T19:12:42.651400] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-08. [2025-10-20T19:12:43.451365] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-06. [2025-10-20T19:12:43.736133] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T19:12:44.061725] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T19:12:44.251366] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-09. [2025-10-20T19:12:44.580488] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T19:12:45.051457] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-05. [2025-10-20T19:12:45.852875] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-02. [2025-10-20T19:12:45.855514] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T19:12:46.651348] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-07. [2025-10-20T19:12:46.749815] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:12:47.451355] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-06. [2025-10-20T19:12:47.748904] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T19:12:47.935908] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T19:12:48.251438] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-10. [2025-10-20T19:12:49.068530] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-08. [2025-10-20T19:12:49.099159] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:12:49.141813] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T19:12:49.150024] [INFO ] [REMOTE ] HLT: Full fetch complete (73617 records). [2025-10-20T19:12:49.150198] [DEBUG] [VALIDATOR ] HLT: Validating primary data. [2025-10-20T19:12:49.543106] [DEBUG] [VALIDATOR ] HLT: All expected trading days are present. [2025-10-20T19:12:49.543197] [DEBUG] [VALIDATOR ] HLT: Primary data validation passed. [2025-10-20T19:12:49.544125] [INFO ] [MAIN ] HLT: Fetched primary data is valid. [2025-10-20T19:12:49.544193] [DEBUG] [LOCAL ] HLT: Saving primary data. [2025-10-20T19:12:49.545713] [DEBUG] [LOCAL ] HLT: Trimmed 73617 records to 50463 (keeping 200 days of data) [2025-10-20T19:12:49.858961] [DEBUG] [LOCAL ] HLT: Primary data saved (50463 records). [2025-10-20T19:12:49.859119] [INFO ] [MAIN ] HLT: Full history saved. [2025-10-20T19:12:49.855031] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-03. [2025-10-20T19:12:49.859236] [INFO ] [MAIN ] Update Progress: 1031/2367 symbols completed. HLT took 44.3s. Estimated remaining: 2h 52m 2s. Rate: 7.8 symbols/min [2025-10-20T19:12:50.030145] [INFO ] [MAIN ] HNGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:50.030380] [DEBUG] [LOCAL ] HNGE: No existing data file found in primary storage. [2025-10-20T19:12:50.030424] [WARN ] [MAIN ] HNGE: No existing data found, performing full refresh instead. [2025-10-20T19:12:50.030528] [INFO ] [MAIN ] HNGE: Wiping existing primary data before full refresh. [2025-10-20T19:12:50.030569] [INFO ] [LOCAL ] HNGE: Starting deletion of primary data. [2025-10-20T19:12:50.030609] [DEBUG] [LOCAL ] HNGE: No primary data file found to delete. [2025-10-20T19:12:50.030682] [INFO ] [MAIN ] HNGE: Starting full primary refresh. [2025-10-20T19:12:50.322542] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T19:12:50.418760] [INFO ] [REMOTE ] HNGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:50.651394] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-09. [2025-10-20T19:12:51.451358] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-01. [2025-10-20T19:12:51.642050] [DEBUG] [REMOTE ] HNGE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:12:51.642416] [WARN ] [REMOTE ] HNGE: No data returned for month 2025-01 [2025-10-20T19:12:51.642517] [FATAL] [REMOTE ] HNGE: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:12:51.642568] [FATAL] [MAIN ] HNGE: Failed to fetch primary data. [2025-10-20T19:12:51.642632] [WARN ] [MAIN ] HNGE: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:12:51.642725] [INFO ] [LOCAL ] HNGE: Starting deletion of primary data. [2025-10-20T19:12:51.642788] [DEBUG] [LOCAL ] HNGE: No primary data file found to delete. [2025-10-20T19:12:51.642823] [INFO ] [LOCAL ] HNGE: Starting deletion of derivative data. [2025-10-20T19:12:51.642862] [DEBUG] [LOCAL ] HNGE: No derivative data file found to delete. [2025-10-20T19:12:51.642907] [INFO ] [MAIN ] Update Progress: 1032/2367 symbols completed. HNGE took 1.8s. Estimated remaining: 2h 51m 46s. Rate: 7.8 symbols/min [2025-10-20T19:12:51.764830] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T19:12:51.846124] [INFO ] [MAIN ] HNI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:51.875761] [DEBUG] [LOCAL ] HNI: Loaded 30085 existing records from primary storage. [2025-10-20T19:12:51.876037] [INFO ] [MAIN ] HNI: Loaded 30085 existing records. [2025-10-20T19:12:52.055109] [WARN ] [MAIN ] HNI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:52.055415] [INFO ] [MAIN ] HNI: Wiping existing primary data before full refresh. [2025-10-20T19:12:52.055473] [INFO ] [LOCAL ] HNI: Starting deletion of primary data. [2025-10-20T19:12:52.055874] [DEBUG] [LOCAL ] HNI: Deleted primary data file. [2025-10-20T19:12:52.066955] [INFO ] [MAIN ] HNI: Starting full primary refresh. [2025-10-20T19:12:52.179490] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-03 (2.32 seconds). [2025-10-20T19:12:52.258789] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-04. [2025-10-20T19:12:52.466394] [INFO ] [REMOTE ] HNI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:53.051350] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-01. [2025-10-20T19:12:53.851358] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-10. [2025-10-20T19:12:54.216429] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T19:12:54.411952] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T19:12:54.654776] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-05. [2025-10-20T19:12:54.657625] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:12:54.660856] [INFO ] [REMOTE ] HMN: Full fetch complete (31583 records). [2025-10-20T19:12:54.660929] [DEBUG] [VALIDATOR ] HMN: Validating primary data. [2025-10-20T19:12:55.030255] [DEBUG] [VALIDATOR ] HMN: All expected trading days are present. [2025-10-20T19:12:55.030344] [DEBUG] [VALIDATOR ] HMN: Primary data validation passed. [2025-10-20T19:12:55.030944] [INFO ] [MAIN ] HMN: Fetched primary data is valid. [2025-10-20T19:12:55.031027] [DEBUG] [LOCAL ] HMN: Saving primary data. [2025-10-20T19:12:55.031884] [DEBUG] [LOCAL ] HMN: Trimmed 31583 records to 23459 (keeping 200 days of data) [2025-10-20T19:12:55.167333] [DEBUG] [LOCAL ] HMN: Primary data saved (23459 records). [2025-10-20T19:12:55.167449] [INFO ] [MAIN ] HMN: Full history saved. [2025-10-20T19:12:55.167521] [INFO ] [MAIN ] Update Progress: 1033/2367 symbols completed. HMN took 23.4s. Estimated remaining: 2h 51m 33s. Rate: 7.8 symbols/min [2025-10-20T19:12:55.336689] [INFO ] [MAIN ] HOG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:55.383959] [DEBUG] [LOCAL ] HOG: Loaded 51493 existing records from primary storage. [2025-10-20T19:12:55.384292] [INFO ] [MAIN ] HOG: Loaded 51493 existing records. [2025-10-20T19:12:55.457132] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-07. [2025-10-20T19:12:55.560305] [WARN ] [MAIN ] HOG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:55.560498] [INFO ] [MAIN ] HOG: Wiping existing primary data before full refresh. [2025-10-20T19:12:55.560543] [INFO ] [LOCAL ] HOG: Starting deletion of primary data. [2025-10-20T19:12:55.560782] [DEBUG] [LOCAL ] HOG: Deleted primary data file. [2025-10-20T19:12:55.560847] [INFO ] [MAIN ] HOG: Starting full primary refresh. [2025-10-20T19:12:55.902359] [INFO ] [REMOTE ] HOG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:56.251346] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-01. [2025-10-20T19:12:56.745472] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-05 (2.09 seconds). [2025-10-20T19:12:57.051442] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-06. [2025-10-20T19:12:57.216797] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:12:57.847200] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:12:57.856676] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-08. [2025-10-20T19:12:58.651409] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-02. [2025-10-20T19:12:58.817219] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T19:12:59.334317] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:12:59.451394] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-09. [2025-10-20T19:12:59.948885] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:13:00.251363] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-03. [2025-10-20T19:13:01.051391] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-07. [2025-10-20T19:13:01.479741] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T19:13:01.851428] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-10. [2025-10-20T19:13:02.286317] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T19:13:02.563859] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T19:13:02.651429] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-08. [2025-10-20T19:13:03.451353] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-04. [2025-10-20T19:13:03.685007] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T19:13:03.691814] [INFO ] [REMOTE ] HMC: Full fetch complete (68739 records). [2025-10-20T19:13:03.691959] [DEBUG] [VALIDATOR ] HMC: Validating primary data. [2025-10-20T19:13:04.077313] [DEBUG] [VALIDATOR ] HMC: All expected trading days are present. [2025-10-20T19:13:04.077395] [DEBUG] [VALIDATOR ] HMC: Primary data validation passed. [2025-10-20T19:13:04.078288] [INFO ] [MAIN ] HMC: Fetched primary data is valid. [2025-10-20T19:13:04.078387] [DEBUG] [LOCAL ] HMC: Saving primary data. [2025-10-20T19:13:04.079722] [DEBUG] [LOCAL ] HMC: Trimmed 68739 records to 46338 (keeping 200 days of data) [2025-10-20T19:13:04.255184] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-02. [2025-10-20T19:13:04.354143] [DEBUG] [LOCAL ] HMC: Primary data saved (46338 records). [2025-10-20T19:13:04.354297] [INFO ] [MAIN ] HMC: Full history saved. [2025-10-20T19:13:04.355047] [INFO ] [MAIN ] Update Progress: 1034/2367 symbols completed. HMC took 34.2s. Estimated remaining: 2h 51m 27s. Rate: 7.8 symbols/min [2025-10-20T19:13:04.533281] [INFO ] [MAIN ] HOMB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:04.577031] [DEBUG] [LOCAL ] HOMB: Loaded 44387 existing records from primary storage. [2025-10-20T19:13:04.577323] [INFO ] [MAIN ] HOMB: Loaded 44387 existing records. [2025-10-20T19:13:04.762932] [WARN ] [MAIN ] HOMB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:04.763192] [INFO ] [MAIN ] HOMB: Wiping existing primary data before full refresh. [2025-10-20T19:13:04.763247] [INFO ] [LOCAL ] HOMB: Starting deletion of primary data. [2025-10-20T19:13:04.763452] [DEBUG] [LOCAL ] HOMB: Deleted primary data file. [2025-10-20T19:13:04.763509] [INFO ] [MAIN ] HOMB: Starting full primary refresh. [2025-10-20T19:13:04.902480] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-10-20T19:13:05.059917] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-09. [2025-10-20T19:13:05.188676] [INFO ] [REMOTE ] HOMB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:05.220078] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:13:05.523711] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:13:05.851431] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-05. [2025-10-20T19:13:06.651427] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-03. [2025-10-20T19:13:07.241808] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-10-20T19:13:07.451400] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-01. [2025-10-20T19:13:07.671546] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T19:13:07.929208] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T19:13:08.251433] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-10. [2025-10-20T19:13:09.051416] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-04. [2025-10-20T19:13:09.096813] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T19:13:09.851422] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-02. [2025-10-20T19:13:10.214770] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T19:13:10.507611] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-10 (2.26 seconds). [2025-10-20T19:13:10.516132] [INFO ] [REMOTE ] HMY: Full fetch complete (77086 records). [2025-10-20T19:13:10.516300] [DEBUG] [VALIDATOR ] HMY: Validating primary data. [2025-10-20T19:13:10.657011] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-05. [2025-10-20T19:13:10.914219] [DEBUG] [VALIDATOR ] HMY: All expected trading days are present. [2025-10-20T19:13:10.914399] [DEBUG] [VALIDATOR ] HMY: Primary data validation passed. [2025-10-20T19:13:10.915387] [INFO ] [MAIN ] HMY: Fetched primary data is valid. [2025-10-20T19:13:10.915448] [DEBUG] [LOCAL ] HMY: Saving primary data. [2025-10-20T19:13:10.916922] [DEBUG] [LOCAL ] HMY: Trimmed 77086 records to 53166 (keeping 200 days of data) [2025-10-20T19:13:11.164702] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T19:13:11.286940] [DEBUG] [LOCAL ] HMY: Primary data saved (53166 records). [2025-10-20T19:13:11.287087] [INFO ] [MAIN ] HMY: Full history saved. [2025-10-20T19:13:11.287170] [INFO ] [MAIN ] Update Progress: 1035/2367 symbols completed. HMY took 37.9s. Estimated remaining: 2h 51m 19s. Rate: 7.8 symbols/min [2025-10-20T19:13:11.456594] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-03. [2025-10-20T19:13:11.486103] [INFO ] [MAIN ] HOUS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:11.528555] [DEBUG] [LOCAL ] HOUS: Loaded 46109 existing records from primary storage. [2025-10-20T19:13:11.528921] [INFO ] [MAIN ] HOUS: Loaded 46109 existing records. [2025-10-20T19:13:11.725882] [WARN ] [MAIN ] HOUS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:11.726247] [INFO ] [MAIN ] HOUS: Wiping existing primary data before full refresh. [2025-10-20T19:13:11.726332] [INFO ] [LOCAL ] HOUS: Starting deletion of primary data. [2025-10-20T19:13:11.726708] [DEBUG] [LOCAL ] HOUS: Deleted primary data file. [2025-10-20T19:13:11.726776] [INFO ] [MAIN ] HOUS: Starting full primary refresh. [2025-10-20T19:13:12.026471] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:13:12.142909] [INFO ] [REMOTE ] HOUS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:12.251435] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-01. [2025-10-20T19:13:12.993443] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:13:13.051332] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-04. [2025-10-20T19:13:13.624358] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:13:13.851430] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-06. [2025-10-20T19:13:14.362518] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T19:13:14.651346] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-05. [2025-10-20T19:13:15.002118] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T19:13:15.451358] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-07. [2025-10-20T19:13:16.123915] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T19:13:16.251342] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-06. [2025-10-20T19:13:16.564976] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T19:13:17.051366] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-08. [2025-10-20T19:13:17.712329] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:13:17.851451] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-06. [2025-10-20T19:13:18.553068] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:13:18.651364] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-09. [2025-10-20T19:13:19.451432] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-07. [2025-10-20T19:13:19.538607] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T19:13:20.158072] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T19:13:20.251364] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-07. [2025-10-20T19:13:21.008336] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T19:13:21.051333] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-08. [2025-10-20T19:13:21.851381] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-10. [2025-10-20T19:13:22.016566] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:13:22.618889] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:13:22.651341] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-09. [2025-10-20T19:13:23.385543] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T19:13:23.390706] [INFO ] [REMOTE ] HNI: Full fetch complete (42110 records). [2025-10-20T19:13:23.390856] [DEBUG] [VALIDATOR ] HNI: Validating primary data. [2025-10-20T19:13:23.456829] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-08. [2025-10-20T19:13:23.766389] [DEBUG] [VALIDATOR ] HNI: All expected trading days are present. [2025-10-20T19:13:23.766461] [DEBUG] [VALIDATOR ] HNI: Primary data validation passed. [2025-10-20T19:13:23.767118] [INFO ] [MAIN ] HNI: Fetched primary data is valid. [2025-10-20T19:13:23.767180] [DEBUG] [LOCAL ] HNI: Saving primary data. [2025-10-20T19:13:23.768036] [DEBUG] [LOCAL ] HNI: Trimmed 42110 records to 29783 (keeping 200 days of data) [2025-10-20T19:13:23.945411] [DEBUG] [LOCAL ] HNI: Primary data saved (29783 records). [2025-10-20T19:13:23.945610] [INFO ] [MAIN ] HNI: Full history saved. [2025-10-20T19:13:23.945812] [INFO ] [MAIN ] Update Progress: 1036/2367 symbols completed. HNI took 32.3s. Estimated remaining: 2h 51m 17s. Rate: 7.8 symbols/min [2025-10-20T19:13:24.114672] [INFO ] [MAIN ] HOV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:24.130453] [DEBUG] [LOCAL ] HOV: Loaded 14171 existing records from primary storage. [2025-10-20T19:13:24.130696] [INFO ] [MAIN ] HOV: Loaded 14171 existing records. [2025-10-20T19:13:24.251615] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-02. [2025-10-20T19:13:24.316966] [WARN ] [MAIN ] HOV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:24.341612] [INFO ] [MAIN ] HOV: Wiping existing primary data before full refresh. [2025-10-20T19:13:24.341778] [INFO ] [LOCAL ] HOV: Starting deletion of primary data. [2025-10-20T19:13:24.343194] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T19:13:24.342305] [DEBUG] [LOCAL ] HOV: Deleted primary data file. [2025-10-20T19:13:24.346516] [INFO ] [MAIN ] HOV: Starting full primary refresh. [2025-10-20T19:13:24.702074] [INFO ] [REMOTE ] HOV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:25.051374] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-10. [2025-10-20T19:13:25.197575] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T19:13:25.709420] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:13:25.851335] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-03. [2025-10-20T19:13:26.437053] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T19:13:26.443915] [INFO ] [REMOTE ] HOMB: Full fetch complete (64431 records). [2025-10-20T19:13:26.444032] [DEBUG] [VALIDATOR ] HOMB: Validating primary data. [2025-10-20T19:13:26.663351] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-01. [2025-10-20T19:13:26.826745] [DEBUG] [VALIDATOR ] HOMB: All expected trading days are present. [2025-10-20T19:13:26.826823] [DEBUG] [VALIDATOR ] HOMB: Primary data validation passed. [2025-10-20T19:13:26.827671] [INFO ] [MAIN ] HOMB: Fetched primary data is valid. [2025-10-20T19:13:26.827732] [DEBUG] [LOCAL ] HOMB: Saving primary data. [2025-10-20T19:13:26.829147] [DEBUG] [LOCAL ] HOMB: Trimmed 64431 records to 43754 (keeping 200 days of data) [2025-10-20T19:13:27.083730] [DEBUG] [LOCAL ] HOMB: Primary data saved (43754 records). [2025-10-20T19:13:27.083843] [INFO ] [MAIN ] HOMB: Full history saved. [2025-10-20T19:13:27.083914] [INFO ] [MAIN ] Update Progress: 1037/2367 symbols completed. HOMB took 22.7s. Estimated remaining: 2h 51m 4s. Rate: 7.8 symbols/min [2025-10-20T19:13:27.291762] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:13:27.308272] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T19:13:27.451331] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-02. [2025-10-20T19:13:28.251349] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-09. [2025-10-20T19:13:28.339348] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T19:13:29.051349] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-03. [2025-10-20T19:13:29.851446] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-04. [2025-10-20T19:13:29.868021] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T19:13:29.929099] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T19:13:30.651432] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-10. [2025-10-20T19:13:31.374370] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:13:31.451401] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-05. [2025-10-20T19:13:32.251434] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-04. [2025-10-20T19:13:32.414683] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T19:13:32.423382] [INFO ] [REMOTE ] HOG: Full fetch complete (74308 records). [2025-10-20T19:13:32.423550] [DEBUG] [VALIDATOR ] HOG: Validating primary data. [2025-10-20T19:13:32.850516] [DEBUG] [VALIDATOR ] HOG: All expected trading days are present. [2025-10-20T19:13:32.852532] [DEBUG] [VALIDATOR ] HOG: Primary data validation passed. [2025-10-20T19:13:32.852297] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T19:13:32.853848] [INFO ] [MAIN ] HOG: Fetched primary data is valid. [2025-10-20T19:13:32.863121] [DEBUG] [LOCAL ] HOG: Saving primary data. [2025-10-20T19:13:32.874419] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T19:13:32.874560] [DEBUG] [LOCAL ] HOG: Trimmed 74308 records to 50715 (keeping 200 days of data) [2025-10-20T19:13:33.059528] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-06. [2025-10-20T19:13:33.187582] [DEBUG] [LOCAL ] HOG: Primary data saved (50715 records). [2025-10-20T19:13:33.187757] [INFO ] [MAIN ] HOG: Full history saved. [2025-10-20T19:13:33.188184] [INFO ] [MAIN ] Update Progress: 1038/2367 symbols completed. HOG took 38.0s. Estimated remaining: 2h 50m 54s. Rate: 7.8 symbols/min [2025-10-20T19:13:33.851428] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-05. [2025-10-20T19:13:34.637483] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:13:34.651324] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-07. [2025-10-20T19:13:34.666580] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T19:13:35.451385] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-06. [2025-10-20T19:13:36.142063] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T19:13:36.251444] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-08. [2025-10-20T19:13:36.493037] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T19:13:37.051450] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-07. [2025-10-20T19:13:37.777041] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T19:13:37.851331] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-09. [2025-10-20T19:13:38.018821] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T19:13:38.651456] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-08. [2025-10-20T19:13:39.576877] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:13:39.596421] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-10. [2025-10-20T19:13:39.597533] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T19:13:40.251363] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-09. [2025-10-20T19:13:41.306437] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:13:41.311879] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-10. [2025-10-20T19:13:41.548775] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T19:13:41.556686] [INFO ] [REMOTE ] HOUS: Full fetch complete (64599 records). [2025-10-20T19:13:41.556844] [DEBUG] [VALIDATOR ] HOUS: Validating primary data. [2025-10-20T19:13:41.953815] [DEBUG] [VALIDATOR ] HOUS: All expected trading days are present. [2025-10-20T19:13:41.953892] [DEBUG] [VALIDATOR ] HOUS: Primary data validation passed. [2025-10-20T19:13:41.954699] [INFO ] [MAIN ] HOUS: Fetched primary data is valid. [2025-10-20T19:13:41.954761] [DEBUG] [LOCAL ] HOUS: Saving primary data. [2025-10-20T19:13:41.955955] [DEBUG] [LOCAL ] HOUS: Trimmed 64599 records to 45656 (keeping 200 days of data) [2025-10-20T19:13:42.207522] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T19:13:42.207608] [DEBUG] [LOCAL ] HOUS: Primary data saved (45656 records). [2025-10-20T19:13:42.210166] [INFO ] [MAIN ] HOUS: Full history saved. [2025-10-20T19:13:42.210287] [INFO ] [REMOTE ] HOV: Full fetch complete (19014 records). [2025-10-20T19:13:42.210546] [DEBUG] [VALIDATOR ] HOV: Validating primary data. [2025-10-20T19:13:42.210457] [INFO ] [MAIN ] Update Progress: 1039/2367 symbols completed. HOUS took 30.9s. Estimated remaining: 2h 50m 48s. Rate: 7.8 symbols/min [2025-10-20T19:13:42.579730] [DEBUG] [VALIDATOR ] HOV: All expected trading days are present. [2025-10-20T19:13:42.579824] [DEBUG] [VALIDATOR ] HOV: Primary data validation passed. [2025-10-20T19:13:42.580260] [INFO ] [MAIN ] HOV: Fetched primary data is valid. [2025-10-20T19:13:42.580299] [DEBUG] [LOCAL ] HOV: Saving primary data. [2025-10-20T19:13:42.580938] [DEBUG] [LOCAL ] HOV: Trimmed 19014 records to 14023 (keeping 200 days of data) [2025-10-20T19:13:42.665980] [DEBUG] [LOCAL ] HOV: Primary data saved (14023 records). [2025-10-20T19:13:42.666134] [INFO ] [MAIN ] HOV: Full history saved. [2025-10-20T19:13:42.666216] [INFO ] [MAIN ] Update Progress: 1040/2367 symbols completed. HOV took 18.7s. Estimated remaining: 2h 50m 31s. Rate: 7.8 symbols/min [2025-10-20T19:13:43.351976] [INFO ] [MAIN ] HPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:43.352846] [INFO ] [MAIN ] HPE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:43.398167] [INFO ] [MAIN ] HPF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:43.403188] [INFO ] [MAIN ] HP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:43.414872] [DEBUG] [LOCAL ] HPF: Loaded 7566 existing records from primary storage. [2025-10-20T19:13:43.415514] [INFO ] [MAIN ] HPF: Loaded 7566 existing records. [2025-10-20T19:13:43.473932] [DEBUG] [LOCAL ] HPI: Loaded 10785 existing records from primary storage. [2025-10-20T19:13:43.503798] [INFO ] [MAIN ] HPI: Loaded 10785 existing records. [2025-10-20T19:13:43.767063] [WARN ] [MAIN ] HPF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:43.803165] [INFO ] [MAIN ] HPF: Wiping existing primary data before full refresh. [2025-10-20T19:13:43.803503] [INFO ] [LOCAL ] HPF: Starting deletion of primary data. [2025-10-20T19:13:43.809796] [DEBUG] [LOCAL ] HPE: Loaded 54477 existing records from primary storage. [2025-10-20T19:13:43.817032] [DEBUG] [LOCAL ] HPF: Deleted primary data file. [2025-10-20T19:13:43.827501] [INFO ] [MAIN ] HPF: Starting full primary refresh. [2025-10-20T19:13:43.816911] [DEBUG] [LOCAL ] HP: Loaded 52245 existing records from primary storage. [2025-10-20T19:13:43.833800] [INFO ] [MAIN ] HP: Loaded 52245 existing records. [2025-10-20T19:13:43.827370] [INFO ] [MAIN ] HPE: Loaded 54477 existing records. [2025-10-20T19:13:44.029186] [WARN ] [MAIN ] HPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:44.164128] [INFO ] [MAIN ] HPI: Wiping existing primary data before full refresh. [2025-10-20T19:13:44.199963] [INFO ] [LOCAL ] HPI: Starting deletion of primary data. [2025-10-20T19:13:44.320487] [DEBUG] [LOCAL ] HPI: Deleted primary data file. [2025-10-20T19:13:44.341151] [WARN ] [MAIN ] HPE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:44.360853] [INFO ] [MAIN ] HPI: Starting full primary refresh. [2025-10-20T19:13:44.371293] [INFO ] [MAIN ] HPE: Wiping existing primary data before full refresh. [2025-10-20T19:13:44.411077] [INFO ] [LOCAL ] HPE: Starting deletion of primary data. [2025-10-20T19:13:44.435028] [DEBUG] [LOCAL ] HPE: Deleted primary data file. [2025-10-20T19:13:44.443579] [INFO ] [MAIN ] HPE: Starting full primary refresh. [2025-10-20T19:13:44.675755] [WARN ] [MAIN ] HP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:44.860181] [INFO ] [MAIN ] HP: Wiping existing primary data before full refresh. [2025-10-20T19:13:44.934421] [INFO ] [LOCAL ] HP: Starting deletion of primary data. [2025-10-20T19:13:45.064804] [DEBUG] [LOCAL ] HP: Deleted primary data file. [2025-10-20T19:13:45.098828] [INFO ] [MAIN ] HP: Starting full primary refresh. [2025-10-20T19:13:45.190434] [INFO ] [REMOTE ] HPF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:45.235867] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-01. [2025-10-20T19:13:45.650893] [INFO ] [REMOTE ] HPE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:45.696900] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-01. [2025-10-20T19:13:45.823016] [INFO ] [REMOTE ] HPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:45.856322] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-01. [2025-10-20T19:13:45.960391] [INFO ] [REMOTE ] HP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:45.960731] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-01. [2025-10-20T19:13:46.143455] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T19:13:46.145789] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-02. [2025-10-20T19:13:46.639319] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T19:13:46.641477] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-02. [2025-10-20T19:13:46.799539] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T19:13:46.801458] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-03. [2025-10-20T19:13:47.029114] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T19:13:47.410852] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T19:13:47.451365] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-03. [2025-10-20T19:13:47.545467] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:13:47.700624] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T19:13:48.217465] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T19:13:48.251358] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-02. [2025-10-20T19:13:49.051436] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-02. [2025-10-20T19:13:49.851415] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-04. [2025-10-20T19:13:50.225295] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-10-20T19:13:50.344245] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:13:50.622494] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T19:13:50.651345] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-03. [2025-10-20T19:13:51.451409] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-03. [2025-10-20T19:13:52.251349] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-05. [2025-10-20T19:13:52.644932] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T19:13:52.956140] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T19:13:53.051345] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-04. [2025-10-20T19:13:53.151305] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T19:13:53.851412] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-06. [2025-10-20T19:13:54.638819] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T19:13:54.651321] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-07. [2025-10-20T19:13:54.903905] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:13:55.451389] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-04. [2025-10-20T19:13:55.499898] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T19:13:56.178721] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-10-20T19:13:56.251400] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-08. [2025-10-20T19:13:56.861177] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:13:57.051356] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-04. [2025-10-20T19:13:57.851385] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-09. [2025-10-20T19:13:58.651442] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-05. [2025-10-20T19:13:58.795536] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T19:13:58.850769] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:13:59.380631] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T19:13:59.451377] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-05. [2025-10-20T19:14:00.251368] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-05. [2025-10-20T19:14:01.051380] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-06. [2025-10-20T19:14:01.619868] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T19:14:01.836804] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T19:14:01.851393] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-07. [2025-10-20T19:14:02.308762] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-05 (2.06 seconds). [2025-10-20T19:14:02.651349] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-06. [2025-10-20T19:14:02.704226] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T19:14:03.451398] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-10. [2025-10-20T19:14:04.251411] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-08. [2025-10-20T19:14:04.460763] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:14:04.462872] [INFO ] [REMOTE ] HPI: Full fetch complete (13918 records). [2025-10-20T19:14:04.462940] [DEBUG] [VALIDATOR ] HPI: Validating primary data. [2025-10-20T19:14:04.860169] [DEBUG] [VALIDATOR ] HPI: All expected trading days are present. [2025-10-20T19:14:04.860324] [DEBUG] [VALIDATOR ] HPI: Primary data validation passed. [2025-10-20T19:14:04.860844] [INFO ] [MAIN ] HPI: Fetched primary data is valid. [2025-10-20T19:14:04.860935] [DEBUG] [LOCAL ] HPI: Saving primary data. [2025-10-20T19:14:04.863322] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-06 (2.21 seconds). [2025-10-20T19:14:04.873946] [DEBUG] [LOCAL ] HPI: Trimmed 13918 records to 10790 (keeping 200 days of data) [2025-10-20T19:14:04.939018] [DEBUG] [LOCAL ] HPI: Primary data saved (10790 records). [2025-10-20T19:14:04.939140] [INFO ] [MAIN ] HPI: Full history saved. [2025-10-20T19:14:04.939217] [INFO ] [MAIN ] Update Progress: 1041/2367 symbols completed. HPI took 22.3s. Estimated remaining: 2h 50m 42s. Rate: 7.8 symbols/min [2025-10-20T19:14:05.044244] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T19:14:05.057987] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-06. [2025-10-20T19:14:05.123359] [INFO ] [MAIN ] HPP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:05.170538] [DEBUG] [LOCAL ] HPP: Loaded 51937 existing records from primary storage. [2025-10-20T19:14:05.170923] [INFO ] [MAIN ] HPP: Loaded 51937 existing records. [2025-10-20T19:14:05.345934] [WARN ] [MAIN ] HPP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:05.346149] [INFO ] [MAIN ] HPP: Wiping existing primary data before full refresh. [2025-10-20T19:14:05.346196] [INFO ] [LOCAL ] HPP: Starting deletion of primary data. [2025-10-20T19:14:05.346471] [DEBUG] [LOCAL ] HPP: Deleted primary data file. [2025-10-20T19:14:05.346527] [INFO ] [MAIN ] HPP: Starting full primary refresh. [2025-10-20T19:14:05.700000] [INFO ] [REMOTE ] HPP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:05.851338] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-01. [2025-10-20T19:14:06.651413] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-09. [2025-10-20T19:14:06.826764] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T19:14:07.443947] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T19:14:07.454169] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-07. [2025-10-20T19:14:07.477687] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T19:14:08.251426] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-10. [2025-10-20T19:14:08.991773] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T19:14:08.993257] [INFO ] [REMOTE ] HPF: Full fetch complete (10569 records). [2025-10-20T19:14:08.993345] [DEBUG] [VALIDATOR ] HPF: Validating primary data. [2025-10-20T19:14:09.056618] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-02. [2025-10-20T19:14:09.353316] [DEBUG] [VALIDATOR ] HPF: All expected trading days are present. [2025-10-20T19:14:09.353389] [DEBUG] [VALIDATOR ] HPF: Primary data validation passed. [2025-10-20T19:14:09.353630] [INFO ] [MAIN ] HPF: Fetched primary data is valid. [2025-10-20T19:14:09.353703] [DEBUG] [LOCAL ] HPF: Saving primary data. [2025-10-20T19:14:09.354121] [DEBUG] [LOCAL ] HPF: Trimmed 10569 records to 7516 (keeping 200 days of data) [2025-10-20T19:14:09.397525] [DEBUG] [LOCAL ] HPF: Primary data saved (7516 records). [2025-10-20T19:14:09.397615] [INFO ] [MAIN ] HPF: Full history saved. [2025-10-20T19:14:09.397722] [INFO ] [MAIN ] Update Progress: 1042/2367 symbols completed. HPF took 26.7s. Estimated remaining: 2h 50m 30s. Rate: 7.8 symbols/min [2025-10-20T19:14:09.569127] [INFO ] [MAIN ] HPQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:09.617255] [DEBUG] [LOCAL ] HPQ: Loaded 54449 existing records from primary storage. [2025-10-20T19:14:09.617573] [INFO ] [MAIN ] HPQ: Loaded 54449 existing records. [2025-10-20T19:14:09.789930] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-07 (2.34 seconds). [2025-10-20T19:14:09.840147] [WARN ] [MAIN ] HPQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:09.840353] [INFO ] [MAIN ] HPQ: Wiping existing primary data before full refresh. [2025-10-20T19:14:09.840404] [INFO ] [LOCAL ] HPQ: Starting deletion of primary data. [2025-10-20T19:14:09.840616] [DEBUG] [LOCAL ] HPQ: Deleted primary data file. [2025-10-20T19:14:09.840699] [INFO ] [MAIN ] HPQ: Starting full primary refresh. [2025-10-20T19:14:09.861127] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-08. [2025-10-20T19:14:10.198205] [INFO ] [REMOTE ] HPQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:10.651359] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-07. [2025-10-20T19:14:10.868007] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T19:14:11.451417] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-03. [2025-10-20T19:14:11.942474] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T19:14:12.251349] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-01. [2025-10-20T19:14:12.407054] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:14:13.018895] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T19:14:13.051362] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-04. [2025-10-20T19:14:13.851393] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-08. [2025-10-20T19:14:14.026157] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T19:14:14.652155] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-02. [2025-10-20T19:14:14.656457] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:14:15.440261] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:14:15.451348] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-09. [2025-10-20T19:14:16.251355] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-05. [2025-10-20T19:14:16.445470] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:14:17.043676] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:14:17.054971] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-03. [2025-10-20T19:14:17.857095] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-10. [2025-10-20T19:14:17.871624] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T19:14:18.651436] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-06. [2025-10-20T19:14:18.967117] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:14:19.451381] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-04. [2025-10-20T19:14:20.035707] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-10 (2.18 seconds). [2025-10-20T19:14:20.043561] [INFO ] [REMOTE ] HPE: Full fetch complete (77876 records). [2025-10-20T19:14:20.043722] [DEBUG] [VALIDATOR ] HPE: Validating primary data. [2025-10-20T19:14:20.256792] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-09. [2025-10-20T19:14:20.481505] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T19:14:20.512075] [DEBUG] [VALIDATOR ] HPE: All expected trading days are present. [2025-10-20T19:14:20.512400] [DEBUG] [VALIDATOR ] HPE: Primary data validation passed. [2025-10-20T19:14:20.513961] [INFO ] [MAIN ] HPE: Fetched primary data is valid. [2025-10-20T19:14:20.514550] [DEBUG] [LOCAL ] HPE: Saving primary data. [2025-10-20T19:14:20.516929] [DEBUG] [LOCAL ] HPE: Trimmed 77876 records to 53697 (keeping 200 days of data) [2025-10-20T19:14:20.875372] [DEBUG] [LOCAL ] HPE: Primary data saved (53697 records). [2025-10-20T19:14:20.875572] [INFO ] [MAIN ] HPE: Full history saved. [2025-10-20T19:14:20.875760] [INFO ] [MAIN ] Update Progress: 1043/2367 symbols completed. HPE took 38.2s. Estimated remaining: 2h 50m 27s. Rate: 7.8 symbols/min [2025-10-20T19:14:21.056958] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-07. [2025-10-20T19:14:21.066880] [INFO ] [MAIN ] HPS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:21.083056] [DEBUG] [LOCAL ] HPS: Loaded 10447 existing records from primary storage. [2025-10-20T19:14:21.083928] [INFO ] [MAIN ] HPS: Loaded 10447 existing records. [2025-10-20T19:14:21.283824] [WARN ] [MAIN ] HPS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:21.284131] [INFO ] [MAIN ] HPS: Wiping existing primary data before full refresh. [2025-10-20T19:14:21.284184] [INFO ] [LOCAL ] HPS: Starting deletion of primary data. [2025-10-20T19:14:21.284452] [DEBUG] [LOCAL ] HPS: Deleted primary data file. [2025-10-20T19:14:21.284512] [INFO ] [MAIN ] HPS: Starting full primary refresh. [2025-10-20T19:14:21.656909] [INFO ] [REMOTE ] HPS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:21.696041] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-04 (2.24 seconds). [2025-10-20T19:14:21.751704] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:14:21.851368] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-10. [2025-10-20T19:14:22.651377] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-01. [2025-10-20T19:14:22.819902] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:14:23.444823] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:14:23.451318] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-02. [2025-10-20T19:14:23.633216] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T19:14:23.640468] [INFO ] [REMOTE ] HP: Full fetch complete (74058 records). [2025-10-20T19:14:23.640587] [DEBUG] [VALIDATOR ] HP: Validating primary data. [2025-10-20T19:14:24.026235] [DEBUG] [VALIDATOR ] HP: All expected trading days are present. [2025-10-20T19:14:24.026386] [DEBUG] [VALIDATOR ] HP: Primary data validation passed. [2025-10-20T19:14:24.027672] [INFO ] [MAIN ] HP: Fetched primary data is valid. [2025-10-20T19:14:24.027780] [DEBUG] [LOCAL ] HP: Saving primary data. [2025-10-20T19:14:24.029430] [DEBUG] [LOCAL ] HP: Trimmed 74058 records to 51484 (keeping 200 days of data) [2025-10-20T19:14:24.127884] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T19:14:24.253065] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-08. [2025-10-20T19:14:24.353885] [DEBUG] [LOCAL ] HP: Primary data saved (51484 records). [2025-10-20T19:14:24.354006] [INFO ] [MAIN ] HP: Full history saved. [2025-10-20T19:14:24.354080] [INFO ] [MAIN ] Update Progress: 1044/2367 symbols completed. HP took 41.7s. Estimated remaining: 2h 50m 14s. Rate: 7.8 symbols/min [2025-10-20T19:14:24.523867] [INFO ] [MAIN ] HQH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:24.550300] [DEBUG] [LOCAL ] HQH: Loaded 26187 existing records from primary storage. [2025-10-20T19:14:24.550545] [INFO ] [MAIN ] HQH: Loaded 26187 existing records. [2025-10-20T19:14:24.722252] [WARN ] [MAIN ] HQH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:24.722443] [INFO ] [MAIN ] HQH: Wiping existing primary data before full refresh. [2025-10-20T19:14:24.722489] [INFO ] [LOCAL ] HQH: Starting deletion of primary data. [2025-10-20T19:14:24.722696] [DEBUG] [LOCAL ] HQH: Deleted primary data file. [2025-10-20T19:14:24.722752] [INFO ] [MAIN ] HQH: Starting full primary refresh. [2025-10-20T19:14:25.060989] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-05. [2025-10-20T19:14:25.065633] [INFO ] [REMOTE ] HQH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:25.851356] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-01. [2025-10-20T19:14:25.990813] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T19:14:26.651362] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-03. [2025-10-20T19:14:27.061630] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T19:14:27.117857] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T19:14:27.451450] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-06. [2025-10-20T19:14:27.495611] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T19:14:28.251438] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-04. [2025-10-20T19:14:28.999332] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T19:14:29.051341] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-05. [2025-10-20T19:14:29.323312] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T19:14:29.851390] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-09. [2025-10-20T19:14:29.866346] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T19:14:30.651363] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-02. [2025-10-20T19:14:31.451411] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-06. [2025-10-20T19:14:31.649452] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:14:31.754194] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T19:14:32.251432] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-10. [2025-10-20T19:14:32.269476] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T19:14:33.051410] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-07. [2025-10-20T19:14:33.851416] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-03. [2025-10-20T19:14:33.993857] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:14:34.001355] [INFO ] [REMOTE ] HPP: Full fetch complete (73402 records). [2025-10-20T19:14:34.001491] [DEBUG] [VALIDATOR ] HPP: Validating primary data. [2025-10-20T19:14:34.411933] [DEBUG] [VALIDATOR ] HPP: All expected trading days are present. [2025-10-20T19:14:34.412017] [DEBUG] [VALIDATOR ] HPP: Primary data validation passed. [2025-10-20T19:14:34.413018] [INFO ] [MAIN ] HPP: Fetched primary data is valid. [2025-10-20T19:14:34.413087] [DEBUG] [LOCAL ] HPP: Saving primary data. [2025-10-20T19:14:34.414484] [DEBUG] [LOCAL ] HPP: Trimmed 73402 records to 51262 (keeping 200 days of data) [2025-10-20T19:14:34.664686] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-07. [2025-10-20T19:14:34.720634] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:14:34.719906] [DEBUG] [LOCAL ] HPP: Primary data saved (51262 records). [2025-10-20T19:14:34.726149] [INFO ] [MAIN ] HPP: Full history saved. [2025-10-20T19:14:34.726236] [INFO ] [MAIN ] Update Progress: 1045/2367 symbols completed. HPP took 29.8s. Estimated remaining: 2h 50m 10s. Rate: 7.8 symbols/min [2025-10-20T19:14:34.896732] [INFO ] [MAIN ] HQL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:34.914952] [DEBUG] [LOCAL ] HQL: Loaded 17218 existing records from primary storage. [2025-10-20T19:14:34.915182] [INFO ] [MAIN ] HQL: Loaded 17218 existing records. [2025-10-20T19:14:35.098159] [WARN ] [MAIN ] HQL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:35.098669] [INFO ] [MAIN ] HQL: Wiping existing primary data before full refresh. [2025-10-20T19:14:35.098808] [INFO ] [LOCAL ] HQL: Starting deletion of primary data. [2025-10-20T19:14:35.099100] [DEBUG] [LOCAL ] HQL: Deleted primary data file. [2025-10-20T19:14:35.099161] [INFO ] [MAIN ] HQL: Starting full primary refresh. [2025-10-20T19:14:35.396279] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-07 (2.34 seconds). [2025-10-20T19:14:35.452909] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-08. [2025-10-20T19:14:35.515109] [INFO ] [REMOTE ] HQL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:35.654758] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T19:14:36.251376] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-04. [2025-10-20T19:14:37.051548] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-08. [2025-10-20T19:14:37.438286] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T19:14:37.477958] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-08 (2.03 seconds). [2025-10-20T19:14:37.846847] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T19:14:37.851317] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-09. [2025-10-20T19:14:38.383526] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T19:14:38.651384] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-05. [2025-10-20T19:14:39.451343] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-10. [2025-10-20T19:14:39.825276] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T19:14:40.236225] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T19:14:40.238295] [INFO ] [REMOTE ] HPS: Full fetch complete (15123 records). [2025-10-20T19:14:40.238384] [DEBUG] [VALIDATOR ] HPS: Validating primary data. [2025-10-20T19:14:40.256498] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-06. [2025-10-20T19:14:40.618498] [DEBUG] [VALIDATOR ] HPS: All expected trading days are present. [2025-10-20T19:14:40.618575] [DEBUG] [VALIDATOR ] HPS: Primary data validation passed. [2025-10-20T19:14:40.618953] [INFO ] [MAIN ] HPS: Fetched primary data is valid. [2025-10-20T19:14:40.619014] [DEBUG] [LOCAL ] HPS: Saving primary data. [2025-10-20T19:14:40.619529] [DEBUG] [LOCAL ] HPS: Trimmed 15123 records to 10377 (keeping 200 days of data) [2025-10-20T19:14:40.685325] [DEBUG] [LOCAL ] HPS: Primary data saved (10377 records). [2025-10-20T19:14:40.685672] [INFO ] [MAIN ] HPS: Full history saved. [2025-10-20T19:14:40.685769] [INFO ] [MAIN ] Update Progress: 1046/2367 symbols completed. HPS took 19.8s. Estimated remaining: 2h 49m 60s. Rate: 7.8 symbols/min [2025-10-20T19:14:40.862181] [INFO ] [MAIN ] HR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:40.913229] [DEBUG] [LOCAL ] HR: Loaded 52870 existing records from primary storage. [2025-10-20T19:14:40.913557] [INFO ] [MAIN ] HR: Loaded 52870 existing records. [2025-10-20T19:14:41.057748] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-01. [2025-10-20T19:14:41.098771] [WARN ] [MAIN ] HR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:41.099061] [INFO ] [MAIN ] HR: Wiping existing primary data before full refresh. [2025-10-20T19:14:41.099107] [INFO ] [LOCAL ] HR: Starting deletion of primary data. [2025-10-20T19:14:41.099315] [DEBUG] [LOCAL ] HR: Deleted primary data file. [2025-10-20T19:14:41.099369] [INFO ] [MAIN ] HR: Starting full primary refresh. [2025-10-20T19:14:41.454456] [INFO ] [REMOTE ] HR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:41.467693] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T19:14:41.851360] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-01. [2025-10-20T19:14:42.118205] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T19:14:42.651398] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-02. [2025-10-20T19:14:43.451389] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-09. [2025-10-20T19:14:43.494452] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:14:43.541836] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T19:14:44.251383] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-03. [2025-10-20T19:14:45.051347] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-07. [2025-10-20T19:14:45.296588] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:14:45.330774] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T19:14:45.851401] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-02. [2025-10-20T19:14:46.066282] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T19:14:46.651376] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-10. [2025-10-20T19:14:47.164634] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T19:14:47.461618] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-04. [2025-10-20T19:14:47.466930] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T19:14:47.475779] [INFO ] [REMOTE ] HPQ: Full fetch complete (77842 records). [2025-10-20T19:14:47.475976] [DEBUG] [VALIDATOR ] HPQ: Validating primary data. [2025-10-20T19:14:47.925451] [DEBUG] [VALIDATOR ] HPQ: All expected trading days are present. [2025-10-20T19:14:47.925535] [DEBUG] [VALIDATOR ] HPQ: Primary data validation passed. [2025-10-20T19:14:47.926529] [INFO ] [MAIN ] HPQ: Fetched primary data is valid. [2025-10-20T19:14:47.926595] [DEBUG] [LOCAL ] HPQ: Saving primary data. [2025-10-20T19:14:47.928040] [DEBUG] [LOCAL ] HPQ: Trimmed 77842 records to 53669 (keeping 200 days of data) [2025-10-20T19:14:48.254039] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-08. [2025-10-20T19:14:48.253941] [DEBUG] [LOCAL ] HPQ: Primary data saved (53669 records). [2025-10-20T19:14:48.254500] [INFO ] [MAIN ] HPQ: Full history saved. [2025-10-20T19:14:48.255247] [INFO ] [MAIN ] Update Progress: 1047/2367 symbols completed. HPQ took 38.9s. Estimated remaining: 2h 49m 52s. Rate: 7.8 symbols/min [2025-10-20T19:14:48.431704] [INFO ] [MAIN ] HRB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:48.464469] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T19:14:48.486170] [DEBUG] [LOCAL ] HRB: Loaded 49064 existing records from primary storage. [2025-10-20T19:14:48.486467] [INFO ] [MAIN ] HRB: Loaded 49064 existing records. [2025-10-20T19:14:48.661408] [WARN ] [MAIN ] HRB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:48.661685] [INFO ] [MAIN ] HRB: Wiping existing primary data before full refresh. [2025-10-20T19:14:48.661763] [INFO ] [LOCAL ] HRB: Starting deletion of primary data. [2025-10-20T19:14:48.662006] [DEBUG] [LOCAL ] HRB: Deleted primary data file. [2025-10-20T19:14:48.662079] [INFO ] [MAIN ] HRB: Starting full primary refresh. [2025-10-20T19:14:49.008684] [INFO ] [REMOTE ] HRB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:49.051335] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-05. [2025-10-20T19:14:49.448936] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T19:14:49.851401] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-03. [2025-10-20T19:14:50.087234] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T19:14:50.651402] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-06. [2025-10-20T19:14:51.434686] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:14:51.451330] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-04. [2025-10-20T19:14:51.479611] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T19:14:52.251390] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-09. [2025-10-20T19:14:53.051361] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-07. [2025-10-20T19:14:53.250092] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:14:53.461288] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T19:14:53.697151] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T19:14:53.851354] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-10. [2025-10-20T19:14:54.651407] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-08. [2025-10-20T19:14:55.057117] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T19:14:55.061695] [INFO ] [REMOTE ] HQH: Full fetch complete (37134 records). [2025-10-20T19:14:55.061774] [DEBUG] [VALIDATOR ] HQH: Validating primary data. [2025-10-20T19:14:55.433181] [DEBUG] [VALIDATOR ] HQH: All expected trading days are present. [2025-10-20T19:14:55.433367] [DEBUG] [VALIDATOR ] HQH: Primary data validation passed. [2025-10-20T19:14:55.434074] [INFO ] [MAIN ] HQH: Fetched primary data is valid. [2025-10-20T19:14:55.434146] [DEBUG] [LOCAL ] HQH: Saving primary data. [2025-10-20T19:14:55.435177] [DEBUG] [LOCAL ] HQH: Trimmed 37134 records to 25838 (keeping 200 days of data) [2025-10-20T19:14:55.485511] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-01. [2025-10-20T19:14:55.587846] [DEBUG] [LOCAL ] HQH: Primary data saved (25838 records). [2025-10-20T19:14:55.593241] [INFO ] [MAIN ] HQH: Full history saved. [2025-10-20T19:14:55.593370] [INFO ] [MAIN ] Update Progress: 1048/2367 symbols completed. HQH took 31.2s. Estimated remaining: 2h 49m 43s. Rate: 7.8 symbols/min [2025-10-20T19:14:55.641501] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:14:55.773263] [INFO ] [MAIN ] HRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:55.805954] [DEBUG] [LOCAL ] HRI: Loaded 32839 existing records from primary storage. [2025-10-20T19:14:55.806226] [INFO ] [MAIN ] HRI: Loaded 32839 existing records. [2025-10-20T19:14:55.981383] [WARN ] [MAIN ] HRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:55.981601] [INFO ] [MAIN ] HRI: Wiping existing primary data before full refresh. [2025-10-20T19:14:55.981673] [INFO ] [LOCAL ] HRI: Starting deletion of primary data. [2025-10-20T19:14:55.981884] [DEBUG] [LOCAL ] HRI: Deleted primary data file. [2025-10-20T19:14:55.981938] [INFO ] [MAIN ] HRI: Starting full primary refresh. [2025-10-20T19:14:56.256603] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-09. [2025-10-20T19:14:56.352142] [INFO ] [REMOTE ] HRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:56.650232] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:14:57.051371] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-05. [2025-10-20T19:14:57.256996] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:14:57.851389] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-02. [2025-10-20T19:14:58.592285] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T19:14:58.651342] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-06. [2025-10-20T19:14:59.427559] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T19:14:59.451362] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-03. [2025-10-20T19:15:00.251417] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-01. [2025-10-20T19:15:00.393484] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T19:15:01.051434] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-10. [2025-10-20T19:15:01.218703] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T19:15:01.661073] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:15:01.851387] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-07. [2025-10-20T19:15:02.432434] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T19:15:02.436500] [INFO ] [REMOTE ] HQL: Full fetch complete (25348 records). [2025-10-20T19:15:02.436578] [DEBUG] [VALIDATOR ] HQL: Validating primary data. [2025-10-20T19:15:02.660244] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-04. [2025-10-20T19:15:02.798911] [DEBUG] [VALIDATOR ] HQL: All expected trading days are present. [2025-10-20T19:15:02.798989] [DEBUG] [VALIDATOR ] HQL: Primary data validation passed. [2025-10-20T19:15:02.799285] [INFO ] [MAIN ] HQL: Fetched primary data is valid. [2025-10-20T19:15:02.799323] [DEBUG] [LOCAL ] HQL: Saving primary data. [2025-10-20T19:15:02.799939] [DEBUG] [LOCAL ] HQL: Trimmed 25348 records to 16958 (keeping 200 days of data) [2025-10-20T19:15:02.897587] [DEBUG] [LOCAL ] HQL: Primary data saved (16958 records). [2025-10-20T19:15:02.897748] [INFO ] [MAIN ] HQL: Full history saved. [2025-10-20T19:15:02.897831] [INFO ] [MAIN ] Update Progress: 1049/2367 symbols completed. HQL took 28.2s. Estimated remaining: 2h 49m 35s. Rate: 7.8 symbols/min [2025-10-20T19:15:03.061144] [INFO ] [MAIN ] HRL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:03.109487] [DEBUG] [LOCAL ] HRL: Loaded 53904 existing records from primary storage. [2025-10-20T19:15:03.109820] [INFO ] [MAIN ] HRL: Loaded 53904 existing records. [2025-10-20T19:15:03.286019] [WARN ] [MAIN ] HRL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:03.286243] [INFO ] [MAIN ] HRL: Wiping existing primary data before full refresh. [2025-10-20T19:15:03.286296] [INFO ] [LOCAL ] HRL: Starting deletion of primary data. [2025-10-20T19:15:03.286727] [DEBUG] [LOCAL ] HRL: Deleted primary data file. [2025-10-20T19:15:03.286783] [INFO ] [MAIN ] HRL: Starting full primary refresh. [2025-10-20T19:15:03.454967] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-02. [2025-10-20T19:15:03.637502] [INFO ] [REMOTE ] HRL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:03.762502] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T19:15:04.251433] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-01. [2025-10-20T19:15:04.641821] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T19:15:04.970628] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T19:15:05.051377] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-03. [2025-10-20T19:15:05.851381] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-05. [2025-10-20T19:15:06.051497] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T19:15:06.649058] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:15:06.651844] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-08. [2025-10-20T19:15:07.456609] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-04. [2025-10-20T19:15:07.472412] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T19:15:08.251415] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-06. [2025-10-20T19:15:08.433049] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T19:15:08.643817] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T19:15:09.051375] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-02. [2025-10-20T19:15:09.536491] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T19:15:09.851380] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-07. [2025-10-20T19:15:10.656987] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-09. [2025-10-20T19:15:10.675967] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T19:15:11.451378] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-03. [2025-10-20T19:15:11.547228] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:15:12.251350] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-05. [2025-10-20T19:15:12.356588] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T19:15:13.051445] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-08. [2025-10-20T19:15:13.359319] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:15:13.706708] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T19:15:13.851381] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-06. [2025-10-20T19:15:14.651405] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-10. [2025-10-20T19:15:14.775598] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:15:15.134548] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T19:15:15.451376] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-04. [2025-10-20T19:15:16.251385] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-09. [2025-10-20T19:15:16.505096] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:15:16.513015] [INFO ] [REMOTE ] HR: Full fetch complete (75068 records). [2025-10-20T19:15:16.513180] [DEBUG] [VALIDATOR ] HR: Validating primary data. [2025-10-20T19:15:16.914506] [DEBUG] [VALIDATOR ] HR: All expected trading days are present. [2025-10-20T19:15:16.914586] [DEBUG] [VALIDATOR ] HR: Primary data validation passed. [2025-10-20T19:15:16.915514] [INFO ] [MAIN ] HR: Fetched primary data is valid. [2025-10-20T19:15:16.915578] [DEBUG] [LOCAL ] HR: Saving primary data. [2025-10-20T19:15:16.917046] [DEBUG] [LOCAL ] HR: Trimmed 75068 records to 52115 (keeping 200 days of data) [2025-10-20T19:15:17.092342] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-07. [2025-10-20T19:15:17.238174] [DEBUG] [LOCAL ] HR: Primary data saved (52115 records). [2025-10-20T19:15:17.238309] [INFO ] [MAIN ] HR: Full history saved. [2025-10-20T19:15:17.238395] [INFO ] [MAIN ] Update Progress: 1050/2367 symbols completed. HR took 36.6s. Estimated remaining: 2h 49m 36s. Rate: 7.8 symbols/min [2025-10-20T19:15:17.418785] [INFO ] [MAIN ] HRTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:17.449955] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T19:15:17.491040] [DEBUG] [LOCAL ] HRTG: Loaded 34930 existing records from primary storage. [2025-10-20T19:15:17.491469] [INFO ] [MAIN ] HRTG: Loaded 34930 existing records. [2025-10-20T19:15:17.677946] [WARN ] [MAIN ] HRTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:17.678220] [INFO ] [MAIN ] HRTG: Wiping existing primary data before full refresh. [2025-10-20T19:15:17.678361] [INFO ] [LOCAL ] HRTG: Starting deletion of primary data. [2025-10-20T19:15:17.678705] [DEBUG] [LOCAL ] HRTG: Deleted primary data file. [2025-10-20T19:15:17.678763] [INFO ] [MAIN ] HRTG: Starting full primary refresh. [2025-10-20T19:15:17.900345] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-05. [2025-10-20T19:15:17.988774] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T19:15:18.116511] [INFO ] [REMOTE ] HRTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:18.632078] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:15:18.651395] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-01. [2025-10-20T19:15:19.451391] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-10. [2025-10-20T19:15:19.735343] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T19:15:20.067130] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T19:15:20.251392] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-08. [2025-10-20T19:15:21.051339] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-02. [2025-10-20T19:15:21.128374] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:15:21.135980] [INFO ] [REMOTE ] HRB: Full fetch complete (70025 records). [2025-10-20T19:15:21.136158] [DEBUG] [VALIDATOR ] HRB: Validating primary data. [2025-10-20T19:15:21.328446] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T19:15:21.577511] [DEBUG] [VALIDATOR ] HRB: All expected trading days are present. [2025-10-20T19:15:21.577585] [DEBUG] [VALIDATOR ] HRB: Primary data validation passed. [2025-10-20T19:15:21.578459] [INFO ] [MAIN ] HRB: Fetched primary data is valid. [2025-10-20T19:15:21.578520] [DEBUG] [LOCAL ] HRB: Saving primary data. [2025-10-20T19:15:21.579803] [DEBUG] [LOCAL ] HRB: Trimmed 70025 records to 48355 (keeping 200 days of data) [2025-10-20T19:15:21.860124] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-09. [2025-10-20T19:15:21.875873] [DEBUG] [LOCAL ] HRB: Primary data saved (48355 records). [2025-10-20T19:15:21.875970] [INFO ] [MAIN ] HRB: Full history saved. [2025-10-20T19:15:21.876047] [INFO ] [MAIN ] Update Progress: 1051/2367 symbols completed. HRB took 33.6s. Estimated remaining: 2h 49m 24s. Rate: 7.8 symbols/min [2025-10-20T19:15:22.055157] [INFO ] [MAIN ] HSBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:22.068765] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T19:15:22.116031] [DEBUG] [LOCAL ] HSBC: Loaded 51777 existing records from primary storage. [2025-10-20T19:15:22.116370] [INFO ] [MAIN ] HSBC: Loaded 51777 existing records. [2025-10-20T19:15:22.292881] [WARN ] [MAIN ] HSBC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:22.293129] [INFO ] [MAIN ] HSBC: Wiping existing primary data before full refresh. [2025-10-20T19:15:22.293180] [INFO ] [LOCAL ] HSBC: Starting deletion of primary data. [2025-10-20T19:15:22.293443] [DEBUG] [LOCAL ] HSBC: Deleted primary data file. [2025-10-20T19:15:22.293483] [INFO ] [MAIN ] HSBC: Starting full primary refresh. [2025-10-20T19:15:22.638699] [INFO ] [REMOTE ] HSBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:22.651330] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-01. [2025-10-20T19:15:23.254435] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T19:15:23.451374] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-10. [2025-10-20T19:15:24.251365] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-06. [2025-10-20T19:15:24.422119] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T19:15:24.471575] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T19:15:24.476079] [INFO ] [REMOTE ] HRI: Full fetch complete (46630 records). [2025-10-20T19:15:24.476168] [DEBUG] [VALIDATOR ] HRI: Validating primary data. [2025-10-20T19:15:24.881761] [DEBUG] [VALIDATOR ] HRI: All expected trading days are present. [2025-10-20T19:15:24.881839] [DEBUG] [VALIDATOR ] HRI: Primary data validation passed. [2025-10-20T19:15:24.882543] [INFO ] [MAIN ] HRI: Fetched primary data is valid. [2025-10-20T19:15:24.882582] [DEBUG] [LOCAL ] HRI: Saving primary data. [2025-10-20T19:15:24.883540] [DEBUG] [LOCAL ] HRI: Trimmed 46630 records to 32387 (keeping 200 days of data) [2025-10-20T19:15:25.068017] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-03. [2025-10-20T19:15:25.097166] [DEBUG] [LOCAL ] HRI: Primary data saved (32387 records). [2025-10-20T19:15:25.097309] [INFO ] [MAIN ] HRI: Full history saved. [2025-10-20T19:15:25.097399] [INFO ] [MAIN ] Update Progress: 1052/2367 symbols completed. HRI took 29.5s. Estimated remaining: 2h 49m 11s. Rate: 7.8 symbols/min [2025-10-20T19:15:25.300206] [INFO ] [MAIN ] HSHP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:25.316903] [DEBUG] [LOCAL ] HSHP: Loaded 15028 existing records from primary storage. [2025-10-20T19:15:25.317144] [INFO ] [MAIN ] HSHP: Loaded 15028 existing records. [2025-10-20T19:15:25.524447] [WARN ] [MAIN ] HSHP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:25.524688] [INFO ] [MAIN ] HSHP: Wiping existing primary data before full refresh. [2025-10-20T19:15:25.524746] [INFO ] [LOCAL ] HSHP: Starting deletion of primary data. [2025-10-20T19:15:25.524964] [DEBUG] [LOCAL ] HSHP: Deleted primary data file. [2025-10-20T19:15:25.525019] [INFO ] [MAIN ] HSHP: Starting full primary refresh. [2025-10-20T19:15:25.852962] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-02. [2025-10-20T19:15:25.898627] [INFO ] [REMOTE ] HSHP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:26.014890] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:15:26.250514] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T19:15:26.651425] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-01. [2025-10-20T19:15:27.451345] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-04. [2025-10-20T19:15:27.688589] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:15:27.733389] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T19:15:28.251373] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-02. [2025-10-20T19:15:28.884728] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T19:15:29.051350] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-07. [2025-10-20T19:15:29.281994] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:15:29.851390] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-05. [2025-10-20T19:15:30.651359] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-03. [2025-10-20T19:15:31.069519] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T19:15:31.355431] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T19:15:31.451349] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-06. [2025-10-20T19:15:31.741348] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T19:15:32.251443] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-03. [2025-10-20T19:15:32.815694] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T19:15:33.051381] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-07. [2025-10-20T19:15:33.851422] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-08. [2025-10-20T19:15:34.264603] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T19:15:34.331599] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T19:15:34.651377] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-04. [2025-10-20T19:15:35.451375] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-04. [2025-10-20T19:15:35.626267] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T19:15:35.716582] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T19:15:36.251392] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-08. [2025-10-20T19:15:37.051465] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-09. [2025-10-20T19:15:37.458362] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T19:15:37.669248] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:15:37.851410] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-05. [2025-10-20T19:15:38.651453] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-09. [2025-10-20T19:15:39.017834] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:15:39.451408] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-05. [2025-10-20T19:15:39.801395] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T19:15:40.108282] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:15:40.251369] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-10. [2025-10-20T19:15:40.315159] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T19:15:41.051379] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-06. [2025-10-20T19:15:41.851382] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-10. [2025-10-20T19:15:42.302207] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T19:15:42.311186] [INFO ] [REMOTE ] HRL: Full fetch complete (76999 records). [2025-10-20T19:15:42.311372] [DEBUG] [VALIDATOR ] HRL: Validating primary data. [2025-10-20T19:15:42.652473] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-06. [2025-10-20T19:15:42.909000] [DEBUG] [VALIDATOR ] HRL: All expected trading days are present. [2025-10-20T19:15:42.909148] [DEBUG] [VALIDATOR ] HRL: Primary data validation passed. [2025-10-20T19:15:42.911095] [INFO ] [MAIN ] HRL: Fetched primary data is valid. [2025-10-20T19:15:42.911344] [DEBUG] [LOCAL ] HRL: Saving primary data. [2025-10-20T19:15:42.935548] [DEBUG] [LOCAL ] HRL: Trimmed 76999 records to 53128 (keeping 200 days of data) [2025-10-20T19:15:42.965355] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T19:15:43.269022] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T19:15:43.294676] [INFO ] [REMOTE ] HRTG: Full fetch complete (44350 records). [2025-10-20T19:15:43.312419] [DEBUG] [VALIDATOR ] HRTG: Validating primary data. [2025-10-20T19:15:43.455804] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-07. [2025-10-20T19:15:43.537767] [DEBUG] [LOCAL ] HRL: Primary data saved (53128 records). [2025-10-20T19:15:43.558898] [INFO ] [MAIN ] HRL: Full history saved. [2025-10-20T19:15:43.565536] [INFO ] [MAIN ] Update Progress: 1053/2367 symbols completed. HRL took 40.7s. Estimated remaining: 2h 49m 17s. Rate: 7.8 symbols/min [2025-10-20T19:15:43.964955] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T19:15:43.957792] [INFO ] [MAIN ] HSY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:44.074757] [DEBUG] [VALIDATOR ] HRTG: All expected trading days are present. [2025-10-20T19:15:44.074864] [DEBUG] [VALIDATOR ] HRTG: Primary data validation passed. [2025-10-20T19:15:44.075836] [INFO ] [MAIN ] HRTG: Fetched primary data is valid. [2025-10-20T19:15:44.076021] [DEBUG] [LOCAL ] HRTG: Saving primary data. [2025-10-20T19:15:44.077233] [DEBUG] [LOCAL ] HRTG: Trimmed 44350 records to 34587 (keeping 200 days of data) [2025-10-20T19:15:44.268127] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-07. [2025-10-20T19:15:44.309597] [DEBUG] [LOCAL ] HSY: Loaded 51192 existing records from primary storage. [2025-10-20T19:15:44.312548] [DEBUG] [LOCAL ] HRTG: Primary data saved (34587 records). [2025-10-20T19:15:44.313248] [INFO ] [MAIN ] HRTG: Full history saved. [2025-10-20T19:15:44.313938] [INFO ] [MAIN ] Update Progress: 1054/2367 symbols completed. HRTG took 27.1s. Estimated remaining: 2h 49m 0s. Rate: 7.8 symbols/min [2025-10-20T19:15:44.314098] [INFO ] [MAIN ] HSY: Loaded 51192 existing records. [2025-10-20T19:15:44.652801] [INFO ] [MAIN ] HTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:44.668198] [DEBUG] [LOCAL ] HTB: No existing data file found in primary storage. [2025-10-20T19:15:44.668285] [WARN ] [MAIN ] HTB: No existing data found, performing full refresh instead. [2025-10-20T19:15:44.668434] [INFO ] [MAIN ] HTB: Wiping existing primary data before full refresh. [2025-10-20T19:15:44.672866] [INFO ] [LOCAL ] HTB: Starting deletion of primary data. [2025-10-20T19:15:44.672766] [WARN ] [MAIN ] HSY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:44.673008] [DEBUG] [LOCAL ] HTB: No primary data file found to delete. [2025-10-20T19:15:44.673166] [INFO ] [MAIN ] HSY: Wiping existing primary data before full refresh. [2025-10-20T19:15:44.673250] [INFO ] [LOCAL ] HSY: Starting deletion of primary data. [2025-10-20T19:15:44.673333] [INFO ] [MAIN ] HTB: Starting full primary refresh. [2025-10-20T19:15:44.704594] [DEBUG] [LOCAL ] HSY: Deleted primary data file. [2025-10-20T19:15:44.736282] [INFO ] [MAIN ] HSY: Starting full primary refresh. [2025-10-20T19:15:45.417541] [INFO ] [REMOTE ] HTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:45.422826] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-01. [2025-10-20T19:15:45.467516] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T19:15:45.465944] [INFO ] [REMOTE ] HSY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:45.510960] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T19:15:45.652410] [DEBUG] [REMOTE ] HTB: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:15:45.652961] [WARN ] [REMOTE ] HTB: No data returned for month 2025-01 [2025-10-20T19:15:45.653064] [FATAL] [REMOTE ] HTB: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:15:45.653117] [FATAL] [MAIN ] HTB: Failed to fetch primary data. [2025-10-20T19:15:45.653179] [WARN ] [MAIN ] HTB: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:15:45.653258] [INFO ] [LOCAL ] HTB: Starting deletion of primary data. [2025-10-20T19:15:45.653333] [DEBUG] [LOCAL ] HTB: No primary data file found to delete. [2025-10-20T19:15:45.653372] [INFO ] [LOCAL ] HTB: Starting deletion of derivative data. [2025-10-20T19:15:45.653413] [DEBUG] [LOCAL ] HTB: No derivative data file found to delete. [2025-10-20T19:15:45.653466] [INFO ] [MAIN ] Update Progress: 1055/2367 symbols completed. HTB took 1.3s. Estimated remaining: 2h 48m 44s. Rate: 7.8 symbols/min [2025-10-20T19:15:45.837494] [INFO ] [MAIN ] HTD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:45.853257] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-01. [2025-10-20T19:15:45.853673] [DEBUG] [LOCAL ] HTD: Loaded 14424 existing records from primary storage. [2025-10-20T19:15:45.855041] [INFO ] [MAIN ] HTD: Loaded 14424 existing records. [2025-10-20T19:15:46.059305] [WARN ] [MAIN ] HTD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:46.059556] [INFO ] [MAIN ] HTD: Wiping existing primary data before full refresh. [2025-10-20T19:15:46.059604] [INFO ] [LOCAL ] HTD: Starting deletion of primary data. [2025-10-20T19:15:46.059895] [DEBUG] [LOCAL ] HTD: Deleted primary data file. [2025-10-20T19:15:46.059952] [INFO ] [MAIN ] HTD: Starting full primary refresh. [2025-10-20T19:15:46.411097] [INFO ] [REMOTE ] HTD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:46.651380] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-08. [2025-10-20T19:15:47.451372] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-08. [2025-10-20T19:15:47.823251] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T19:15:47.992840] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T19:15:48.251356] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-01. [2025-10-20T19:15:48.305771] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T19:15:49.051431] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-09. [2025-10-20T19:15:49.093714] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T19:15:49.851400] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-09. [2025-10-20T19:15:50.083736] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T19:15:50.651413] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-10. [2025-10-20T19:15:51.451379] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-02. [2025-10-20T19:15:51.617316] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:15:51.733288] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T19:15:51.735602] [INFO ] [REMOTE ] HSHP: Full fetch complete (23285 records). [2025-10-20T19:15:51.735705] [DEBUG] [VALIDATOR ] HSHP: Validating primary data. [2025-10-20T19:15:52.107430] [DEBUG] [VALIDATOR ] HSHP: All expected trading days are present. [2025-10-20T19:15:52.107512] [DEBUG] [VALIDATOR ] HSHP: Primary data validation passed. [2025-10-20T19:15:52.108108] [INFO ] [MAIN ] HSHP: Fetched primary data is valid. [2025-10-20T19:15:52.108205] [DEBUG] [LOCAL ] HSHP: Saving primary data. [2025-10-20T19:15:52.108875] [DEBUG] [LOCAL ] HSHP: Trimmed 23285 records to 14817 (keeping 200 days of data) [2025-10-20T19:15:52.197573] [DEBUG] [LOCAL ] HSHP: Primary data saved (14817 records). [2025-10-20T19:15:52.197986] [INFO ] [MAIN ] HSHP: Full history saved. [2025-10-20T19:15:52.198089] [INFO ] [MAIN ] Update Progress: 1056/2367 symbols completed. HSHP took 27.1s. Estimated remaining: 2h 48m 35s. Rate: 7.8 symbols/min [2025-10-20T19:15:52.260811] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-02. [2025-10-20T19:15:52.311568] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T19:15:52.387675] [INFO ] [MAIN ] HTFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:52.387901] [DEBUG] [LOCAL ] HTFC: No existing data file found in primary storage. [2025-10-20T19:15:52.387946] [WARN ] [MAIN ] HTFC: No existing data found, performing full refresh instead. [2025-10-20T19:15:52.388089] [INFO ] [MAIN ] HTFC: Wiping existing primary data before full refresh. [2025-10-20T19:15:52.388130] [INFO ] [LOCAL ] HTFC: Starting deletion of primary data. [2025-10-20T19:15:52.388172] [DEBUG] [LOCAL ] HTFC: No primary data file found to delete. [2025-10-20T19:15:52.388249] [INFO ] [MAIN ] HTFC: Starting full primary refresh. [2025-10-20T19:15:52.751612] [INFO ] [REMOTE ] HTFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:53.051423] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-10. [2025-10-20T19:15:53.851348] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-03. [2025-10-20T19:15:54.261077] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-10-20T19:15:54.651340] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-01. [2025-10-20T19:15:54.733186] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T19:15:55.034153] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-01 (0.38 seconds). [2025-10-20T19:15:55.057585] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T19:15:55.065111] [INFO ] [REMOTE ] HSBC: Full fetch complete (74147 records). [2025-10-20T19:15:55.065241] [DEBUG] [VALIDATOR ] HSBC: Validating primary data. [2025-10-20T19:15:55.456484] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-03. [2025-10-20T19:15:55.456987] [DEBUG] [VALIDATOR ] HSBC: All expected trading days are present. [2025-10-20T19:15:55.457327] [DEBUG] [VALIDATOR ] HSBC: Primary data validation passed. [2025-10-20T19:15:55.459160] [INFO ] [MAIN ] HSBC: Fetched primary data is valid. [2025-10-20T19:15:55.459333] [DEBUG] [LOCAL ] HSBC: Saving primary data. [2025-10-20T19:15:55.460714] [DEBUG] [LOCAL ] HSBC: Trimmed 74147 records to 50998 (keeping 200 days of data) [2025-10-20T19:15:55.776550] [DEBUG] [LOCAL ] HSBC: Primary data saved (50998 records). [2025-10-20T19:15:55.776837] [INFO ] [MAIN ] HSBC: Full history saved. [2025-10-20T19:15:55.776965] [INFO ] [MAIN ] Update Progress: 1057/2367 symbols completed. HSBC took 33.9s. Estimated remaining: 2h 48m 22s. Rate: 7.8 symbols/min [2025-10-20T19:15:55.949951] [INFO ] [MAIN ] HTGC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:55.950147] [DEBUG] [LOCAL ] HTGC: No existing data file found in primary storage. [2025-10-20T19:15:55.950186] [WARN ] [MAIN ] HTGC: No existing data found, performing full refresh instead. [2025-10-20T19:15:55.950297] [INFO ] [MAIN ] HTGC: Wiping existing primary data before full refresh. [2025-10-20T19:15:55.950338] [INFO ] [LOCAL ] HTGC: Starting deletion of primary data. [2025-10-20T19:15:55.950379] [DEBUG] [LOCAL ] HTGC: No primary data file found to delete. [2025-10-20T19:15:55.950412] [INFO ] [MAIN ] HTGC: Starting full primary refresh. [2025-10-20T19:15:56.266675] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-04. [2025-10-20T19:15:56.308714] [INFO ] [REMOTE ] HTGC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:57.051343] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-01. [2025-10-20T19:15:57.214283] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T19:15:57.428318] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T19:15:57.851397] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-05. [2025-10-20T19:15:58.645437] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T19:15:58.651348] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-04. [2025-10-20T19:15:58.732331] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T19:15:59.451384] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-06. [2025-10-20T19:16:00.162497] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T19:16:00.251339] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-02. [2025-10-20T19:16:00.533482] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T19:16:01.051450] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-02. [2025-10-20T19:16:01.701159] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T19:16:01.851387] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-05. [2025-10-20T19:16:01.991481] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:16:02.651371] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-03. [2025-10-20T19:16:02.980811] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-10-20T19:16:03.451347] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-04. [2025-10-20T19:16:03.648841] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T19:16:03.858499] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T19:16:04.251534] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-05. [2025-10-20T19:16:04.621949] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-10-20T19:16:05.051411] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-07. [2025-10-20T19:16:05.851347] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-06. [2025-10-20T19:16:05.996489] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T19:16:06.184847] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-10-20T19:16:06.651371] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-08. [2025-10-20T19:16:07.451425] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-06. [2025-10-20T19:16:07.497629] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T19:16:08.251413] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-07. [2025-10-20T19:16:08.502971] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-07 (0.25 seconds). [2025-10-20T19:16:09.051391] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-08. [2025-10-20T19:16:09.265773] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T19:16:09.352320] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-08 (0.30 seconds). [2025-10-20T19:16:09.851385] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-09. [2025-10-20T19:16:10.651362] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-03. [2025-10-20T19:16:10.801535] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T19:16:11.451404] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-10. [2025-10-20T19:16:12.251389] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-09. [2025-10-20T19:16:12.479186] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T19:16:12.483047] [INFO ] [REMOTE ] HTD: Full fetch complete (19664 records). [2025-10-20T19:16:12.492633] [DEBUG] [VALIDATOR ] HTD: Validating primary data. [2025-10-20T19:16:12.599204] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T19:16:12.620527] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-10-20T19:16:12.899447] [DEBUG] [VALIDATOR ] HTD: All expected trading days are present. [2025-10-20T19:16:12.899533] [DEBUG] [VALIDATOR ] HTD: Primary data validation passed. [2025-10-20T19:16:12.899949] [INFO ] [MAIN ] HTD: Fetched primary data is valid. [2025-10-20T19:16:12.900047] [DEBUG] [LOCAL ] HTD: Saving primary data. [2025-10-20T19:16:12.900846] [DEBUG] [LOCAL ] HTD: Trimmed 19664 records to 14305 (keeping 200 days of data) [2025-10-20T19:16:12.987115] [DEBUG] [LOCAL ] HTD: Primary data saved (14305 records). [2025-10-20T19:16:12.987259] [INFO ] [MAIN ] HTD: Full history saved. [2025-10-20T19:16:12.987340] [INFO ] [MAIN ] Update Progress: 1058/2367 symbols completed. HTD took 27.3s. Estimated remaining: 2h 48m 26s. Rate: 7.8 symbols/min [2025-10-20T19:16:13.056469] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-04. [2025-10-20T19:16:13.166311] [INFO ] [MAIN ] HTH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:13.198496] [DEBUG] [LOCAL ] HTH: Loaded 33038 existing records from primary storage. [2025-10-20T19:16:13.198788] [INFO ] [MAIN ] HTH: Loaded 33038 existing records. [2025-10-20T19:16:13.376898] [WARN ] [MAIN ] HTH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:13.377128] [INFO ] [MAIN ] HTH: Wiping existing primary data before full refresh. [2025-10-20T19:16:13.377176] [INFO ] [LOCAL ] HTH: Starting deletion of primary data. [2025-10-20T19:16:13.377411] [DEBUG] [LOCAL ] HTH: Deleted primary data file. [2025-10-20T19:16:13.377468] [INFO ] [MAIN ] HTH: Starting full primary refresh. [2025-10-20T19:16:13.730482] [INFO ] [REMOTE ] HTH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:13.851341] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-07. [2025-10-20T19:16:14.651354] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-10. [2025-10-20T19:16:14.826527] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:16:15.051616] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T19:16:15.052302] [INFO ] [REMOTE ] HTFC: Full fetch complete (839 records). [2025-10-20T19:16:15.052371] [DEBUG] [VALIDATOR ] HTFC: Validating primary data. [2025-10-20T19:16:15.233539] [WARN ] [VALIDATOR ] HTFC: Validation failed - insufficient historical data. Have 182 unique days, need 200 days [2025-10-20T19:16:15.233746] [FATAL] [MAIN ] HTFC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:16:15.233821] [WARN ] [MAIN ] HTFC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:16:15.233904] [INFO ] [LOCAL ] HTFC: Starting deletion of primary data. [2025-10-20T19:16:15.233993] [DEBUG] [LOCAL ] HTFC: No primary data file found to delete. [2025-10-20T19:16:15.234037] [INFO ] [LOCAL ] HTFC: Starting deletion of derivative data. [2025-10-20T19:16:15.234079] [DEBUG] [LOCAL ] HTFC: No derivative data file found to delete. [2025-10-20T19:16:15.234135] [INFO ] [MAIN ] Update Progress: 1059/2367 symbols completed. HTFC took 23.0s. Estimated remaining: 2h 48m 12s. Rate: 7.8 symbols/min [2025-10-20T19:16:15.429270] [INFO ] [MAIN ] HUBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:15.451378] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-01. [2025-10-20T19:16:15.466628] [DEBUG] [LOCAL ] HUBB: Loaded 36909 existing records from primary storage. [2025-10-20T19:16:15.466946] [INFO ] [MAIN ] HUBB: Loaded 36909 existing records. [2025-10-20T19:16:15.665975] [WARN ] [MAIN ] HUBB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:15.666299] [INFO ] [MAIN ] HUBB: Wiping existing primary data before full refresh. [2025-10-20T19:16:15.666423] [INFO ] [LOCAL ] HUBB: Starting deletion of primary data. [2025-10-20T19:16:15.667016] [DEBUG] [LOCAL ] HUBB: Deleted primary data file. [2025-10-20T19:16:15.667075] [INFO ] [MAIN ] HUBB: Starting full primary refresh. [2025-10-20T19:16:15.990061] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-10-20T19:16:16.078261] [INFO ] [REMOTE ] HUBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:16.251376] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-08. [2025-10-20T19:16:16.674496] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T19:16:17.051386] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-01. [2025-10-20T19:16:17.851383] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-05. [2025-10-20T19:16:18.070269] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T19:16:18.521291] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T19:16:18.651371] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-02. [2025-10-20T19:16:19.451357] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-09. [2025-10-20T19:16:19.666896] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T19:16:20.101538] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T19:16:20.251361] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-03. [2025-10-20T19:16:21.051396] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-02. [2025-10-20T19:16:21.233231] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:16:21.301067] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:16:21.851392] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-10. [2025-10-20T19:16:22.658869] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-04. [2025-10-20T19:16:22.664287] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T19:16:22.873301] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T19:16:22.881114] [INFO ] [REMOTE ] HSY: Full fetch complete (73654 records). [2025-10-20T19:16:22.881260] [DEBUG] [VALIDATOR ] HSY: Validating primary data. [2025-10-20T19:16:23.275097] [DEBUG] [VALIDATOR ] HSY: All expected trading days are present. [2025-10-20T19:16:23.275218] [DEBUG] [VALIDATOR ] HSY: Primary data validation passed. [2025-10-20T19:16:23.276295] [INFO ] [MAIN ] HSY: Fetched primary data is valid. [2025-10-20T19:16:23.276357] [DEBUG] [LOCAL ] HSY: Saving primary data. [2025-10-20T19:16:23.277807] [DEBUG] [LOCAL ] HSY: Trimmed 73654 records to 50468 (keeping 200 days of data) [2025-10-20T19:16:23.472828] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-06. [2025-10-20T19:16:23.607867] [DEBUG] [LOCAL ] HSY: Primary data saved (50468 records). [2025-10-20T19:16:23.607995] [INFO ] [MAIN ] HSY: Full history saved. [2025-10-20T19:16:23.608073] [INFO ] [MAIN ] Update Progress: 1060/2367 symbols completed. HSY took 40.0s. Estimated remaining: 2h 48m 5s. Rate: 7.8 symbols/min [2025-10-20T19:16:23.777601] [INFO ] [MAIN ] HUBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:23.815365] [DEBUG] [LOCAL ] HUBS: Loaded 40651 existing records from primary storage. [2025-10-20T19:16:23.815681] [INFO ] [MAIN ] HUBS: Loaded 40651 existing records. [2025-10-20T19:16:23.997500] [WARN ] [MAIN ] HUBS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:23.997861] [INFO ] [MAIN ] HUBS: Wiping existing primary data before full refresh. [2025-10-20T19:16:23.997931] [INFO ] [LOCAL ] HUBS: Starting deletion of primary data. [2025-10-20T19:16:23.999128] [DEBUG] [LOCAL ] HUBS: Deleted primary data file. [2025-10-20T19:16:23.999185] [INFO ] [MAIN ] HUBS: Starting full primary refresh. [2025-10-20T19:16:24.161760] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T19:16:24.257007] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-05. [2025-10-20T19:16:24.380925] [INFO ] [REMOTE ] HUBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:25.051402] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-03. [2025-10-20T19:16:25.134314] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:16:25.562478] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:16:25.851391] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-01. [2025-10-20T19:16:26.651376] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-07. [2025-10-20T19:16:26.781724] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:16:26.874541] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T19:16:27.451370] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-04. [2025-10-20T19:16:28.251391] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-06. [2025-10-20T19:16:28.373181] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T19:16:29.051338] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-08. [2025-10-20T19:16:29.133341] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:16:29.485732] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T19:16:29.851391] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-05. [2025-10-20T19:16:30.651379] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-02. [2025-10-20T19:16:30.709982] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:16:31.366919] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T19:16:31.451359] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-09. [2025-10-20T19:16:32.101021] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T19:16:32.251386] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-06. [2025-10-20T19:16:33.051349] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-03. [2025-10-20T19:16:33.236345] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:16:33.599584] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T19:16:33.851419] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-07. [2025-10-20T19:16:34.651364] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-07. [2025-10-20T19:16:34.678067] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:16:34.751400] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:16:35.451455] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-08. [2025-10-20T19:16:36.226608] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:16:36.251405] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-04. [2025-10-20T19:16:36.866842] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:16:37.051417] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-10. [2025-10-20T19:16:37.612055] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T19:16:37.851363] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-05. [2025-10-20T19:16:38.651370] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-08. [2025-10-20T19:16:38.822248] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T19:16:38.832953] [INFO ] [REMOTE ] HTGC: Full fetch complete (68896 records). [2025-10-20T19:16:38.833135] [DEBUG] [VALIDATOR ] HTGC: Validating primary data. [2025-10-20T19:16:39.261982] [DEBUG] [VALIDATOR ] HTGC: All expected trading days are present. [2025-10-20T19:16:39.262096] [DEBUG] [VALIDATOR ] HTGC: Primary data validation passed. [2025-10-20T19:16:39.263048] [INFO ] [MAIN ] HTGC: Fetched primary data is valid. [2025-10-20T19:16:39.263129] [DEBUG] [LOCAL ] HTGC: Saving primary data. [2025-10-20T19:16:39.265005] [DEBUG] [LOCAL ] HTGC: Trimmed 68896 records to 47761 (keeping 200 days of data) [2025-10-20T19:16:39.290491] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T19:16:39.470731] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-09. [2025-10-20T19:16:39.620845] [DEBUG] [LOCAL ] HTGC: Primary data saved (47761 records). [2025-10-20T19:16:39.622170] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T19:16:39.628981] [INFO ] [MAIN ] HTGC: Full history saved. [2025-10-20T19:16:39.629409] [INFO ] [MAIN ] Update Progress: 1061/2367 symbols completed. HTGC took 43.9s. Estimated remaining: 2h 48m 8s. Rate: 7.8 symbols/min [2025-10-20T19:16:39.797292] [INFO ] [MAIN ] HUM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:39.840362] [DEBUG] [LOCAL ] HUM: Loaded 50015 existing records from primary storage. [2025-10-20T19:16:39.840697] [INFO ] [MAIN ] HUM: Loaded 50015 existing records. [2025-10-20T19:16:40.015584] [WARN ] [MAIN ] HUM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:40.015799] [INFO ] [MAIN ] HUM: Wiping existing primary data before full refresh. [2025-10-20T19:16:40.015847] [INFO ] [LOCAL ] HUM: Starting deletion of primary data. [2025-10-20T19:16:40.016409] [DEBUG] [LOCAL ] HUM: Deleted primary data file. [2025-10-20T19:16:40.016449] [INFO ] [MAIN ] HUM: Starting full primary refresh. [2025-10-20T19:16:40.256611] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-06. [2025-10-20T19:16:40.368703] [INFO ] [REMOTE ] HUM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:40.770635] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T19:16:41.051385] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-01. [2025-10-20T19:16:41.785893] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:16:41.851363] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-10. [2025-10-20T19:16:42.651370] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-07. [2025-10-20T19:16:42.708192] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T19:16:43.227612] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T19:16:43.232018] [INFO ] [REMOTE ] HTH: Full fetch complete (46847 records). [2025-10-20T19:16:43.232101] [DEBUG] [VALIDATOR ] HTH: Validating primary data. [2025-10-20T19:16:43.458084] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-09. [2025-10-20T19:16:43.604731] [DEBUG] [VALIDATOR ] HTH: All expected trading days are present. [2025-10-20T19:16:43.604832] [DEBUG] [VALIDATOR ] HTH: Primary data validation passed. [2025-10-20T19:16:43.605530] [INFO ] [MAIN ] HTH: Fetched primary data is valid. [2025-10-20T19:16:43.605569] [DEBUG] [LOCAL ] HTH: Saving primary data. [2025-10-20T19:16:43.606539] [DEBUG] [LOCAL ] HTH: Trimmed 46847 records to 32664 (keeping 200 days of data) [2025-10-20T19:16:43.800421] [DEBUG] [LOCAL ] HTH: Primary data saved (32664 records). [2025-10-20T19:16:43.800559] [INFO ] [MAIN ] HTH: Full history saved. [2025-10-20T19:16:43.800635] [INFO ] [MAIN ] Update Progress: 1062/2367 symbols completed. HTH took 30.8s. Estimated remaining: 2h 47m 56s. Rate: 7.8 symbols/min [2025-10-20T19:16:43.969445] [INFO ] [MAIN ] HUN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:44.019197] [DEBUG] [LOCAL ] HUN: Loaded 53992 existing records from primary storage. [2025-10-20T19:16:44.019507] [INFO ] [MAIN ] HUN: Loaded 53992 existing records. [2025-10-20T19:16:44.200512] [WARN ] [MAIN ] HUN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:44.200830] [INFO ] [MAIN ] HUN: Wiping existing primary data before full refresh. [2025-10-20T19:16:44.200894] [INFO ] [LOCAL ] HUN: Starting deletion of primary data. [2025-10-20T19:16:44.208603] [DEBUG] [LOCAL ] HUN: Deleted primary data file. [2025-10-20T19:16:44.221583] [INFO ] [MAIN ] HUN: Starting full primary refresh. [2025-10-20T19:16:44.253105] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-02. [2025-10-20T19:16:44.287613] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T19:16:44.604197] [INFO ] [REMOTE ] HUN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:44.896806] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T19:16:45.051427] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-08. [2025-10-20T19:16:45.851352] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-01. [2025-10-20T19:16:46.325961] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-10-20T19:16:46.651432] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-03. [2025-10-20T19:16:46.878763] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T19:16:47.457732] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-10. [2025-10-20T19:16:47.464027] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T19:16:48.206621] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:16:48.251419] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-09. [2025-10-20T19:16:48.992174] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T19:16:48.997204] [INFO ] [REMOTE ] HUBB: Full fetch complete (54278 records). [2025-10-20T19:16:48.997323] [DEBUG] [VALIDATOR ] HUBB: Validating primary data. [2025-10-20T19:16:49.060137] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-04. [2025-10-20T19:16:49.376469] [DEBUG] [VALIDATOR ] HUBB: All expected trading days are present. [2025-10-20T19:16:49.376555] [DEBUG] [VALIDATOR ] HUBB: Primary data validation passed. [2025-10-20T19:16:49.377329] [INFO ] [MAIN ] HUBB: Fetched primary data is valid. [2025-10-20T19:16:49.377392] [DEBUG] [LOCAL ] HUBB: Saving primary data. [2025-10-20T19:16:49.378543] [DEBUG] [LOCAL ] HUBB: Trimmed 54278 records to 36261 (keeping 200 days of data) [2025-10-20T19:16:49.604875] [DEBUG] [LOCAL ] HUBB: Primary data saved (36261 records). [2025-10-20T19:16:49.605083] [INFO ] [MAIN ] HUBB: Full history saved. [2025-10-20T19:16:49.605182] [INFO ] [MAIN ] Update Progress: 1063/2367 symbols completed. HUBB took 34.4s. Estimated remaining: 2h 47m 45s. Rate: 7.8 symbols/min [2025-10-20T19:16:49.774121] [INFO ] [MAIN ] HUYA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:49.816418] [DEBUG] [LOCAL ] HUYA: Loaded 44415 existing records from primary storage. [2025-10-20T19:16:49.816777] [INFO ] [MAIN ] HUYA: Loaded 44415 existing records. [2025-10-20T19:16:49.859731] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-02. [2025-10-20T19:16:50.004478] [WARN ] [MAIN ] HUYA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:50.004822] [INFO ] [MAIN ] HUYA: Wiping existing primary data before full refresh. [2025-10-20T19:16:50.004881] [INFO ] [LOCAL ] HUYA: Starting deletion of primary data. [2025-10-20T19:16:50.005419] [DEBUG] [LOCAL ] HUYA: Deleted primary data file. [2025-10-20T19:16:50.005477] [INFO ] [MAIN ] HUYA: Starting full primary refresh. [2025-10-20T19:16:50.313765] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T19:16:50.404721] [INFO ] [REMOTE ] HUYA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:50.651376] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-10. [2025-10-20T19:16:50.917931] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:16:51.451356] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-05. [2025-10-20T19:16:51.582980] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:16:52.251407] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-03. [2025-10-20T19:16:52.384900] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T19:16:52.391496] [INFO ] [REMOTE ] HUBS: Full fetch complete (56978 records). [2025-10-20T19:16:52.391636] [DEBUG] [VALIDATOR ] HUBS: Validating primary data. [2025-10-20T19:16:52.775184] [DEBUG] [VALIDATOR ] HUBS: All expected trading days are present. [2025-10-20T19:16:52.775285] [DEBUG] [VALIDATOR ] HUBS: Primary data validation passed. [2025-10-20T19:16:52.776100] [INFO ] [MAIN ] HUBS: Fetched primary data is valid. [2025-10-20T19:16:52.776161] [DEBUG] [LOCAL ] HUBS: Saving primary data. [2025-10-20T19:16:52.777243] [DEBUG] [LOCAL ] HUBS: Trimmed 56978 records to 40086 (keeping 200 days of data) [2025-10-20T19:16:53.013456] [DEBUG] [LOCAL ] HUBS: Primary data saved (40086 records). [2025-10-20T19:16:53.013572] [INFO ] [MAIN ] HUBS: Full history saved. [2025-10-20T19:16:53.013671] [INFO ] [MAIN ] Update Progress: 1064/2367 symbols completed. HUBS took 29.4s. Estimated remaining: 2h 47m 32s. Rate: 7.8 symbols/min [2025-10-20T19:16:53.078456] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-01. [2025-10-20T19:16:53.202446] [INFO ] [MAIN ] HVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:53.244482] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:16:53.264165] [DEBUG] [LOCAL ] HVT: Loaded 15340 existing records from primary storage. [2025-10-20T19:16:53.265212] [INFO ] [MAIN ] HVT: Loaded 15340 existing records. [2025-10-20T19:16:53.450044] [WARN ] [MAIN ] HVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:53.450282] [INFO ] [MAIN ] HVT: Wiping existing primary data before full refresh. [2025-10-20T19:16:53.450336] [INFO ] [LOCAL ] HVT: Starting deletion of primary data. [2025-10-20T19:16:53.450525] [DEBUG] [LOCAL ] HVT: Deleted primary data file. [2025-10-20T19:16:53.450581] [INFO ] [MAIN ] HVT: Starting full primary refresh. [2025-10-20T19:16:53.834914] [INFO ] [REMOTE ] HVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:53.851371] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-06. [2025-10-20T19:16:53.940330] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:16:54.651360] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-01. [2025-10-20T19:16:54.718864] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:16:55.451415] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-04. [2025-10-20T19:16:55.525748] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:16:55.670746] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T19:16:56.251373] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-02. [2025-10-20T19:16:57.009999] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T19:16:57.058881] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-03. [2025-10-20T19:16:57.080767] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T19:16:57.851406] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-05. [2025-10-20T19:16:58.181852] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:16:58.651429] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-04. [2025-10-20T19:16:59.451387] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-02. [2025-10-20T19:16:59.704339] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T19:16:59.735575] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T19:17:00.251395] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-06. [2025-10-20T19:17:01.051383] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-05. [2025-10-20T19:17:01.276350] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T19:17:01.851444] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-03. [2025-10-20T19:17:02.116791] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T19:17:02.337406] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-06 (2.09 seconds). [2025-10-20T19:17:02.651379] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-07. [2025-10-20T19:17:03.451386] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-06. [2025-10-20T19:17:03.536369] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T19:17:04.251362] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-07. [2025-10-20T19:17:04.373895] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T19:17:04.613173] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T19:17:05.051399] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-04. [2025-10-20T19:17:05.851358] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-07. [2025-10-20T19:17:06.127412] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T19:17:06.651346] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-08. [2025-10-20T19:17:07.289786] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-04 (2.24 seconds). [2025-10-20T19:17:07.451384] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-05. [2025-10-20T19:17:08.251376] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-08. [2025-10-20T19:17:08.291701] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-07 (2.44 seconds). [2025-10-20T19:17:09.051420] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-08. [2025-10-20T19:17:09.444047] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T19:17:09.572212] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-08 (2.92 seconds). [2025-10-20T19:17:09.681621] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T19:17:09.851340] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-09. [2025-10-20T19:17:10.203635] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-10-20T19:17:10.651369] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-09. [2025-10-20T19:17:11.451403] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-06. [2025-10-20T19:17:11.546851] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T19:17:11.578704] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:17:12.251386] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-10. [2025-10-20T19:17:12.915978] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:17:13.051377] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-10. [2025-10-20T19:17:13.383839] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T19:17:13.386354] [INFO ] [REMOTE ] HVT: Full fetch complete (23113 records). [2025-10-20T19:17:13.386439] [DEBUG] [VALIDATOR ] HVT: Validating primary data. [2025-10-20T19:17:13.745697] [DEBUG] [VALIDATOR ] HVT: All expected trading days are present. [2025-10-20T19:17:13.745784] [DEBUG] [VALIDATOR ] HVT: Primary data validation passed. [2025-10-20T19:17:13.746241] [INFO ] [MAIN ] HVT: Fetched primary data is valid. [2025-10-20T19:17:13.746278] [DEBUG] [LOCAL ] HVT: Saving primary data. [2025-10-20T19:17:13.746888] [DEBUG] [LOCAL ] HVT: Trimmed 23113 records to 14991 (keeping 200 days of data) [2025-10-20T19:17:13.833034] [DEBUG] [LOCAL ] HVT: Primary data saved (14991 records). [2025-10-20T19:17:13.833153] [INFO ] [MAIN ] HVT: Full history saved. [2025-10-20T19:17:13.833226] [INFO ] [MAIN ] Update Progress: 1065/2367 symbols completed. HVT took 20.8s. Estimated remaining: 2h 47m 41s. Rate: 7.8 symbols/min [2025-10-20T19:17:13.863789] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-09. [2025-10-20T19:17:14.003426] [INFO ] [MAIN ] HWM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:14.055275] [DEBUG] [LOCAL ] HWM: Loaded 53536 existing records from primary storage. [2025-10-20T19:17:14.055573] [INFO ] [MAIN ] HWM: Loaded 53536 existing records. [2025-10-20T19:17:14.228410] [WARN ] [MAIN ] HWM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:14.228618] [INFO ] [MAIN ] HWM: Wiping existing primary data before full refresh. [2025-10-20T19:17:14.228692] [INFO ] [LOCAL ] HWM: Starting deletion of primary data. [2025-10-20T19:17:14.228930] [DEBUG] [LOCAL ] HWM: Deleted primary data file. [2025-10-20T19:17:14.228986] [INFO ] [MAIN ] HWM: Starting full primary refresh. [2025-10-20T19:17:14.574314] [INFO ] [REMOTE ] HWM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:14.651357] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-01. [2025-10-20T19:17:15.032740] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T19:17:15.039966] [INFO ] [REMOTE ] HUM: Full fetch complete (71139 records). [2025-10-20T19:17:15.040083] [DEBUG] [VALIDATOR ] HUM: Validating primary data. [2025-10-20T19:17:15.423818] [DEBUG] [VALIDATOR ] HUM: All expected trading days are present. [2025-10-20T19:17:15.423899] [DEBUG] [VALIDATOR ] HUM: Primary data validation passed. [2025-10-20T19:17:15.424823] [INFO ] [MAIN ] HUM: Fetched primary data is valid. [2025-10-20T19:17:15.424888] [DEBUG] [LOCAL ] HUM: Saving primary data. [2025-10-20T19:17:15.426207] [DEBUG] [LOCAL ] HUM: Trimmed 71139 records to 49347 (keeping 200 days of data) [2025-10-20T19:17:15.480880] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-07. [2025-10-20T19:17:15.735460] [DEBUG] [LOCAL ] HUM: Primary data saved (49347 records). [2025-10-20T19:17:15.735783] [INFO ] [MAIN ] HUM: Full history saved. [2025-10-20T19:17:15.735906] [INFO ] [MAIN ] Update Progress: 1066/2367 symbols completed. HUM took 36.1s. Estimated remaining: 2h 47m 26s. Rate: 7.8 symbols/min [2025-10-20T19:17:15.915766] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T19:17:15.979764] [INFO ] [MAIN ] HXL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:16.018626] [DEBUG] [LOCAL ] HXL: Loaded 46542 existing records from primary storage. [2025-10-20T19:17:16.019010] [INFO ] [MAIN ] HXL: Loaded 46542 existing records. [2025-10-20T19:17:16.195593] [WARN ] [MAIN ] HXL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:16.195807] [INFO ] [MAIN ] HXL: Wiping existing primary data before full refresh. [2025-10-20T19:17:16.195853] [INFO ] [LOCAL ] HXL: Starting deletion of primary data. [2025-10-20T19:17:16.196101] [DEBUG] [LOCAL ] HXL: Deleted primary data file. [2025-10-20T19:17:16.196138] [INFO ] [MAIN ] HXL: Starting full primary refresh. [2025-10-20T19:17:16.263759] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-10. [2025-10-20T19:17:16.554386] [INFO ] [REMOTE ] HXL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:16.583354] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T19:17:17.051397] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-02. [2025-10-20T19:17:17.084184] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:17:17.851366] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-08. [2025-10-20T19:17:18.431330] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-10 (2.17 seconds). [2025-10-20T19:17:18.439506] [INFO ] [REMOTE ] HUN: Full fetch complete (76983 records). [2025-10-20T19:17:18.439739] [DEBUG] [VALIDATOR ] HUN: Validating primary data. [2025-10-20T19:17:18.659696] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-01. [2025-10-20T19:17:18.793317] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:17:18.892129] [DEBUG] [VALIDATOR ] HUN: All expected trading days are present. [2025-10-20T19:17:18.892218] [DEBUG] [VALIDATOR ] HUN: Primary data validation passed. [2025-10-20T19:17:18.893175] [INFO ] [MAIN ] HUN: Fetched primary data is valid. [2025-10-20T19:17:18.893240] [DEBUG] [LOCAL ] HUN: Saving primary data. [2025-10-20T19:17:18.894786] [DEBUG] [LOCAL ] HUN: Trimmed 76983 records to 53219 (keeping 200 days of data) [2025-10-20T19:17:19.208259] [DEBUG] [LOCAL ] HUN: Primary data saved (53219 records). [2025-10-20T19:17:19.208383] [INFO ] [MAIN ] HUN: Full history saved. [2025-10-20T19:17:19.208457] [INFO ] [MAIN ] Update Progress: 1067/2367 symbols completed. HUN took 35.4s. Estimated remaining: 2h 47m 13s. Rate: 7.8 symbols/min [2025-10-20T19:17:19.381567] [INFO ] [MAIN ] HY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:19.392862] [DEBUG] [LOCAL ] HY: Loaded 10191 existing records from primary storage. [2025-10-20T19:17:19.393037] [INFO ] [MAIN ] HY: Loaded 10191 existing records. [2025-10-20T19:17:19.456236] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-03. [2025-10-20T19:17:19.605200] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T19:17:19.609004] [WARN ] [MAIN ] HY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:19.618480] [INFO ] [MAIN ] HY: Wiping existing primary data before full refresh. [2025-10-20T19:17:19.618678] [INFO ] [LOCAL ] HY: Starting deletion of primary data. [2025-10-20T19:17:19.619129] [DEBUG] [LOCAL ] HY: Deleted primary data file. [2025-10-20T19:17:19.619191] [INFO ] [MAIN ] HY: Starting full primary refresh. [2025-10-20T19:17:19.970512] [INFO ] [REMOTE ] HY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:20.251342] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-01. [2025-10-20T19:17:20.297629] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:17:20.837699] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T19:17:21.051398] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-02. [2025-10-20T19:17:21.106711] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:17:21.851365] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-04. [2025-10-20T19:17:21.876577] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T19:17:22.651415] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-03. [2025-10-20T19:17:23.451392] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-09. [2025-10-20T19:17:23.664298] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:17:23.698400] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:17:24.251370] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-05. [2025-10-20T19:17:25.041018] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:17:25.051342] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-02. [2025-10-20T19:17:25.841220] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T19:17:25.851699] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-04. [2025-10-20T19:17:26.392607] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T19:17:26.656195] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-03. [2025-10-20T19:17:26.658949] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T19:17:27.451376] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-10. [2025-10-20T19:17:28.167687] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T19:17:28.251428] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-04. [2025-10-20T19:17:28.596074] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T19:17:28.608737] [INFO ] [REMOTE ] HUYA: Full fetch complete (63990 records). [2025-10-20T19:17:28.608988] [DEBUG] [VALIDATOR ] HUYA: Validating primary data. [2025-10-20T19:17:29.056551] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-05. [2025-10-20T19:17:29.157809] [DEBUG] [VALIDATOR ] HUYA: All expected trading days are present. [2025-10-20T19:17:29.157921] [DEBUG] [VALIDATOR ] HUYA: Primary data validation passed. [2025-10-20T19:17:29.158973] [INFO ] [MAIN ] HUYA: Fetched primary data is valid. [2025-10-20T19:17:29.159043] [DEBUG] [LOCAL ] HUYA: Saving primary data. [2025-10-20T19:17:29.160611] [DEBUG] [LOCAL ] HUYA: Trimmed 63990 records to 43607 (keeping 200 days of data) [2025-10-20T19:17:29.414277] [DEBUG] [LOCAL ] HUYA: Primary data saved (43607 records). [2025-10-20T19:17:29.414404] [INFO ] [MAIN ] HUYA: Full history saved. [2025-10-20T19:17:29.414481] [INFO ] [MAIN ] Update Progress: 1068/2367 symbols completed. HUYA took 39.8s. Estimated remaining: 2h 47m 8s. Rate: 7.8 symbols/min [2025-10-20T19:17:29.586321] [INFO ] [MAIN ] HYAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:29.592674] [DEBUG] [LOCAL ] HYAC: No existing data file found in primary storage. [2025-10-20T19:17:29.593392] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T19:17:29.593169] [WARN ] [MAIN ] HYAC: No existing data found, performing full refresh instead. [2025-10-20T19:17:29.596539] [INFO ] [MAIN ] HYAC: Wiping existing primary data before full refresh. [2025-10-20T19:17:29.596686] [INFO ] [LOCAL ] HYAC: Starting deletion of primary data. [2025-10-20T19:17:29.596754] [DEBUG] [LOCAL ] HYAC: No primary data file found to delete. [2025-10-20T19:17:29.596791] [INFO ] [MAIN ] HYAC: Starting full primary refresh. [2025-10-20T19:17:29.854939] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-06. [2025-10-20T19:17:29.966820] [INFO ] [REMOTE ] HYAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:30.007440] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T19:17:30.651367] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-05. [2025-10-20T19:17:31.451392] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-06. [2025-10-20T19:17:31.887372] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T19:17:31.923924] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T19:17:32.251414] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-07. [2025-10-20T19:17:32.330368] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T19:17:33.051376] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-06. [2025-10-20T19:17:33.851365] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-07. [2025-10-20T19:17:34.637272] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T19:17:34.651348] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-08. [2025-10-20T19:17:34.860304] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-07 (2.61 seconds). [2025-10-20T19:17:35.097639] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:17:35.175904] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-06 (2.12 seconds). [2025-10-20T19:17:35.451375] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-09. [2025-10-20T19:17:36.249232] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-10-20T19:17:36.253274] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-01. [2025-10-20T19:17:36.586128] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T19:17:37.051384] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-08. [2025-10-20T19:17:37.851379] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-07. [2025-10-20T19:17:38.651396] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-02. [2025-10-20T19:17:38.894339] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T19:17:38.976805] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T19:17:39.450834] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:17:39.460205] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-03. [2025-10-20T19:17:39.685118] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-03 (0.22 seconds). [2025-10-20T19:17:40.251407] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-04. [2025-10-20T19:17:40.577615] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-10-20T19:17:41.051373] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-05. [2025-10-20T19:17:41.364530] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T19:17:41.851362] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-09. [2025-10-20T19:17:42.651380] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-08. [2025-10-20T19:17:43.451380] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-10. [2025-10-20T19:17:43.600067] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:17:44.259676] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-06. [2025-10-20T19:17:44.270343] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T19:17:44.272970] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T19:17:44.283127] [INFO ] [REMOTE ] HY: Full fetch complete (14978 records). [2025-10-20T19:17:44.283625] [DEBUG] [VALIDATOR ] HY: Validating primary data. [2025-10-20T19:17:44.640181] [DEBUG] [VALIDATOR ] HY: All expected trading days are present. [2025-10-20T19:17:44.642502] [DEBUG] [VALIDATOR ] HY: Primary data validation passed. [2025-10-20T19:17:44.642590] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-10-20T19:17:44.643169] [INFO ] [MAIN ] HY: Fetched primary data is valid. [2025-10-20T19:17:44.644198] [DEBUG] [LOCAL ] HY: Saving primary data. [2025-10-20T19:17:44.644766] [DEBUG] [LOCAL ] HY: Trimmed 14978 records to 10046 (keeping 200 days of data) [2025-10-20T19:17:44.702526] [DEBUG] [LOCAL ] HY: Primary data saved (10046 records). [2025-10-20T19:17:44.702670] [INFO ] [MAIN ] HY: Full history saved. [2025-10-20T19:17:44.702764] [INFO ] [MAIN ] Update Progress: 1069/2367 symbols completed. HY took 25.5s. Estimated remaining: 2h 47m 10s. Rate: 7.8 symbols/min [2025-10-20T19:17:44.868622] [INFO ] [MAIN ] HYBX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:44.868841] [DEBUG] [LOCAL ] HYBX: No existing data file found in primary storage. [2025-10-20T19:17:44.868894] [WARN ] [MAIN ] HYBX: No existing data found, performing full refresh instead. [2025-10-20T19:17:44.868999] [INFO ] [MAIN ] HYBX: Wiping existing primary data before full refresh. [2025-10-20T19:17:44.869040] [INFO ] [LOCAL ] HYBX: Starting deletion of primary data. [2025-10-20T19:17:44.869079] [DEBUG] [LOCAL ] HYBX: No primary data file found to delete. [2025-10-20T19:17:44.869110] [INFO ] [MAIN ] HYBX: Starting full primary refresh. [2025-10-20T19:17:45.056703] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-10. [2025-10-20T19:17:45.221876] [INFO ] [REMOTE ] HYBX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:45.851366] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-01. [2025-10-20T19:17:46.185347] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T19:17:46.651366] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-09. [2025-10-20T19:17:47.204748] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T19:17:47.212247] [INFO ] [REMOTE ] HWM: Full fetch complete (76649 records). [2025-10-20T19:17:47.212411] [DEBUG] [VALIDATOR ] HWM: Validating primary data. [2025-10-20T19:17:47.462505] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-02. [2025-10-20T19:17:47.603394] [DEBUG] [VALIDATOR ] HWM: All expected trading days are present. [2025-10-20T19:17:47.603477] [DEBUG] [VALIDATOR ] HWM: Primary data validation passed. [2025-10-20T19:17:47.604417] [INFO ] [MAIN ] HWM: Fetched primary data is valid. [2025-10-20T19:17:47.604479] [DEBUG] [LOCAL ] HWM: Saving primary data. [2025-10-20T19:17:47.605885] [DEBUG] [LOCAL ] HWM: Trimmed 76649 records to 52717 (keeping 200 days of data) [2025-10-20T19:17:47.822881] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T19:17:47.934495] [DEBUG] [LOCAL ] HWM: Primary data saved (52717 records). [2025-10-20T19:17:47.934662] [INFO ] [MAIN ] HWM: Full history saved. [2025-10-20T19:17:47.934808] [INFO ] [MAIN ] Update Progress: 1070/2367 symbols completed. HWM took 34.1s. Estimated remaining: 2h 46m 57s. Rate: 7.8 symbols/min [2025-10-20T19:17:48.102601] [INFO ] [MAIN ] HYI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:48.114429] [DEBUG] [LOCAL ] HYI: Loaded 9962 existing records from primary storage. [2025-10-20T19:17:48.114700] [INFO ] [MAIN ] HYI: Loaded 9962 existing records. [2025-10-20T19:17:48.255358] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-03. [2025-10-20T19:17:48.295944] [WARN ] [MAIN ] HYI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:48.296426] [INFO ] [MAIN ] HYI: Wiping existing primary data before full refresh. [2025-10-20T19:17:48.296633] [INFO ] [LOCAL ] HYI: Starting deletion of primary data. [2025-10-20T19:17:48.297042] [DEBUG] [LOCAL ] HYI: Deleted primary data file. [2025-10-20T19:17:48.297208] [INFO ] [MAIN ] HYI: Starting full primary refresh. [2025-10-20T19:17:48.445342] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T19:17:48.657997] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T19:17:48.719976] [INFO ] [REMOTE ] HYI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:49.051368] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-10. [2025-10-20T19:17:49.851427] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-01. [2025-10-20T19:17:50.576994] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:17:50.651379] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-04. [2025-10-20T19:17:50.654395] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T19:17:50.664747] [INFO ] [REMOTE ] HXL: Full fetch complete (66634 records). [2025-10-20T19:17:50.664933] [DEBUG] [VALIDATOR ] HXL: Validating primary data. [2025-10-20T19:17:51.041601] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-10-20T19:17:51.059666] [DEBUG] [VALIDATOR ] HXL: All expected trading days are present. [2025-10-20T19:17:51.059726] [DEBUG] [VALIDATOR ] HXL: Primary data validation passed. [2025-10-20T19:17:51.060552] [INFO ] [MAIN ] HXL: Fetched primary data is valid. [2025-10-20T19:17:51.060591] [DEBUG] [LOCAL ] HXL: Saving primary data. [2025-10-20T19:17:51.061799] [DEBUG] [LOCAL ] HXL: Trimmed 66634 records to 45814 (keeping 200 days of data) [2025-10-20T19:17:51.328961] [DEBUG] [LOCAL ] HXL: Primary data saved (45814 records). [2025-10-20T19:17:51.329079] [INFO ] [MAIN ] HXL: Full history saved. [2025-10-20T19:17:51.329153] [INFO ] [MAIN ] Update Progress: 1071/2367 symbols completed. HXL took 35.6s. Estimated remaining: 2h 46m 44s. Rate: 7.8 symbols/min [2025-10-20T19:17:51.466139] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-02. [2025-10-20T19:17:51.496457] [INFO ] [MAIN ] HYT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:51.528812] [DEBUG] [LOCAL ] HYT: Loaded 34156 existing records from primary storage. [2025-10-20T19:17:51.529080] [INFO ] [MAIN ] HYT: Loaded 34156 existing records. [2025-10-20T19:17:51.705425] [WARN ] [MAIN ] HYT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:51.705621] [INFO ] [MAIN ] HYT: Wiping existing primary data before full refresh. [2025-10-20T19:17:51.705700] [INFO ] [LOCAL ] HYT: Starting deletion of primary data. [2025-10-20T19:17:51.705944] [DEBUG] [LOCAL ] HYT: Deleted primary data file. [2025-10-20T19:17:51.706000] [INFO ] [MAIN ] HYT: Starting full primary refresh. [2025-10-20T19:17:52.087335] [INFO ] [REMOTE ] HYT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:52.251376] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-05. [2025-10-20T19:17:52.261594] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T19:17:52.557379] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T19:17:53.051379] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-03. [2025-10-20T19:17:53.851371] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-07. [2025-10-20T19:17:53.869102] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T19:17:54.278833] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T19:17:54.651418] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-06. [2025-10-20T19:17:55.102171] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T19:17:55.451371] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-04. [2025-10-20T19:17:56.251374] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-01. [2025-10-20T19:17:57.051392] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-07. [2025-10-20T19:17:57.399145] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T19:17:57.407763] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T19:17:57.851334] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-05. [2025-10-20T19:17:58.267865] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T19:17:58.609008] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T19:17:58.651336] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-02. [2025-10-20T19:17:59.451386] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-06. [2025-10-20T19:17:59.898152] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T19:18:00.137690] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T19:18:00.251391] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-07. [2025-10-20T19:18:01.051349] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-03. [2025-10-20T19:18:01.209807] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T19:18:01.851372] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-08. [2025-10-20T19:18:02.095751] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-08 (0.24 seconds). [2025-10-20T19:18:02.468218] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T19:18:02.651382] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-09. [2025-10-20T19:18:02.887883] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-09 (0.24 seconds). [2025-10-20T19:18:03.451367] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-04. [2025-10-20T19:18:04.251393] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-08. [2025-10-20T19:18:04.961537] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:18:04.977808] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T19:18:05.051381] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-08. [2025-10-20T19:18:05.365252] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-10-20T19:18:05.851396] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-10. [2025-10-20T19:18:06.276589] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-10-20T19:18:06.277568] [INFO ] [REMOTE ] HYAC: Full fetch complete (923 records). [2025-10-20T19:18:06.277637] [DEBUG] [VALIDATOR ] HYAC: Validating primary data. [2025-10-20T19:18:06.453514] [WARN ] [VALIDATOR ] HYAC: Validation failed - insufficient historical data. Have 166 unique days, need 200 days [2025-10-20T19:18:06.453705] [FATAL] [MAIN ] HYAC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:18:06.453759] [WARN ] [MAIN ] HYAC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:18:06.453799] [INFO ] [LOCAL ] HYAC: Starting deletion of primary data. [2025-10-20T19:18:06.453872] [DEBUG] [LOCAL ] HYAC: No primary data file found to delete. [2025-10-20T19:18:06.453907] [INFO ] [LOCAL ] HYAC: Starting deletion of derivative data. [2025-10-20T19:18:06.453946] [DEBUG] [LOCAL ] HYAC: No derivative data file found to delete. [2025-10-20T19:18:06.453998] [INFO ] [MAIN ] Update Progress: 1072/2367 symbols completed. HYAC took 37.0s. Estimated remaining: 2h 46m 45s. Rate: 7.8 symbols/min [2025-10-20T19:18:06.625419] [INFO ] [MAIN ] HYTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:06.630068] [DEBUG] [LOCAL ] HYTR: Loaded 3189 existing records from primary storage. [2025-10-20T19:18:06.630234] [INFO ] [MAIN ] HYTR: Loaded 3189 existing records. [2025-10-20T19:18:06.658796] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-09. [2025-10-20T19:18:06.809829] [WARN ] [MAIN ] HYTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:06.810065] [INFO ] [MAIN ] HYTR: Wiping existing primary data before full refresh. [2025-10-20T19:18:06.810122] [INFO ] [LOCAL ] HYTR: Starting deletion of primary data. [2025-10-20T19:18:06.810411] [DEBUG] [LOCAL ] HYTR: Deleted primary data file. [2025-10-20T19:18:06.810469] [INFO ] [MAIN ] HYTR: Starting full primary refresh. [2025-10-20T19:18:07.097992] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T19:18:07.177553] [INFO ] [REMOTE ] HYTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:07.451449] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-10. [2025-10-20T19:18:07.808372] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T19:18:07.809317] [INFO ] [REMOTE ] HYBX: Full fetch complete (704 records). [2025-10-20T19:18:07.809396] [DEBUG] [VALIDATOR ] HYBX: Validating primary data. [2025-10-20T19:18:08.003456] [WARN ] [VALIDATOR ] HYBX: Validation failed - insufficient historical data. Have 179 unique days, need 200 days [2025-10-20T19:18:08.003732] [FATAL] [MAIN ] HYBX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:18:08.003803] [WARN ] [MAIN ] HYBX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:18:08.003875] [INFO ] [LOCAL ] HYBX: Starting deletion of primary data. [2025-10-20T19:18:08.003972] [DEBUG] [LOCAL ] HYBX: No primary data file found to delete. [2025-10-20T19:18:08.004028] [INFO ] [LOCAL ] HYBX: Starting deletion of derivative data. [2025-10-20T19:18:08.004071] [DEBUG] [LOCAL ] HYBX: No derivative data file found to delete. [2025-10-20T19:18:08.004150] [INFO ] [MAIN ] Update Progress: 1073/2367 symbols completed. HYBX took 23.3s. Estimated remaining: 2h 46m 30s. Rate: 7.8 symbols/min [2025-10-20T19:18:08.170895] [INFO ] [MAIN ] HZO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:08.199281] [DEBUG] [LOCAL ] HZO: Loaded 29442 existing records from primary storage. [2025-10-20T19:18:08.199571] [INFO ] [MAIN ] HZO: Loaded 29442 existing records. [2025-10-20T19:18:08.260414] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-01. [2025-10-20T19:18:08.379839] [WARN ] [MAIN ] HZO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:08.380049] [INFO ] [MAIN ] HZO: Wiping existing primary data before full refresh. [2025-10-20T19:18:08.380099] [INFO ] [LOCAL ] HZO: Starting deletion of primary data. [2025-10-20T19:18:08.380448] [DEBUG] [LOCAL ] HZO: Deleted primary data file. [2025-10-20T19:18:08.380506] [INFO ] [MAIN ] HZO: Starting full primary refresh. [2025-10-20T19:18:08.752268] [INFO ] [REMOTE ] HZO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:08.782966] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T19:18:09.051421] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-05. [2025-10-20T19:18:09.851384] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-01. [2025-10-20T19:18:10.420809] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:18:10.651353] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-02. [2025-10-20T19:18:10.994839] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:18:11.197970] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T19:18:11.451363] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-03. [2025-10-20T19:18:12.005155] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T19:18:12.251379] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-02. [2025-10-20T19:18:13.051344] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-06. [2025-10-20T19:18:13.365311] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:18:13.851409] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-09. [2025-10-20T19:18:14.171896] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:18:14.600384] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T19:18:14.651401] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-07. [2025-10-20T19:18:15.451384] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-03. [2025-10-20T19:18:16.230167] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:18:16.251335] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-08. [2025-10-20T19:18:16.826017] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T19:18:17.051583] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-04. [2025-10-20T19:18:17.386919] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-10-20T19:18:17.755038] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:18:17.851347] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-09. [2025-10-20T19:18:18.651380] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-10. [2025-10-20T19:18:19.277494] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T19:18:19.451346] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-10. [2025-10-20T19:18:19.551959] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T19:18:19.554380] [INFO ] [REMOTE ] HYI: Full fetch complete (13967 records). [2025-10-20T19:18:19.554457] [DEBUG] [VALIDATOR ] HYI: Validating primary data. [2025-10-20T19:18:19.911128] [DEBUG] [VALIDATOR ] HYI: All expected trading days are present. [2025-10-20T19:18:19.911203] [DEBUG] [VALIDATOR ] HYI: Primary data validation passed. [2025-10-20T19:18:19.911495] [INFO ] [MAIN ] HYI: Fetched primary data is valid. [2025-10-20T19:18:19.911533] [DEBUG] [LOCAL ] HYI: Saving primary data. [2025-10-20T19:18:19.912039] [DEBUG] [LOCAL ] HYI: Trimmed 13967 records to 9771 (keeping 200 days of data) [2025-10-20T19:18:19.970342] [DEBUG] [LOCAL ] HYI: Primary data saved (9771 records). [2025-10-20T19:18:19.970478] [INFO ] [MAIN ] HYI: Full history saved. [2025-10-20T19:18:19.970553] [INFO ] [MAIN ] Update Progress: 1074/2367 symbols completed. HYI took 32.0s. Estimated remaining: 2h 46m 27s. Rate: 7.8 symbols/min [2025-10-20T19:18:20.140266] [INFO ] [MAIN ] IAE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:20.147884] [DEBUG] [LOCAL ] IAE: Loaded 6362 existing records from primary storage. [2025-10-20T19:18:20.148115] [INFO ] [MAIN ] IAE: Loaded 6362 existing records. [2025-10-20T19:18:20.256521] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-05. [2025-10-20T19:18:20.326245] [WARN ] [MAIN ] IAE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:20.326420] [INFO ] [MAIN ] IAE: Wiping existing primary data before full refresh. [2025-10-20T19:18:20.326465] [INFO ] [LOCAL ] IAE: Starting deletion of primary data. [2025-10-20T19:18:20.326633] [DEBUG] [LOCAL ] IAE: Deleted primary data file. [2025-10-20T19:18:20.326710] [INFO ] [MAIN ] IAE: Starting full primary refresh. [2025-10-20T19:18:20.681358] [INFO ] [REMOTE ] IAE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:20.793007] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T19:18:20.809544] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T19:18:20.814990] [INFO ] [REMOTE ] HYT: Full fetch complete (47953 records). [2025-10-20T19:18:20.815131] [DEBUG] [VALIDATOR ] HYT: Validating primary data. [2025-10-20T19:18:21.056608] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-04. [2025-10-20T19:18:21.208952] [DEBUG] [VALIDATOR ] HYT: All expected trading days are present. [2025-10-20T19:18:21.209042] [DEBUG] [VALIDATOR ] HYT: Primary data validation passed. [2025-10-20T19:18:21.209820] [INFO ] [MAIN ] HYT: Fetched primary data is valid. [2025-10-20T19:18:21.209885] [DEBUG] [LOCAL ] HYT: Saving primary data. [2025-10-20T19:18:21.210995] [DEBUG] [LOCAL ] HYT: Trimmed 47953 records to 33697 (keeping 200 days of data) [2025-10-20T19:18:21.412977] [DEBUG] [LOCAL ] HYT: Primary data saved (33697 records). [2025-10-20T19:18:21.413118] [INFO ] [MAIN ] HYT: Full history saved. [2025-10-20T19:18:21.413198] [INFO ] [MAIN ] Update Progress: 1075/2367 symbols completed. HYT took 30.1s. Estimated remaining: 2h 46m 12s. Rate: 7.8 symbols/min [2025-10-20T19:18:21.583711] [INFO ] [MAIN ] IAG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:21.634275] [DEBUG] [LOCAL ] IAG: Loaded 54268 existing records from primary storage. [2025-10-20T19:18:21.634626] [INFO ] [MAIN ] IAG: Loaded 54268 existing records. [2025-10-20T19:18:21.813241] [WARN ] [MAIN ] IAG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:21.813448] [INFO ] [MAIN ] IAG: Wiping existing primary data before full refresh. [2025-10-20T19:18:21.813493] [INFO ] [LOCAL ] IAG: Starting deletion of primary data. [2025-10-20T19:18:21.813738] [DEBUG] [LOCAL ] IAG: Deleted primary data file. [2025-10-20T19:18:21.813794] [INFO ] [MAIN ] IAG: Starting full primary refresh. [2025-10-20T19:18:21.856687] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-01. [2025-10-20T19:18:22.190182] [INFO ] [REMOTE ] IAG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:22.536330] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T19:18:22.560200] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T19:18:22.651341] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-05. [2025-10-20T19:18:23.451397] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-01. [2025-10-20T19:18:23.885234] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T19:18:24.251376] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-02. [2025-10-20T19:18:24.868950] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T19:18:25.051403] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-03. [2025-10-20T19:18:25.260681] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T19:18:25.737287] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T19:18:25.851394] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-06. [2025-10-20T19:18:26.651401] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-06. [2025-10-20T19:18:26.799912] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T19:18:27.211446] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T19:18:27.451414] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-07. [2025-10-20T19:18:28.251436] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-07. [2025-10-20T19:18:28.766843] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T19:18:28.790685] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T19:18:29.051464] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-08. [2025-10-20T19:18:29.851406] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-04. [2025-10-20T19:18:30.335880] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T19:18:30.432407] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T19:18:30.651415] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-02. [2025-10-20T19:18:31.451461] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-08. [2025-10-20T19:18:31.900353] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:18:32.252056] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-09. [2025-10-20T19:18:32.257839] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T19:18:33.051393] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-03. [2025-10-20T19:18:33.401606] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:18:33.851440] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-05. [2025-10-20T19:18:34.462733] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T19:18:34.651344] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-09. [2025-10-20T19:18:34.797982] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:18:35.451372] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-10. [2025-10-20T19:18:36.251397] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-04. [2025-10-20T19:18:36.707610] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T19:18:36.711969] [INFO ] [REMOTE ] HZO: Full fetch complete (41270 records). [2025-10-20T19:18:36.712099] [DEBUG] [VALIDATOR ] HZO: Validating primary data. [2025-10-20T19:18:37.058603] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-06. [2025-10-20T19:18:37.103555] [DEBUG] [VALIDATOR ] HZO: All expected trading days are present. [2025-10-20T19:18:37.103684] [DEBUG] [VALIDATOR ] HZO: Primary data validation passed. [2025-10-20T19:18:37.104451] [INFO ] [MAIN ] HZO: Fetched primary data is valid. [2025-10-20T19:18:37.104516] [DEBUG] [LOCAL ] HZO: Saving primary data. [2025-10-20T19:18:37.105542] [DEBUG] [LOCAL ] HZO: Trimmed 41270 records to 28914 (keeping 200 days of data) [2025-10-20T19:18:37.288373] [DEBUG] [LOCAL ] HZO: Primary data saved (28914 records). [2025-10-20T19:18:37.288483] [INFO ] [MAIN ] HZO: Full history saved. [2025-10-20T19:18:37.288558] [INFO ] [MAIN ] Update Progress: 1076/2367 symbols completed. HZO took 29.3s. Estimated remaining: 2h 46m 14s. Rate: 7.8 symbols/min [2025-10-20T19:18:37.481161] [INFO ] [MAIN ] IBM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:37.535868] [DEBUG] [LOCAL ] IBM: Loaded 54411 existing records from primary storage. [2025-10-20T19:18:37.536269] [INFO ] [MAIN ] IBM: Loaded 54411 existing records. [2025-10-20T19:18:37.762491] [WARN ] [MAIN ] IBM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:37.762789] [INFO ] [MAIN ] IBM: Wiping existing primary data before full refresh. [2025-10-20T19:18:37.762864] [INFO ] [LOCAL ] IBM: Starting deletion of primary data. [2025-10-20T19:18:37.763135] [DEBUG] [LOCAL ] IBM: Deleted primary data file. [2025-10-20T19:18:37.763202] [INFO ] [MAIN ] IBM: Starting full primary refresh. [2025-10-20T19:18:37.909318] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T19:18:37.933242] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-07. [2025-10-20T19:18:38.310749] [INFO ] [REMOTE ] IBM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:38.366711] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T19:18:38.649898] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T19:18:38.651585] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-01. [2025-10-20T19:18:39.451426] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-08. [2025-10-20T19:18:40.061735] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:18:40.251444] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-05. [2025-10-20T19:18:40.801299] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-01 (2.15 seconds). [2025-10-20T19:18:41.051430] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-09. [2025-10-20T19:18:41.826478] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:18:41.851343] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-10. [2025-10-20T19:18:41.882983] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-09 (7.23 seconds). [2025-10-20T19:18:42.092771] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T19:18:42.651463] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-06. [2025-10-20T19:18:42.689980] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T19:18:42.692140] [INFO ] [REMOTE ] IAE: Full fetch complete (9029 records). [2025-10-20T19:18:42.692218] [DEBUG] [VALIDATOR ] IAE: Validating primary data. [2025-10-20T19:18:43.086858] [DEBUG] [VALIDATOR ] IAE: All expected trading days are present. [2025-10-20T19:18:43.086970] [DEBUG] [VALIDATOR ] IAE: Primary data validation passed. [2025-10-20T19:18:43.087361] [INFO ] [MAIN ] IAE: Fetched primary data is valid. [2025-10-20T19:18:43.087407] [DEBUG] [LOCAL ] IAE: Saving primary data. [2025-10-20T19:18:43.087942] [DEBUG] [LOCAL ] IAE: Trimmed 9029 records to 6325 (keeping 200 days of data) [2025-10-20T19:18:43.127281] [DEBUG] [LOCAL ] IAE: Primary data saved (6325 records). [2025-10-20T19:18:43.127468] [INFO ] [MAIN ] IAE: Full history saved. [2025-10-20T19:18:43.127566] [INFO ] [MAIN ] Update Progress: 1077/2367 symbols completed. IAE took 23.2s. Estimated remaining: 2h 46m 4s. Rate: 7.8 symbols/min [2025-10-20T19:18:43.451371] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-10. [2025-10-20T19:18:44.021529] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T19:18:44.022588] [INFO ] [REMOTE ] HYTR: Full fetch complete (4665 records). [2025-10-20T19:18:44.022690] [DEBUG] [VALIDATOR ] HYTR: Validating primary data. [2025-10-20T19:18:44.251733] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-02. [2025-10-20T19:18:44.400754] [DEBUG] [VALIDATOR ] HYTR: All expected trading days are present. [2025-10-20T19:18:44.400857] [DEBUG] [VALIDATOR ] HYTR: Primary data validation passed. [2025-10-20T19:18:44.401074] [INFO ] [MAIN ] HYTR: Fetched primary data is valid. [2025-10-20T19:18:44.401184] [DEBUG] [LOCAL ] HYTR: Saving primary data. [2025-10-20T19:18:44.430729] [DEBUG] [LOCAL ] HYTR: Trimmed 4665 records to 3098 (keeping 200 days of data) [2025-10-20T19:18:44.446995] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T19:18:44.468394] [DEBUG] [LOCAL ] HYTR: Primary data saved (3098 records). [2025-10-20T19:18:44.468516] [INFO ] [MAIN ] HYTR: Full history saved. [2025-10-20T19:18:44.468591] [INFO ] [MAIN ] Update Progress: 1078/2367 symbols completed. HYTR took 38.0s. Estimated remaining: 2h 45m 49s. Rate: 7.8 symbols/min [2025-10-20T19:18:45.051398] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-07. [2025-10-20T19:18:46.071459] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T19:18:46.081496] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-03. [2025-10-20T19:18:47.015622] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T19:18:47.028100] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-08. [2025-10-20T19:18:48.261625] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-10-20T19:18:48.272849] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-04. [2025-10-20T19:18:48.894558] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T19:18:48.905889] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-09. [2025-10-20T19:18:50.449305] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T19:18:50.460432] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-05. [2025-10-20T19:18:50.908117] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T19:18:50.919624] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-10. [2025-10-20T19:18:52.370791] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:18:52.382228] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-06. [2025-10-20T19:18:52.545418] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T19:18:52.553355] [INFO ] [REMOTE ] IAG: Full fetch complete (77503 records). [2025-10-20T19:18:52.553481] [DEBUG] [VALIDATOR ] IAG: Validating primary data. [2025-10-20T19:18:52.945281] [DEBUG] [VALIDATOR ] IAG: All expected trading days are present. [2025-10-20T19:18:52.945357] [DEBUG] [VALIDATOR ] IAG: Primary data validation passed. [2025-10-20T19:18:52.946297] [INFO ] [MAIN ] IAG: Fetched primary data is valid. [2025-10-20T19:18:52.946373] [DEBUG] [LOCAL ] IAG: Saving primary data. [2025-10-20T19:18:52.947744] [DEBUG] [LOCAL ] IAG: Trimmed 77503 records to 53491 (keeping 200 days of data) [2025-10-20T19:18:53.248784] [DEBUG] [LOCAL ] IAG: Primary data saved (53491 records). [2025-10-20T19:18:53.248954] [INFO ] [MAIN ] IAG: Full history saved. [2025-10-20T19:18:53.249046] [INFO ] [MAIN ] Update Progress: 1079/2367 symbols completed. IAG took 31.8s. Estimated remaining: 2h 45m 42s. Rate: 7.8 symbols/min [2025-10-20T19:18:54.552146] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T19:18:54.563167] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-07. [2025-10-20T19:18:57.002978] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-07 (2.44 seconds). [2025-10-20T19:18:57.014957] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-08. [2025-10-20T19:18:59.537836] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-08 (2.52 seconds). [2025-10-20T19:18:59.549499] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-09. [2025-10-20T19:19:02.188353] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-09 (2.64 seconds). [2025-10-20T19:19:02.200902] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-10. [2025-10-20T19:19:03.062445] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:19:03.070501] [INFO ] [REMOTE ] IBM: Full fetch complete (77740 records). [2025-10-20T19:19:03.070636] [DEBUG] [VALIDATOR ] IBM: Validating primary data. [2025-10-20T19:19:03.468721] [DEBUG] [VALIDATOR ] IBM: All expected trading days are present. [2025-10-20T19:19:03.468838] [DEBUG] [VALIDATOR ] IBM: Primary data validation passed. [2025-10-20T19:19:03.469833] [INFO ] [MAIN ] IBM: Fetched primary data is valid. [2025-10-20T19:19:03.469895] [DEBUG] [LOCAL ] IBM: Saving primary data. [2025-10-20T19:19:03.471355] [DEBUG] [LOCAL ] IBM: Trimmed 77740 records to 53629 (keeping 200 days of data) [2025-10-20T19:19:03.801190] [DEBUG] [LOCAL ] IBM: Primary data saved (53629 records). [2025-10-20T19:19:03.801338] [INFO ] [MAIN ] IBM: Full history saved. [2025-10-20T19:19:03.801429] [INFO ] [MAIN ] Update Progress: 1080/2367 symbols completed. IBM took 26.5s. Estimated remaining: 2h 45m 38s. Rate: 7.8 symbols/min [2025-10-20T19:19:04.522816] [INFO ] [MAIN ] ICE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:04.541357] [INFO ] [MAIN ] IBTA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:04.565581] [INFO ] [MAIN ] IBP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:04.559585] [INFO ] [MAIN ] IBN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:04.608588] [DEBUG] [LOCAL ] IBTA: Loaded 37129 existing records from primary storage. [2025-10-20T19:19:04.609077] [INFO ] [MAIN ] IBTA: Loaded 37129 existing records. [2025-10-20T19:19:04.700539] [DEBUG] [LOCAL ] IBP: Loaded 29142 existing records from primary storage. [2025-10-20T19:19:04.721945] [INFO ] [MAIN ] IBP: Loaded 29142 existing records. [2025-10-20T19:19:04.871191] [DEBUG] [LOCAL ] ICE: Loaded 53798 existing records from primary storage. [2025-10-20T19:19:04.915282] [INFO ] [MAIN ] ICE: Loaded 53798 existing records. [2025-10-20T19:19:04.944265] [DEBUG] [LOCAL ] IBN: Loaded 53593 existing records from primary storage. [2025-10-20T19:19:04.958403] [WARN ] [MAIN ] IBTA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:05.009861] [INFO ] [MAIN ] IBTA: Wiping existing primary data before full refresh. [2025-10-20T19:19:05.035052] [INFO ] [LOCAL ] IBTA: Starting deletion of primary data. [2025-10-20T19:19:05.004291] [INFO ] [MAIN ] IBN: Loaded 53593 existing records. [2025-10-20T19:19:05.104086] [DEBUG] [LOCAL ] IBTA: Deleted primary data file. [2025-10-20T19:19:05.195459] [INFO ] [MAIN ] IBTA: Starting full primary refresh. [2025-10-20T19:19:05.208956] [WARN ] [MAIN ] IBP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:05.228599] [INFO ] [MAIN ] IBP: Wiping existing primary data before full refresh. [2025-10-20T19:19:05.260111] [INFO ] [LOCAL ] IBP: Starting deletion of primary data. [2025-10-20T19:19:05.285882] [DEBUG] [LOCAL ] IBP: Deleted primary data file. [2025-10-20T19:19:05.350845] [INFO ] [MAIN ] IBP: Starting full primary refresh. [2025-10-20T19:19:05.448821] [WARN ] [MAIN ] ICE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:05.518148] [INFO ] [MAIN ] ICE: Wiping existing primary data before full refresh. [2025-10-20T19:19:05.581535] [INFO ] [LOCAL ] ICE: Starting deletion of primary data. [2025-10-20T19:19:05.576138] [WARN ] [MAIN ] IBN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:05.610167] [INFO ] [MAIN ] IBN: Wiping existing primary data before full refresh. [2025-10-20T19:19:05.681705] [INFO ] [LOCAL ] IBN: Starting deletion of primary data. [2025-10-20T19:19:05.686832] [DEBUG] [LOCAL ] ICE: Deleted primary data file. [2025-10-20T19:19:05.699033] [DEBUG] [LOCAL ] IBN: Deleted primary data file. [2025-10-20T19:19:05.699166] [INFO ] [MAIN ] ICE: Starting full primary refresh. [2025-10-20T19:19:05.724823] [INFO ] [MAIN ] IBN: Starting full primary refresh. [2025-10-20T19:19:06.485001] [INFO ] [REMOTE ] IBTA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:06.568711] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-01. [2025-10-20T19:19:06.845040] [INFO ] [REMOTE ] IBP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:06.885223] [INFO ] [REMOTE ] IBN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:06.896213] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-01. [2025-10-20T19:19:06.895794] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-01. [2025-10-20T19:19:06.928679] [INFO ] [REMOTE ] ICE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:06.929282] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-01. [2025-10-20T19:19:08.331616] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:19:08.337456] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-02. [2025-10-20T19:19:08.478835] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:19:08.484006] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-02. [2025-10-20T19:19:08.762631] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T19:19:08.773394] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-02. [2025-10-20T19:19:09.044798] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-01 (2.12 seconds). [2025-10-20T19:19:09.055776] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-02. [2025-10-20T19:19:09.146386] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T19:19:09.151309] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-03. [2025-10-20T19:19:09.695569] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T19:19:09.701544] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-03. [2025-10-20T19:19:10.485343] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T19:19:10.497309] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-03. [2025-10-20T19:19:10.528903] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:19:10.540274] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-03. [2025-10-20T19:19:10.639459] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:19:10.649350] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-04. [2025-10-20T19:19:10.925183] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T19:19:10.932415] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-04. [2025-10-20T19:19:12.195830] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T19:19:12.206105] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-05. [2025-10-20T19:19:12.267965] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T19:19:12.275039] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-05. [2025-10-20T19:19:12.337186] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T19:19:12.348202] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-04. [2025-10-20T19:19:12.646208] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T19:19:12.657378] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-04. [2025-10-20T19:19:13.613091] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T19:19:13.650018] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-06. [2025-10-20T19:19:13.651327] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:19:13.662258] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-06. [2025-10-20T19:19:14.342580] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T19:19:14.354812] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-05. [2025-10-20T19:19:14.430134] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:19:14.441718] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-05. [2025-10-20T19:19:14.663166] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T19:19:14.669409] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-07. [2025-10-20T19:19:14.913869] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T19:19:14.920159] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-07. [2025-10-20T19:19:15.634603] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T19:19:15.641168] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-08. [2025-10-20T19:19:15.834082] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:19:15.845728] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-06. [2025-10-20T19:19:16.356570] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T19:19:16.368354] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-06. [2025-10-20T19:19:16.391996] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:19:16.399045] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-08. [2025-10-20T19:19:16.972014] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T19:19:17.051367] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-09. [2025-10-20T19:19:17.615007] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T19:19:17.812555] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T19:19:17.851407] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-07. [2025-10-20T19:19:18.063788] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:19:18.541348] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:19:18.651333] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-10. [2025-10-20T19:19:19.317934] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:19:19.451357] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-08. [2025-10-20T19:19:20.112937] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:19:20.118132] [INFO ] [REMOTE ] IBTA: Full fetch complete (51313 records). [2025-10-20T19:19:20.118244] [DEBUG] [VALIDATOR ] IBTA: Validating primary data. [2025-10-20T19:19:20.264709] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-09. [2025-10-20T19:19:20.513411] [DEBUG] [VALIDATOR ] IBTA: All expected trading days are present. [2025-10-20T19:19:20.513491] [DEBUG] [VALIDATOR ] IBTA: Primary data validation passed. [2025-10-20T19:19:20.514310] [INFO ] [MAIN ] IBTA: Fetched primary data is valid. [2025-10-20T19:19:20.514372] [DEBUG] [LOCAL ] IBTA: Saving primary data. [2025-10-20T19:19:20.515396] [DEBUG] [LOCAL ] IBTA: Trimmed 51313 records to 36338 (keeping 200 days of data) [2025-10-20T19:19:20.722942] [DEBUG] [LOCAL ] IBTA: Primary data saved (36338 records). [2025-10-20T19:19:20.723133] [INFO ] [MAIN ] IBTA: Full history saved. [2025-10-20T19:19:20.723228] [INFO ] [MAIN ] Update Progress: 1081/2367 symbols completed. IBTA took 16.9s. Estimated remaining: 2h 45m 41s. Rate: 7.8 symbols/min [2025-10-20T19:19:20.901918] [INFO ] [MAIN ] ICL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:20.938964] [DEBUG] [LOCAL ] ICL: Loaded 39044 existing records from primary storage. [2025-10-20T19:19:20.939275] [INFO ] [MAIN ] ICL: Loaded 39044 existing records. [2025-10-20T19:19:21.056542] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-07. [2025-10-20T19:19:21.130509] [WARN ] [MAIN ] ICL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:21.130831] [INFO ] [MAIN ] ICL: Wiping existing primary data before full refresh. [2025-10-20T19:19:21.130880] [INFO ] [LOCAL ] ICL: Starting deletion of primary data. [2025-10-20T19:19:21.131096] [DEBUG] [LOCAL ] ICL: Deleted primary data file. [2025-10-20T19:19:21.131155] [INFO ] [MAIN ] ICL: Starting full primary refresh. [2025-10-20T19:19:21.538792] [INFO ] [REMOTE ] ICL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:21.534064] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T19:19:21.625900] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:19:21.851367] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-09. [2025-10-20T19:19:22.651344] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-10. [2025-10-20T19:19:22.946308] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:19:23.451366] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-01. [2025-10-20T19:19:23.643669] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T19:19:23.877054] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:19:23.881151] [INFO ] [REMOTE ] IBP: Full fetch complete (41089 records). [2025-10-20T19:19:23.881261] [DEBUG] [VALIDATOR ] IBP: Validating primary data. [2025-10-20T19:19:24.265544] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-08. [2025-10-20T19:19:24.315470] [DEBUG] [VALIDATOR ] IBP: All expected trading days are present. [2025-10-20T19:19:24.315552] [DEBUG] [VALIDATOR ] IBP: Primary data validation passed. [2025-10-20T19:19:24.316239] [INFO ] [MAIN ] IBP: Fetched primary data is valid. [2025-10-20T19:19:24.316300] [DEBUG] [LOCAL ] IBP: Saving primary data. [2025-10-20T19:19:24.317144] [DEBUG] [LOCAL ] IBP: Trimmed 41089 records to 28823 (keeping 200 days of data) [2025-10-20T19:19:24.491825] [DEBUG] [LOCAL ] IBP: Primary data saved (28823 records). [2025-10-20T19:19:24.491953] [INFO ] [MAIN ] IBP: Full history saved. [2025-10-20T19:19:24.492071] [INFO ] [MAIN ] Update Progress: 1082/2367 symbols completed. IBP took 20.7s. Estimated remaining: 2h 45m 29s. Rate: 7.8 symbols/min [2025-10-20T19:19:24.670468] [INFO ] [MAIN ] IDA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:24.707617] [DEBUG] [LOCAL ] IDA: Loaded 33149 existing records from primary storage. [2025-10-20T19:19:24.707908] [INFO ] [MAIN ] IDA: Loaded 33149 existing records. [2025-10-20T19:19:24.890759] [WARN ] [MAIN ] IDA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:24.891078] [INFO ] [MAIN ] IDA: Wiping existing primary data before full refresh. [2025-10-20T19:19:24.891293] [INFO ] [LOCAL ] IDA: Starting deletion of primary data. [2025-10-20T19:19:24.911780] [DEBUG] [LOCAL ] IDA: Deleted primary data file. [2025-10-20T19:19:24.911889] [INFO ] [MAIN ] IDA: Starting full primary refresh. [2025-10-20T19:19:24.993350] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T19:19:25.061058] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-02. [2025-10-20T19:19:25.273812] [INFO ] [REMOTE ] IDA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:25.851343] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-01. [2025-10-20T19:19:26.085856] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T19:19:26.651356] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-10. [2025-10-20T19:19:26.741253] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T19:19:27.272870] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:19:27.451360] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-03. [2025-10-20T19:19:28.251407] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-02. [2025-10-20T19:19:28.549013] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T19:19:28.556908] [INFO ] [REMOTE ] IBN: Full fetch complete (76628 records). [2025-10-20T19:19:28.557038] [DEBUG] [VALIDATOR ] IBN: Validating primary data. [2025-10-20T19:19:28.969742] [DEBUG] [VALIDATOR ] IBN: All expected trading days are present. [2025-10-20T19:19:28.970007] [DEBUG] [VALIDATOR ] IBN: Primary data validation passed. [2025-10-20T19:19:28.971009] [INFO ] [MAIN ] IBN: Fetched primary data is valid. [2025-10-20T19:19:28.971093] [DEBUG] [LOCAL ] IBN: Saving primary data. [2025-10-20T19:19:28.994685] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:19:29.002994] [DEBUG] [LOCAL ] IBN: Trimmed 76628 records to 52844 (keeping 200 days of data) [2025-10-20T19:19:29.058615] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-04. [2025-10-20T19:19:29.327239] [DEBUG] [LOCAL ] IBN: Primary data saved (52844 records). [2025-10-20T19:19:29.341703] [INFO ] [MAIN ] IBN: Full history saved. [2025-10-20T19:19:29.342330] [INFO ] [MAIN ] Update Progress: 1083/2367 symbols completed. IBN took 25.5s. Estimated remaining: 2h 45m 17s. Rate: 7.8 symbols/min [2025-10-20T19:19:29.342907] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T19:19:29.515115] [INFO ] [MAIN ] IDE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:29.525980] [DEBUG] [LOCAL ] IDE: Loaded 9258 existing records from primary storage. [2025-10-20T19:19:29.526246] [INFO ] [MAIN ] IDE: Loaded 9258 existing records. [2025-10-20T19:19:29.700682] [WARN ] [MAIN ] IDE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:29.700954] [INFO ] [MAIN ] IDE: Wiping existing primary data before full refresh. [2025-10-20T19:19:29.701003] [INFO ] [LOCAL ] IDE: Starting deletion of primary data. [2025-10-20T19:19:29.701161] [DEBUG] [LOCAL ] IDE: Deleted primary data file. [2025-10-20T19:19:29.701198] [INFO ] [MAIN ] IDE: Starting full primary refresh. [2025-10-20T19:19:29.856133] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-09. [2025-10-20T19:19:30.063112] [INFO ] [REMOTE ] IDE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:30.616245] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T19:19:30.651432] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-03. [2025-10-20T19:19:31.451377] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-01. [2025-10-20T19:19:31.775338] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:19:31.806193] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T19:19:32.239273] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:19:32.251334] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-02. [2025-10-20T19:19:32.952550] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T19:19:33.051357] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-10. [2025-10-20T19:19:33.851424] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-05. [2025-10-20T19:19:34.651355] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-04. [2025-10-20T19:19:35.014631] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T19:19:35.022477] [INFO ] [REMOTE ] ICE: Full fetch complete (76991 records). [2025-10-20T19:19:35.022625] [DEBUG] [VALIDATOR ] ICE: Validating primary data. [2025-10-20T19:19:35.418424] [DEBUG] [VALIDATOR ] ICE: All expected trading days are present. [2025-10-20T19:19:35.418674] [DEBUG] [VALIDATOR ] ICE: Primary data validation passed. [2025-10-20T19:19:35.419702] [INFO ] [MAIN ] ICE: Fetched primary data is valid. [2025-10-20T19:19:35.419806] [DEBUG] [LOCAL ] ICE: Saving primary data. [2025-10-20T19:19:35.451415] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-03. [2025-10-20T19:19:35.452543] [DEBUG] [LOCAL ] ICE: Trimmed 76991 records to 53012 (keeping 200 days of data) [2025-10-20T19:19:35.453885] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T19:19:35.799951] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:19:35.827927] [DEBUG] [LOCAL ] ICE: Primary data saved (53012 records). [2025-10-20T19:19:35.828049] [INFO ] [MAIN ] ICE: Full history saved. [2025-10-20T19:19:35.828122] [INFO ] [MAIN ] Update Progress: 1084/2367 symbols completed. ICE took 32.0s. Estimated remaining: 2h 45m 8s. Rate: 7.8 symbols/min [2025-10-20T19:19:36.004221] [INFO ] [MAIN ] IDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:36.009967] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T19:19:36.027945] [DEBUG] [LOCAL ] IDT: Loaded 18788 existing records from primary storage. [2025-10-20T19:19:36.028173] [INFO ] [MAIN ] IDT: Loaded 18788 existing records. [2025-10-20T19:19:36.209343] [WARN ] [MAIN ] IDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:36.209578] [INFO ] [MAIN ] IDT: Wiping existing primary data before full refresh. [2025-10-20T19:19:36.209626] [INFO ] [LOCAL ] IDT: Starting deletion of primary data. [2025-10-20T19:19:36.209823] [DEBUG] [LOCAL ] IDT: Deleted primary data file. [2025-10-20T19:19:36.209869] [INFO ] [MAIN ] IDT: Starting full primary refresh. [2025-10-20T19:19:36.256785] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-06. [2025-10-20T19:19:36.564993] [INFO ] [REMOTE ] IDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:37.051381] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-04. [2025-10-20T19:19:37.763935] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T19:19:37.820352] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:19:37.851336] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-05. [2025-10-20T19:19:38.367299] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T19:19:38.651370] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-06. [2025-10-20T19:19:39.373632] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T19:19:39.451391] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-07. [2025-10-20T19:19:40.254989] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-07. [2025-10-20T19:19:40.256618] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T19:19:41.051387] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-05. [2025-10-20T19:19:41.722388] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:19:41.851413] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-08. [2025-10-20T19:19:42.607749] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T19:19:42.651436] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-01. [2025-10-20T19:19:43.248045] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T19:19:43.451347] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-06. [2025-10-20T19:19:43.524472] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T19:19:44.251401] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-02. [2025-10-20T19:19:44.628576] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T19:19:44.852875] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T19:19:45.051385] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-07. [2025-10-20T19:19:45.851383] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-08. [2025-10-20T19:19:46.552677] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:19:46.638340] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T19:19:46.651330] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-09. [2025-10-20T19:19:47.433771] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:19:47.451339] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-09. [2025-10-20T19:19:48.251421] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-03. [2025-10-20T19:19:48.912963] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:19:49.051423] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-10. [2025-10-20T19:19:49.329484] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T19:19:49.851387] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-10. [2025-10-20T19:19:50.450888] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:19:50.456420] [INFO ] [REMOTE ] ICL: Full fetch complete (56131 records). [2025-10-20T19:19:50.456534] [DEBUG] [VALIDATOR ] ICL: Validating primary data. [2025-10-20T19:19:50.656540] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-08. [2025-10-20T19:19:50.837249] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T19:19:50.858427] [DEBUG] [VALIDATOR ] ICL: All expected trading days are present. [2025-10-20T19:19:50.858280] [INFO ] [REMOTE ] IDE: Full fetch complete (12783 records). [2025-10-20T19:19:50.858566] [DEBUG] [VALIDATOR ] ICL: Primary data validation passed. [2025-10-20T19:19:50.858705] [DEBUG] [VALIDATOR ] IDE: Validating primary data. [2025-10-20T19:19:50.859616] [INFO ] [MAIN ] ICL: Fetched primary data is valid. [2025-10-20T19:19:50.859739] [DEBUG] [LOCAL ] ICL: Saving primary data. [2025-10-20T19:19:50.963990] [DEBUG] [LOCAL ] ICL: Trimmed 56131 records to 38359 (keeping 200 days of data) [2025-10-20T19:19:51.431974] [DEBUG] [VALIDATOR ] IDE: All expected trading days are present. [2025-10-20T19:19:51.434306] [DEBUG] [VALIDATOR ] IDE: Primary data validation passed. [2025-10-20T19:19:51.435477] [INFO ] [MAIN ] IDE: Fetched primary data is valid. [2025-10-20T19:19:51.435843] [DEBUG] [LOCAL ] IDE: Saving primary data. [2025-10-20T19:19:51.441407] [DEBUG] [LOCAL ] IDE: Trimmed 12783 records to 9091 (keeping 200 days of data) [2025-10-20T19:19:51.471414] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-04. [2025-10-20T19:19:51.481057] [DEBUG] [LOCAL ] ICL: Primary data saved (38359 records). [2025-10-20T19:19:51.493461] [INFO ] [MAIN ] ICL: Full history saved. [2025-10-20T19:19:51.493943] [INFO ] [MAIN ] Update Progress: 1085/2367 symbols completed. ICL took 30.8s. Estimated remaining: 2h 45m 10s. Rate: 7.8 symbols/min [2025-10-20T19:19:51.621105] [DEBUG] [LOCAL ] IDE: Primary data saved (9091 records). [2025-10-20T19:19:51.632132] [INFO ] [MAIN ] IDE: Full history saved. [2025-10-20T19:19:51.642580] [INFO ] [MAIN ] Update Progress: 1086/2367 symbols completed. IDE took 22.3s. Estimated remaining: 2h 44m 53s. Rate: 7.8 symbols/min [2025-10-20T19:19:51.758965] [INFO ] [MAIN ] IEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:51.869453] [INFO ] [MAIN ] IFF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:51.891367] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T19:19:51.928528] [DEBUG] [LOCAL ] IEX: Loaded 38977 existing records from primary storage. [2025-10-20T19:19:51.928944] [INFO ] [MAIN ] IEX: Loaded 38977 existing records. [2025-10-20T19:19:52.078777] [DEBUG] [LOCAL ] IFF: Loaded 51919 existing records from primary storage. [2025-10-20T19:19:52.094697] [INFO ] [MAIN ] IFF: Loaded 51919 existing records. [2025-10-20T19:19:52.270299] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-09. [2025-10-20T19:19:52.270235] [WARN ] [MAIN ] IEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:52.271475] [INFO ] [MAIN ] IEX: Wiping existing primary data before full refresh. [2025-10-20T19:19:52.271539] [INFO ] [LOCAL ] IEX: Starting deletion of primary data. [2025-10-20T19:19:52.282122] [DEBUG] [LOCAL ] IEX: Deleted primary data file. [2025-10-20T19:19:52.320696] [INFO ] [MAIN ] IEX: Starting full primary refresh. [2025-10-20T19:19:52.362301] [WARN ] [MAIN ] IFF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:52.374275] [INFO ] [MAIN ] IFF: Wiping existing primary data before full refresh. [2025-10-20T19:19:52.399562] [INFO ] [LOCAL ] IFF: Starting deletion of primary data. [2025-10-20T19:19:52.430602] [DEBUG] [LOCAL ] IFF: Deleted primary data file. [2025-10-20T19:19:52.450105] [INFO ] [MAIN ] IFF: Starting full primary refresh. [2025-10-20T19:19:52.720727] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T19:19:53.091402] [INFO ] [REMOTE ] IEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:53.063752] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-05. [2025-10-20T19:19:53.128283] [INFO ] [REMOTE ] IFF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:53.532272] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T19:19:53.851427] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-01. [2025-10-20T19:19:54.609886] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:19:54.651397] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-01. [2025-10-20T19:19:55.451390] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-06. [2025-10-20T19:19:56.251354] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-10. [2025-10-20T19:19:56.571389] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T19:19:56.613734] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-01 (2.76 seconds). [2025-10-20T19:19:56.985551] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:19:57.051425] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-02. [2025-10-20T19:19:57.680456] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T19:19:57.685313] [INFO ] [REMOTE ] IDA: Full fetch complete (45461 records). [2025-10-20T19:19:57.685440] [DEBUG] [VALIDATOR ] IDA: Validating primary data. [2025-10-20T19:19:57.862241] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-07. [2025-10-20T19:19:58.065719] [DEBUG] [VALIDATOR ] IDA: All expected trading days are present. [2025-10-20T19:19:58.065804] [DEBUG] [VALIDATOR ] IDA: Primary data validation passed. [2025-10-20T19:19:58.066505] [INFO ] [MAIN ] IDA: Fetched primary data is valid. [2025-10-20T19:19:58.066545] [DEBUG] [LOCAL ] IDA: Saving primary data. [2025-10-20T19:19:58.067502] [DEBUG] [LOCAL ] IDA: Trimmed 45461 records to 32795 (keeping 200 days of data) [2025-10-20T19:19:58.270760] [DEBUG] [LOCAL ] IDA: Primary data saved (32795 records). [2025-10-20T19:19:58.270890] [INFO ] [MAIN ] IDA: Full history saved. [2025-10-20T19:19:58.270964] [INFO ] [MAIN ] Update Progress: 1087/2367 symbols completed. IDA took 33.8s. Estimated remaining: 2h 44m 44s. Rate: 7.8 symbols/min [2025-10-20T19:19:58.442853] [INFO ] [MAIN ] IFN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:58.462056] [DEBUG] [LOCAL ] IFN: Loaded 18730 existing records from primary storage. [2025-10-20T19:19:58.462291] [INFO ] [MAIN ] IFN: Loaded 18730 existing records. [2025-10-20T19:19:58.641459] [WARN ] [MAIN ] IFN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:58.642050] [INFO ] [MAIN ] IFN: Wiping existing primary data before full refresh. [2025-10-20T19:19:58.675424] [INFO ] [LOCAL ] IFN: Starting deletion of primary data. [2025-10-20T19:19:58.675262] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-02. [2025-10-20T19:19:58.689844] [DEBUG] [LOCAL ] IFN: Deleted primary data file. [2025-10-20T19:19:58.689939] [INFO ] [MAIN ] IFN: Starting full primary refresh. [2025-10-20T19:19:58.712777] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:19:59.038380] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T19:19:59.071677] [INFO ] [REMOTE ] IFN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:59.451359] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-01. [2025-10-20T19:20:00.251347] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-08. [2025-10-20T19:20:00.338454] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:20:00.641370] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:20:01.051433] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-03. [2025-10-20T19:20:01.534768] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T19:20:01.851500] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-02. [2025-10-20T19:20:02.651386] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-03. [2025-10-20T19:20:03.125193] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T19:20:03.149715] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:20:03.451477] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-03. [2025-10-20T19:20:04.237380] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T19:20:04.253730] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-04. [2025-10-20T19:20:04.257163] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T19:20:05.051347] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-09. [2025-10-20T19:20:05.851395] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-04. [2025-10-20T19:20:06.056551] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:20:06.123933] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T19:20:06.651396] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-04. [2025-10-20T19:20:07.456780] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-10. [2025-10-20T19:20:07.468485] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:20:07.689343] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T19:20:08.251371] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-05. [2025-10-20T19:20:08.710071] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T19:20:08.713857] [INFO ] [REMOTE ] IDT: Full fetch complete (23953 records). [2025-10-20T19:20:08.713933] [DEBUG] [VALIDATOR ] IDT: Validating primary data. [2025-10-20T19:20:09.055003] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-05. [2025-10-20T19:20:09.093172] [DEBUG] [VALIDATOR ] IDT: All expected trading days are present. [2025-10-20T19:20:09.093347] [DEBUG] [VALIDATOR ] IDT: Primary data validation passed. [2025-10-20T19:20:09.093967] [INFO ] [MAIN ] IDT: Fetched primary data is valid. [2025-10-20T19:20:09.094033] [DEBUG] [LOCAL ] IDT: Saving primary data. [2025-10-20T19:20:09.094878] [DEBUG] [LOCAL ] IDT: Trimmed 23953 records to 18522 (keeping 200 days of data) [2025-10-20T19:20:09.206954] [DEBUG] [LOCAL ] IDT: Primary data saved (18522 records). [2025-10-20T19:20:09.207077] [INFO ] [MAIN ] IDT: Full history saved. [2025-10-20T19:20:09.207153] [INFO ] [MAIN ] Update Progress: 1088/2367 symbols completed. IDT took 33.4s. Estimated remaining: 2h 44m 40s. Rate: 7.8 symbols/min [2025-10-20T19:20:09.340001] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T19:20:09.401057] [INFO ] [MAIN ] IFS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:09.421839] [DEBUG] [LOCAL ] IFS: Loaded 22863 existing records from primary storage. [2025-10-20T19:20:09.422078] [INFO ] [MAIN ] IFS: Loaded 22863 existing records. [2025-10-20T19:20:09.604233] [WARN ] [MAIN ] IFS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:09.604453] [INFO ] [MAIN ] IFS: Wiping existing primary data before full refresh. [2025-10-20T19:20:09.604512] [INFO ] [LOCAL ] IFS: Starting deletion of primary data. [2025-10-20T19:20:09.604856] [DEBUG] [LOCAL ] IFS: Deleted primary data file. [2025-10-20T19:20:09.604915] [INFO ] [MAIN ] IFS: Starting full primary refresh. [2025-10-20T19:20:09.856818] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-05. [2025-10-20T19:20:09.963770] [INFO ] [REMOTE ] IFS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:10.651406] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-06. [2025-10-20T19:20:10.961303] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:20:11.407359] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:20:11.451357] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-06. [2025-10-20T19:20:11.514075] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T19:20:12.251413] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-06. [2025-10-20T19:20:13.007627] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:20:13.051344] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-07. [2025-10-20T19:20:13.837008] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T19:20:13.851327] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-07. [2025-10-20T19:20:14.651371] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-07. [2025-10-20T19:20:15.006736] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T19:20:15.065376] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T19:20:15.451382] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-08. [2025-10-20T19:20:16.251391] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-01. [2025-10-20T19:20:17.010129] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-10-20T19:20:17.047204] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T19:20:17.051316] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-02. [2025-10-20T19:20:17.428155] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-10-20T19:20:17.851402] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-09. [2025-10-20T19:20:18.132887] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T19:20:18.651431] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-03. [2025-10-20T19:20:19.102633] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T19:20:19.451384] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-10. [2025-10-20T19:20:19.904809] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T19:20:20.251377] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-08. [2025-10-20T19:20:20.984703] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T19:20:20.989991] [INFO ] [REMOTE ] IEX: Full fetch complete (53824 records). [2025-10-20T19:20:20.990090] [DEBUG] [VALIDATOR ] IEX: Validating primary data. [2025-10-20T19:20:21.056704] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-04. [2025-10-20T19:20:21.367477] [DEBUG] [VALIDATOR ] IEX: All expected trading days are present. [2025-10-20T19:20:21.367564] [DEBUG] [VALIDATOR ] IEX: Primary data validation passed. [2025-10-20T19:20:21.368355] [INFO ] [MAIN ] IEX: Fetched primary data is valid. [2025-10-20T19:20:21.368418] [DEBUG] [LOCAL ] IEX: Saving primary data. [2025-10-20T19:20:21.369588] [DEBUG] [LOCAL ] IEX: Trimmed 53824 records to 38423 (keeping 200 days of data) [2025-10-20T19:20:21.606777] [DEBUG] [LOCAL ] IEX: Primary data saved (38423 records). [2025-10-20T19:20:21.606923] [INFO ] [MAIN ] IEX: Full history saved. [2025-10-20T19:20:21.607003] [INFO ] [MAIN ] Update Progress: 1089/2367 symbols completed. IEX took 30.1s. Estimated remaining: 2h 44m 38s. Rate: 7.8 symbols/min [2025-10-20T19:20:21.775436] [INFO ] [MAIN ] IGA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:21.786775] [DEBUG] [LOCAL ] IGA: Loaded 10395 existing records from primary storage. [2025-10-20T19:20:21.786997] [INFO ] [MAIN ] IGA: Loaded 10395 existing records. [2025-10-20T19:20:21.855865] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-08. [2025-10-20T19:20:21.970384] [WARN ] [MAIN ] IGA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:21.970741] [INFO ] [MAIN ] IGA: Wiping existing primary data before full refresh. [2025-10-20T19:20:21.970805] [INFO ] [LOCAL ] IGA: Starting deletion of primary data. [2025-10-20T19:20:21.971049] [DEBUG] [LOCAL ] IGA: Deleted primary data file. [2025-10-20T19:20:21.971152] [INFO ] [MAIN ] IGA: Starting full primary refresh. [2025-10-20T19:20:22.322875] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-10-20T19:20:22.373739] [INFO ] [REMOTE ] IGA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:22.389821] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T19:20:22.651436] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-09. [2025-10-20T19:20:22.952576] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T19:20:23.451450] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-05. [2025-10-20T19:20:24.251417] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-09. [2025-10-20T19:20:24.464993] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T19:20:24.564739] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:20:25.051444] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-01. [2025-10-20T19:20:25.219135] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:20:25.829953] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T19:20:25.851346] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-02. [2025-10-20T19:20:26.634357] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:20:26.651342] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-03. [2025-10-20T19:20:27.428771] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T19:20:27.451373] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-04. [2025-10-20T19:20:28.251352] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-10. [2025-10-20T19:20:28.410403] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-10-20T19:20:29.051415] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-05. [2025-10-20T19:20:29.259983] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:20:29.263594] [INFO ] [REMOTE ] IFN: Full fetch complete (28344 records). [2025-10-20T19:20:29.263770] [DEBUG] [VALIDATOR ] IFN: Validating primary data. [2025-10-20T19:20:29.656351] [DEBUG] [VALIDATOR ] IFN: All expected trading days are present. [2025-10-20T19:20:29.656538] [DEBUG] [VALIDATOR ] IFN: Primary data validation passed. [2025-10-20T19:20:29.657432] [INFO ] [MAIN ] IFN: Fetched primary data is valid. [2025-10-20T19:20:29.657497] [DEBUG] [LOCAL ] IFN: Saving primary data. [2025-10-20T19:20:29.658593] [DEBUG] [LOCAL ] IFN: Trimmed 28344 records to 18570 (keeping 200 days of data) [2025-10-20T19:20:29.777268] [DEBUG] [LOCAL ] IFN: Primary data saved (18570 records). [2025-10-20T19:20:29.783301] [INFO ] [MAIN ] IFN: Full history saved. [2025-10-20T19:20:29.784375] [INFO ] [MAIN ] Update Progress: 1090/2367 symbols completed. IFN took 31.5s. Estimated remaining: 2h 44m 31s. Rate: 7.8 symbols/min [2025-10-20T19:20:29.784200] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T19:20:29.856265] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-06. [2025-10-20T19:20:29.979019] [INFO ] [MAIN ] IGD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:30.000876] [DEBUG] [LOCAL ] IGD: Loaded 19676 existing records from primary storage. [2025-10-20T19:20:30.001220] [INFO ] [MAIN ] IGD: Loaded 19676 existing records. [2025-10-20T19:20:30.185517] [WARN ] [MAIN ] IGD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:30.185858] [INFO ] [MAIN ] IGD: Wiping existing primary data before full refresh. [2025-10-20T19:20:30.185914] [INFO ] [LOCAL ] IGD: Starting deletion of primary data. [2025-10-20T19:20:30.186227] [DEBUG] [LOCAL ] IGD: Deleted primary data file. [2025-10-20T19:20:30.186272] [INFO ] [MAIN ] IGD: Starting full primary refresh. [2025-10-20T19:20:30.555140] [INFO ] [REMOTE ] IGD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:30.651427] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-10. [2025-10-20T19:20:31.153616] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T19:20:31.451383] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-01. [2025-10-20T19:20:32.251351] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-07. [2025-10-20T19:20:32.582530] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-10-20T19:20:32.590042] [INFO ] [REMOTE ] IFF: Full fetch complete (73707 records). [2025-10-20T19:20:32.590165] [DEBUG] [VALIDATOR ] IFF: Validating primary data. [2025-10-20T19:20:32.980695] [DEBUG] [VALIDATOR ] IFF: All expected trading days are present. [2025-10-20T19:20:32.980821] [DEBUG] [VALIDATOR ] IFF: Primary data validation passed. [2025-10-20T19:20:32.982570] [INFO ] [MAIN ] IFF: Fetched primary data is valid. [2025-10-20T19:20:32.982666] [DEBUG] [LOCAL ] IFF: Saving primary data. [2025-10-20T19:20:33.000019] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T19:20:33.007271] [DEBUG] [LOCAL ] IFF: Trimmed 73707 records to 51225 (keeping 200 days of data) [2025-10-20T19:20:33.062485] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-02. [2025-10-20T19:20:33.327328] [DEBUG] [LOCAL ] IFF: Primary data saved (51225 records). [2025-10-20T19:20:33.327467] [INFO ] [MAIN ] IFF: Full history saved. [2025-10-20T19:20:33.327543] [INFO ] [MAIN ] Update Progress: 1091/2367 symbols completed. IFF took 41.7s. Estimated remaining: 2h 44m 18s. Rate: 7.8 symbols/min [2025-10-20T19:20:33.464087] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T19:20:33.519266] [INFO ] [MAIN ] IGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:33.525123] [DEBUG] [LOCAL ] IGI: Loaded 4092 existing records from primary storage. [2025-10-20T19:20:33.525300] [INFO ] [MAIN ] IGI: Loaded 4092 existing records. [2025-10-20T19:20:33.700746] [WARN ] [MAIN ] IGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:33.700943] [INFO ] [MAIN ] IGI: Wiping existing primary data before full refresh. [2025-10-20T19:20:33.700993] [INFO ] [LOCAL ] IGI: Starting deletion of primary data. [2025-10-20T19:20:33.701121] [DEBUG] [LOCAL ] IGI: Deleted primary data file. [2025-10-20T19:20:33.701158] [INFO ] [MAIN ] IGI: Starting full primary refresh. [2025-10-20T19:20:33.858346] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-06. [2025-10-20T19:20:34.054527] [INFO ] [REMOTE ] IGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:34.202256] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:20:34.651416] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-08. [2025-10-20T19:20:34.674472] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T19:20:35.451416] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-07. [2025-10-20T19:20:35.742001] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T19:20:36.251413] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-09. [2025-10-20T19:20:36.326778] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T19:20:37.051532] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-03. [2025-10-20T19:20:37.302724] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T19:20:37.828299] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T19:20:37.851340] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-04. [2025-10-20T19:20:38.651404] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-08. [2025-10-20T19:20:38.875142] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T19:20:39.386315] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T19:20:39.451392] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-05. [2025-10-20T19:20:40.251417] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-10. [2025-10-20T19:20:40.426813] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T19:20:41.051345] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-01. [2025-10-20T19:20:41.233488] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:20:41.236834] [INFO ] [REMOTE ] IFS: Full fetch complete (32212 records). [2025-10-20T19:20:41.236942] [DEBUG] [VALIDATOR ] IFS: Validating primary data. [2025-10-20T19:20:41.611372] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:20:41.644240] [DEBUG] [VALIDATOR ] IFS: All expected trading days are present. [2025-10-20T19:20:41.644330] [DEBUG] [VALIDATOR ] IFS: Primary data validation passed. [2025-10-20T19:20:41.645040] [INFO ] [MAIN ] IFS: Fetched primary data is valid. [2025-10-20T19:20:41.645108] [DEBUG] [LOCAL ] IFS: Saving primary data. [2025-10-20T19:20:41.645976] [DEBUG] [LOCAL ] IFS: Trimmed 32212 records to 22330 (keeping 200 days of data) [2025-10-20T19:20:41.777713] [DEBUG] [LOCAL ] IFS: Primary data saved (22330 records). [2025-10-20T19:20:41.777857] [INFO ] [MAIN ] IFS: Full history saved. [2025-10-20T19:20:41.777942] [INFO ] [MAIN ] Update Progress: 1092/2367 symbols completed. IFS took 32.6s. Estimated remaining: 2h 44m 11s. Rate: 7.8 symbols/min [2025-10-20T19:20:41.856715] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-02. [2025-10-20T19:20:41.955114] [INFO ] [MAIN ] IGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:41.983838] [DEBUG] [LOCAL ] IGR: Loaded 30813 existing records from primary storage. [2025-10-20T19:20:41.984115] [INFO ] [MAIN ] IGR: Loaded 30813 existing records. [2025-10-20T19:20:42.167441] [WARN ] [MAIN ] IGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:42.167707] [INFO ] [MAIN ] IGR: Wiping existing primary data before full refresh. [2025-10-20T19:20:42.167766] [INFO ] [LOCAL ] IGR: Starting deletion of primary data. [2025-10-20T19:20:42.167977] [DEBUG] [LOCAL ] IGR: Deleted primary data file. [2025-10-20T19:20:42.168017] [INFO ] [MAIN ] IGR: Starting full primary refresh. [2025-10-20T19:20:42.565566] [INFO ] [REMOTE ] IGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:42.651384] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-01. [2025-10-20T19:20:42.692970] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T19:20:43.451409] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-09. [2025-10-20T19:20:44.029864] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T19:20:44.044088] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:20:44.251385] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-02. [2025-10-20T19:20:45.051357] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-03. [2025-10-20T19:20:45.483045] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:20:45.616399] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T19:20:45.851372] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-03. [2025-10-20T19:20:46.651407] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-04. [2025-10-20T19:20:47.069484] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T19:20:47.122225] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-10-20T19:20:47.451371] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-06. [2025-10-20T19:20:48.251349] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-05. [2025-10-20T19:20:48.517092] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T19:20:48.683826] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T19:20:49.051431] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-06. [2025-10-20T19:20:49.601519] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T19:20:49.851392] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-10. [2025-10-20T19:20:50.651373] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-07. [2025-10-20T19:20:50.732475] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T19:20:50.735429] [INFO ] [REMOTE ] IGA: Full fetch complete (14366 records). [2025-10-20T19:20:50.735586] [DEBUG] [VALIDATOR ] IGA: Validating primary data. [2025-10-20T19:20:51.140424] [DEBUG] [VALIDATOR ] IGA: All expected trading days are present. [2025-10-20T19:20:51.140561] [DEBUG] [VALIDATOR ] IGA: Primary data validation passed. [2025-10-20T19:20:51.141151] [INFO ] [MAIN ] IGA: Fetched primary data is valid. [2025-10-20T19:20:51.141215] [DEBUG] [LOCAL ] IGA: Saving primary data. [2025-10-20T19:20:51.141811] [DEBUG] [LOCAL ] IGA: Trimmed 14366 records to 10268 (keeping 200 days of data) [2025-10-20T19:20:51.208149] [DEBUG] [LOCAL ] IGA: Primary data saved (10268 records). [2025-10-20T19:20:51.208345] [INFO ] [MAIN ] IGA: Full history saved. [2025-10-20T19:20:51.208446] [INFO ] [MAIN ] Update Progress: 1093/2367 symbols completed. IGA took 29.6s. Estimated remaining: 2h 44m 6s. Rate: 7.8 symbols/min [2025-10-20T19:20:51.325130] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-10-20T19:20:51.407016] [INFO ] [MAIN ] IH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:51.407261] [DEBUG] [LOCAL ] IH: No existing data file found in primary storage. [2025-10-20T19:20:51.407300] [WARN ] [MAIN ] IH: No existing data found, performing full refresh instead. [2025-10-20T19:20:51.407429] [INFO ] [MAIN ] IH: Wiping existing primary data before full refresh. [2025-10-20T19:20:51.407471] [INFO ] [LOCAL ] IH: Starting deletion of primary data. [2025-10-20T19:20:51.407513] [DEBUG] [LOCAL ] IH: No primary data file found to delete. [2025-10-20T19:20:51.407546] [INFO ] [MAIN ] IH: Starting full primary refresh. [2025-10-20T19:20:51.456574] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-08. [2025-10-20T19:20:51.782499] [INFO ] [REMOTE ] IH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:52.134883] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T19:20:52.251431] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-09. [2025-10-20T19:20:52.830020] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T19:20:53.051411] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-10. [2025-10-20T19:20:53.851390] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-01. [2025-10-20T19:20:53.864983] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:20:53.866788] [INFO ] [REMOTE ] IGI: Full fetch complete (5502 records). [2025-10-20T19:20:53.866877] [DEBUG] [VALIDATOR ] IGI: Validating primary data. [2025-10-20T19:20:54.242279] [DEBUG] [VALIDATOR ] IGI: All expected trading days are present. [2025-10-20T19:20:54.242382] [DEBUG] [VALIDATOR ] IGI: Primary data validation passed. [2025-10-20T19:20:54.242564] [INFO ] [MAIN ] IGI: Fetched primary data is valid. [2025-10-20T19:20:54.242604] [DEBUG] [LOCAL ] IGI: Saving primary data. [2025-10-20T19:20:54.242989] [DEBUG] [LOCAL ] IGI: Trimmed 5502 records to 4077 (keeping 200 days of data) [2025-10-20T19:20:54.267376] [DEBUG] [LOCAL ] IGI: Primary data saved (4077 records). [2025-10-20T19:20:54.267505] [INFO ] [MAIN ] IGI: Full history saved. [2025-10-20T19:20:54.267591] [INFO ] [MAIN ] Update Progress: 1094/2367 symbols completed. IGI took 20.9s. Estimated remaining: 2h 43m 52s. Rate: 7.8 symbols/min [2025-10-20T19:20:54.376563] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-10-20T19:20:54.448212] [INFO ] [MAIN ] IHD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:54.458168] [DEBUG] [LOCAL ] IHD: Loaded 7396 existing records from primary storage. [2025-10-20T19:20:54.458398] [INFO ] [MAIN ] IHD: Loaded 7396 existing records. [2025-10-20T19:20:54.630020] [WARN ] [MAIN ] IHD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:54.630254] [INFO ] [MAIN ] IHD: Wiping existing primary data before full refresh. [2025-10-20T19:20:54.630301] [INFO ] [LOCAL ] IHD: Starting deletion of primary data. [2025-10-20T19:20:54.630461] [DEBUG] [LOCAL ] IHD: Deleted primary data file. [2025-10-20T19:20:54.630526] [INFO ] [MAIN ] IHD: Starting full primary refresh. [2025-10-20T19:20:54.658957] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-02. [2025-10-20T19:20:55.011377] [INFO ] [REMOTE ] IHD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:55.326176] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T19:20:55.451444] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-01. [2025-10-20T19:20:56.250214] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T19:20:56.252824] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-03. [2025-10-20T19:20:56.991943] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T19:20:57.051355] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-04. [2025-10-20T19:20:57.851384] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-04. [2025-10-20T19:20:58.352381] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T19:20:58.394835] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T19:20:58.651378] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-07. [2025-10-20T19:20:59.451358] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-02. [2025-10-20T19:20:59.748839] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T19:21:00.176633] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T19:21:00.251417] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-08. [2025-10-20T19:21:01.051371] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-05. [2025-10-20T19:21:01.514478] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T19:21:01.614971] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T19:21:01.851406] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-09. [2025-10-20T19:21:02.651370] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-03. [2025-10-20T19:21:02.870584] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:21:03.401867] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T19:21:03.451337] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-04. [2025-10-20T19:21:04.109938] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T19:21:04.251365] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-05. [2025-10-20T19:21:05.051429] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-05. [2025-10-20T19:21:05.574679] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T19:21:05.833378] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T19:21:05.851335] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-06. [2025-10-20T19:21:06.584253] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:21:06.651398] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-06. [2025-10-20T19:21:07.091795] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T19:21:07.451383] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-10. [2025-10-20T19:21:08.251384] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-06. [2025-10-20T19:21:08.579105] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T19:21:08.582989] [INFO ] [REMOTE ] IGD: Full fetch complete (30169 records). [2025-10-20T19:21:08.583132] [DEBUG] [VALIDATOR ] IGD: Validating primary data. [2025-10-20T19:21:08.978584] [DEBUG] [VALIDATOR ] IGD: All expected trading days are present. [2025-10-20T19:21:08.978685] [DEBUG] [VALIDATOR ] IGD: Primary data validation passed. [2025-10-20T19:21:08.979339] [INFO ] [MAIN ] IGD: Fetched primary data is valid. [2025-10-20T19:21:08.979400] [DEBUG] [LOCAL ] IGD: Saving primary data. [2025-10-20T19:21:08.980105] [DEBUG] [LOCAL ] IGD: Trimmed 30169 records to 19266 (keeping 200 days of data) [2025-10-20T19:21:09.057688] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-07. [2025-10-20T19:21:09.093160] [DEBUG] [LOCAL ] IGD: Primary data saved (19266 records). [2025-10-20T19:21:09.093307] [INFO ] [MAIN ] IGD: Full history saved. [2025-10-20T19:21:09.093389] [INFO ] [MAIN ] Update Progress: 1095/2367 symbols completed. IGD took 39.3s. Estimated remaining: 2h 43m 53s. Rate: 7.8 symbols/min [2025-10-20T19:21:09.271233] [INFO ] [MAIN ] IHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:09.297089] [DEBUG] [LOCAL ] IHG: Loaded 24671 existing records from primary storage. [2025-10-20T19:21:09.297351] [INFO ] [MAIN ] IHG: Loaded 24671 existing records. [2025-10-20T19:21:09.461197] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T19:21:09.510058] [WARN ] [MAIN ] IHG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:09.510282] [INFO ] [MAIN ] IHG: Wiping existing primary data before full refresh. [2025-10-20T19:21:09.510331] [INFO ] [LOCAL ] IHG: Starting deletion of primary data. [2025-10-20T19:21:09.510631] [DEBUG] [LOCAL ] IHG: Deleted primary data file. [2025-10-20T19:21:09.510725] [INFO ] [MAIN ] IHG: Starting full primary refresh. [2025-10-20T19:21:09.615166] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T19:21:09.856596] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-08. [2025-10-20T19:21:09.909297] [INFO ] [REMOTE ] IHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:10.554054] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T19:21:10.651396] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-07. [2025-10-20T19:21:11.451365] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-01. [2025-10-20T19:21:12.203252] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:21:12.251351] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-08. [2025-10-20T19:21:12.421228] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T19:21:13.051383] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-07. [2025-10-20T19:21:13.358994] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T19:21:13.713335] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T19:21:13.851416] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-09. [2025-10-20T19:21:14.342730] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T19:21:14.651371] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-08. [2025-10-20T19:21:15.266107] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:21:15.451373] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-09. [2025-10-20T19:21:16.225501] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T19:21:16.251415] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-02. [2025-10-20T19:21:17.051431] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-10. [2025-10-20T19:21:17.456715] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T19:21:17.688480] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T19:21:17.689787] [INFO ] [REMOTE ] IH: Full fetch complete (5035 records). [2025-10-20T19:21:17.689861] [DEBUG] [VALIDATOR ] IH: Validating primary data. [2025-10-20T19:21:17.861449] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-10. [2025-10-20T19:21:18.102552] [DEBUG] [VALIDATOR ] IH: All expected trading days are present. [2025-10-20T19:21:18.102624] [DEBUG] [VALIDATOR ] IH: Primary data validation passed. [2025-10-20T19:21:18.102799] [INFO ] [MAIN ] IH: Fetched primary data is valid. [2025-10-20T19:21:18.102847] [DEBUG] [LOCAL ] IH: Saving primary data. [2025-10-20T19:21:18.103186] [DEBUG] [LOCAL ] IH: Trimmed 5035 records to 2768 (keeping 200 days of data) [2025-10-20T19:21:18.118253] [DEBUG] [LOCAL ] IH: Primary data saved (2768 records). [2025-10-20T19:21:18.118314] [INFO ] [MAIN ] IH: Full history saved. [2025-10-20T19:21:18.118371] [INFO ] [MAIN ] Update Progress: 1096/2367 symbols completed. IH took 26.9s. Estimated remaining: 2h 43m 47s. Rate: 7.8 symbols/min [2025-10-20T19:21:18.285488] [INFO ] [MAIN ] IHS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:18.318935] [DEBUG] [LOCAL ] IHS: Loaded 38062 existing records from primary storage. [2025-10-20T19:21:18.319237] [INFO ] [MAIN ] IHS: Loaded 38062 existing records. [2025-10-20T19:21:18.491949] [WARN ] [MAIN ] IHS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:18.492613] [INFO ] [MAIN ] IHS: Wiping existing primary data before full refresh. [2025-10-20T19:21:18.492714] [INFO ] [LOCAL ] IHS: Starting deletion of primary data. [2025-10-20T19:21:18.492936] [DEBUG] [LOCAL ] IHS: Deleted primary data file. [2025-10-20T19:21:18.492992] [INFO ] [MAIN ] IHS: Starting full primary refresh. [2025-10-20T19:21:18.654333] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-09. [2025-10-20T19:21:18.833801] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T19:21:18.862214] [INFO ] [REMOTE ] IHS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:18.864463] [INFO ] [REMOTE ] IHD: Full fetch complete (11763 records). [2025-10-20T19:21:18.864539] [DEBUG] [VALIDATOR ] IHD: Validating primary data. [2025-10-20T19:21:19.211940] [DEBUG] [VALIDATOR ] IHD: All expected trading days are present. [2025-10-20T19:21:19.212011] [DEBUG] [VALIDATOR ] IHD: Primary data validation passed. [2025-10-20T19:21:19.212241] [INFO ] [MAIN ] IHD: Fetched primary data is valid. [2025-10-20T19:21:19.212279] [DEBUG] [LOCAL ] IHD: Saving primary data. [2025-10-20T19:21:19.212695] [DEBUG] [LOCAL ] IHD: Trimmed 11763 records to 7362 (keeping 200 days of data) [2025-10-20T19:21:19.253717] [DEBUG] [LOCAL ] IHD: Primary data saved (7362 records). [2025-10-20T19:21:19.253806] [INFO ] [MAIN ] IHD: Full history saved. [2025-10-20T19:21:19.253894] [INFO ] [MAIN ] Update Progress: 1097/2367 symbols completed. IHD took 25.0s. Estimated remaining: 2h 43m 31s. Rate: 7.8 symbols/min [2025-10-20T19:21:19.417035] [INFO ] [MAIN ] IIF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:19.424336] [DEBUG] [LOCAL ] IIF: Loaded 5622 existing records from primary storage. [2025-10-20T19:21:19.424524] [INFO ] [MAIN ] IIF: Loaded 5622 existing records. [2025-10-20T19:21:19.456521] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-01. [2025-10-20T19:21:19.599816] [WARN ] [MAIN ] IIF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:19.600013] [INFO ] [MAIN ] IIF: Wiping existing primary data before full refresh. [2025-10-20T19:21:19.600059] [INFO ] [LOCAL ] IIF: Starting deletion of primary data. [2025-10-20T19:21:19.600255] [DEBUG] [LOCAL ] IIF: Deleted primary data file. [2025-10-20T19:21:19.600311] [INFO ] [MAIN ] IIF: Starting full primary refresh. [2025-10-20T19:21:19.962948] [INFO ] [REMOTE ] IIF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:19.979952] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T19:21:20.251372] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-01. [2025-10-20T19:21:20.726342] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T19:21:21.038434] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:21:21.051359] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-02. [2025-10-20T19:21:21.760525] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T19:21:21.851412] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-03. [2025-10-20T19:21:22.447531] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T19:21:22.651407] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-02. [2025-10-20T19:21:23.451365] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-10. [2025-10-20T19:21:23.768105] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T19:21:24.251420] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-03. [2025-10-20T19:21:24.685365] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:21:24.689593] [INFO ] [REMOTE ] IGR: Full fetch complete (44722 records). [2025-10-20T19:21:24.689704] [DEBUG] [VALIDATOR ] IGR: Validating primary data. [2025-10-20T19:21:25.056752] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-04. [2025-10-20T19:21:25.074922] [DEBUG] [VALIDATOR ] IGR: All expected trading days are present. [2025-10-20T19:21:25.074998] [DEBUG] [VALIDATOR ] IGR: Primary data validation passed. [2025-10-20T19:21:25.075766] [INFO ] [MAIN ] IGR: Fetched primary data is valid. [2025-10-20T19:21:25.075912] [DEBUG] [LOCAL ] IGR: Saving primary data. [2025-10-20T19:21:25.076901] [DEBUG] [LOCAL ] IGR: Trimmed 44722 records to 30502 (keeping 200 days of data) [2025-10-20T19:21:25.247097] [DEBUG] [LOCAL ] IGR: Primary data saved (30502 records). [2025-10-20T19:21:25.247214] [INFO ] [MAIN ] IGR: Full history saved. [2025-10-20T19:21:25.247284] [INFO ] [MAIN ] Update Progress: 1098/2367 symbols completed. IGR took 43.5s. Estimated remaining: 2h 43m 22s. Rate: 7.8 symbols/min [2025-10-20T19:21:25.412177] [INFO ] [MAIN ] IIIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:25.428162] [DEBUG] [LOCAL ] IIIN: Loaded 16427 existing records from primary storage. [2025-10-20T19:21:25.428364] [INFO ] [MAIN ] IIIN: Loaded 16427 existing records. [2025-10-20T19:21:25.603042] [WARN ] [MAIN ] IIIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:25.603262] [INFO ] [MAIN ] IIIN: Wiping existing primary data before full refresh. [2025-10-20T19:21:25.603329] [INFO ] [LOCAL ] IIIN: Starting deletion of primary data. [2025-10-20T19:21:25.603481] [DEBUG] [LOCAL ] IIIN: Deleted primary data file. [2025-10-20T19:21:25.603518] [INFO ] [MAIN ] IIIN: Starting full primary refresh. [2025-10-20T19:21:25.870601] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-03. [2025-10-20T19:21:25.918991] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:21:25.999789] [INFO ] [REMOTE ] IIIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:26.006616] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T19:21:26.651409] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-05. [2025-10-20T19:21:27.179704] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T19:21:27.232433] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:21:27.451397] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-01. [2025-10-20T19:21:28.251403] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-06. [2025-10-20T19:21:28.562747] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T19:21:28.629166] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-10-20T19:21:29.051461] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-04. [2025-10-20T19:21:29.851382] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-04. [2025-10-20T19:21:30.437107] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T19:21:30.651423] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-02. [2025-10-20T19:21:30.919514] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T19:21:31.451416] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-07. [2025-10-20T19:21:31.627844] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T19:21:32.215709] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T19:21:32.251367] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-08. [2025-10-20T19:21:32.860687] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:21:33.051395] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-05. [2025-10-20T19:21:33.851422] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-09. [2025-10-20T19:21:34.362243] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:21:34.427598] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T19:21:34.651380] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-10. [2025-10-20T19:21:35.285531] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T19:21:35.286508] [INFO ] [REMOTE ] IIF: Full fetch complete (8919 records). [2025-10-20T19:21:35.286594] [DEBUG] [VALIDATOR ] IIF: Validating primary data. [2025-10-20T19:21:35.451400] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-03. [2025-10-20T19:21:35.666343] [DEBUG] [VALIDATOR ] IIF: All expected trading days are present. [2025-10-20T19:21:35.666410] [DEBUG] [VALIDATOR ] IIF: Primary data validation passed. [2025-10-20T19:21:35.666677] [INFO ] [MAIN ] IIF: Fetched primary data is valid. [2025-10-20T19:21:35.666742] [DEBUG] [LOCAL ] IIF: Saving primary data. [2025-10-20T19:21:35.667281] [DEBUG] [LOCAL ] IIF: Trimmed 8919 records to 5448 (keeping 200 days of data) [2025-10-20T19:21:35.698105] [DEBUG] [LOCAL ] IIF: Primary data saved (5448 records). [2025-10-20T19:21:35.698210] [INFO ] [MAIN ] IIF: Full history saved. [2025-10-20T19:21:35.698283] [INFO ] [MAIN ] Update Progress: 1099/2367 symbols completed. IIF took 16.4s. Estimated remaining: 2h 43m 17s. Rate: 7.8 symbols/min [2025-10-20T19:21:35.866621] [INFO ] [MAIN ] IIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:35.883362] [DEBUG] [LOCAL ] IIM: Loaded 15903 existing records from primary storage. [2025-10-20T19:21:35.883605] [INFO ] [MAIN ] IIM: Loaded 15903 existing records. [2025-10-20T19:21:36.058516] [WARN ] [MAIN ] IIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:36.058757] [INFO ] [MAIN ] IIM: Wiping existing primary data before full refresh. [2025-10-20T19:21:36.058824] [INFO ] [LOCAL ] IIM: Starting deletion of primary data. [2025-10-20T19:21:36.059055] [DEBUG] [LOCAL ] IIM: Deleted primary data file. [2025-10-20T19:21:36.059095] [INFO ] [MAIN ] IIM: Starting full primary refresh. [2025-10-20T19:21:36.261706] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-06. [2025-10-20T19:21:36.424112] [INFO ] [REMOTE ] IIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:36.561145] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T19:21:37.051384] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-05. [2025-10-20T19:21:37.428506] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T19:21:37.851366] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-01. [2025-10-20T19:21:38.444690] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:21:38.610150] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T19:21:38.651333] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-02. [2025-10-20T19:21:39.451372] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-07. [2025-10-20T19:21:39.792890] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:21:40.251340] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-04. [2025-10-20T19:21:40.654521] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T19:21:41.051373] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-03. [2025-10-20T19:21:41.401327] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:21:41.851365] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-05. [2025-10-20T19:21:42.034691] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T19:21:42.651349] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-06. [2025-10-20T19:21:42.883849] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T19:21:43.451426] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-04. [2025-10-20T19:21:44.084520] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T19:21:44.138799] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T19:21:44.251380] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-05. [2025-10-20T19:21:44.957951] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T19:21:45.051406] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-06. [2025-10-20T19:21:45.851351] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-06. [2025-10-20T19:21:46.005072] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T19:21:46.651340] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-07. [2025-10-20T19:21:46.888570] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T19:21:47.451346] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-08. [2025-10-20T19:21:47.637288] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T19:21:48.251336] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-07. [2025-10-20T19:21:48.353949] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T19:21:49.051369] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-09. [2025-10-20T19:21:49.750440] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:21:49.851333] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-08. [2025-10-20T19:21:50.112003] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:21:50.651372] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-08. [2025-10-20T19:21:51.157196] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:21:51.451356] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-07. [2025-10-20T19:21:51.753464] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T19:21:52.251365] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-10. [2025-10-20T19:21:52.481535] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T19:21:53.051459] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-09. [2025-10-20T19:21:53.453753] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T19:21:53.458756] [INFO ] [REMOTE ] IHG: Full fetch complete (34224 records). [2025-10-20T19:21:53.458891] [DEBUG] [VALIDATOR ] IHG: Validating primary data. [2025-10-20T19:21:53.833081] [DEBUG] [VALIDATOR ] IHG: All expected trading days are present. [2025-10-20T19:21:53.833172] [DEBUG] [VALIDATOR ] IHG: Primary data validation passed. [2025-10-20T19:21:53.834091] [INFO ] [MAIN ] IHG: Fetched primary data is valid. [2025-10-20T19:21:53.834153] [DEBUG] [LOCAL ] IHG: Saving primary data. [2025-10-20T19:21:53.835271] [DEBUG] [LOCAL ] IHG: Trimmed 34224 records to 24215 (keeping 200 days of data) [2025-10-20T19:21:53.892348] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-08. [2025-10-20T19:21:53.992041] [DEBUG] [LOCAL ] IHG: Primary data saved (24215 records). [2025-10-20T19:21:53.992172] [INFO ] [MAIN ] IHG: Full history saved. [2025-10-20T19:21:53.992252] [INFO ] [MAIN ] Update Progress: 1100/2367 symbols completed. IHG took 44.9s. Estimated remaining: 2h 43m 21s. Rate: 7.8 symbols/min [2025-10-20T19:21:54.162406] [INFO ] [MAIN ] IIPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:54.174576] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T19:21:54.206690] [DEBUG] [LOCAL ] IIPR: Loaded 29862 existing records from primary storage. [2025-10-20T19:21:54.206964] [INFO ] [MAIN ] IIPR: Loaded 29862 existing records. [2025-10-20T19:21:54.386956] [WARN ] [MAIN ] IIPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:54.387161] [INFO ] [MAIN ] IIPR: Wiping existing primary data before full refresh. [2025-10-20T19:21:54.387209] [INFO ] [LOCAL ] IIPR: Starting deletion of primary data. [2025-10-20T19:21:54.387568] [DEBUG] [LOCAL ] IIPR: Deleted primary data file. [2025-10-20T19:21:54.387609] [INFO ] [MAIN ] IIPR: Starting full primary refresh. [2025-10-20T19:21:54.656522] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-09. [2025-10-20T19:21:54.761324] [INFO ] [REMOTE ] IIPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:54.762049] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T19:21:55.451370] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-10. [2025-10-20T19:21:55.899207] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T19:21:56.127433] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T19:21:56.130300] [INFO ] [REMOTE ] IIM: Full fetch complete (24908 records). [2025-10-20T19:21:56.130382] [DEBUG] [VALIDATOR ] IIM: Validating primary data. [2025-10-20T19:21:56.256537] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-01. [2025-10-20T19:21:56.513274] [DEBUG] [VALIDATOR ] IIM: All expected trading days are present. [2025-10-20T19:21:56.513348] [DEBUG] [VALIDATOR ] IIM: Primary data validation passed. [2025-10-20T19:21:56.513922] [INFO ] [MAIN ] IIM: Fetched primary data is valid. [2025-10-20T19:21:56.513985] [DEBUG] [LOCAL ] IIM: Saving primary data. [2025-10-20T19:21:56.514823] [DEBUG] [LOCAL ] IIM: Trimmed 24908 records to 15541 (keeping 200 days of data) [2025-10-20T19:21:56.609057] [DEBUG] [LOCAL ] IIM: Primary data saved (15541 records). [2025-10-20T19:21:56.609188] [INFO ] [MAIN ] IIM: Full history saved. [2025-10-20T19:21:56.609261] [INFO ] [MAIN ] Update Progress: 1101/2367 symbols completed. IIM took 20.9s. Estimated remaining: 2h 43m 8s. Rate: 7.8 symbols/min [2025-10-20T19:21:56.783863] [INFO ] [MAIN ] IMAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:56.824800] [DEBUG] [LOCAL ] IMAX: Loaded 43881 existing records from primary storage. [2025-10-20T19:21:56.825095] [INFO ] [MAIN ] IMAX: Loaded 43881 existing records. [2025-10-20T19:21:56.999053] [WARN ] [MAIN ] IMAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:56.999278] [INFO ] [MAIN ] IMAX: Wiping existing primary data before full refresh. [2025-10-20T19:21:56.999324] [INFO ] [LOCAL ] IMAX: Starting deletion of primary data. [2025-10-20T19:21:56.999570] [DEBUG] [LOCAL ] IMAX: Deleted primary data file. [2025-10-20T19:21:56.999607] [INFO ] [MAIN ] IMAX: Starting full primary refresh. [2025-10-20T19:21:57.061738] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-09. [2025-10-20T19:21:57.359459] [INFO ] [REMOTE ] IMAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:57.813837] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:21:57.851335] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-01. [2025-10-20T19:21:57.909449] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T19:21:58.651418] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-02. [2025-10-20T19:21:59.308871] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T19:21:59.451411] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-10. [2025-10-20T19:21:59.843267] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T19:22:00.251358] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-03. [2025-10-20T19:22:00.450997] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:22:00.453848] [INFO ] [REMOTE ] IIIN: Full fetch complete (24865 records). [2025-10-20T19:22:00.453917] [DEBUG] [VALIDATOR ] IIIN: Validating primary data. [2025-10-20T19:22:00.841332] [DEBUG] [VALIDATOR ] IIIN: All expected trading days are present. [2025-10-20T19:22:00.841459] [DEBUG] [VALIDATOR ] IIIN: Primary data validation passed. [2025-10-20T19:22:00.841996] [INFO ] [MAIN ] IIIN: Fetched primary data is valid. [2025-10-20T19:22:00.842056] [DEBUG] [LOCAL ] IIIN: Saving primary data. [2025-10-20T19:22:00.842673] [DEBUG] [LOCAL ] IIIN: Trimmed 24865 records to 16310 (keeping 200 days of data) [2025-10-20T19:22:00.939827] [DEBUG] [LOCAL ] IIIN: Primary data saved (16310 records). [2025-10-20T19:22:00.939952] [INFO ] [MAIN ] IIIN: Full history saved. [2025-10-20T19:22:00.940029] [INFO ] [MAIN ] Update Progress: 1102/2367 symbols completed. IIIN took 35.7s. Estimated remaining: 2h 42m 56s. Rate: 7.8 symbols/min [2025-10-20T19:22:01.056596] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-02. [2025-10-20T19:22:01.112739] [INFO ] [MAIN ] INFA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:01.156945] [DEBUG] [LOCAL ] INFA: Loaded 49646 existing records from primary storage. [2025-10-20T19:22:01.157327] [INFO ] [MAIN ] INFA: Loaded 49646 existing records. [2025-10-20T19:22:01.334435] [WARN ] [MAIN ] INFA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:01.334879] [INFO ] [MAIN ] INFA: Wiping existing primary data before full refresh. [2025-10-20T19:22:01.334936] [INFO ] [LOCAL ] INFA: Starting deletion of primary data. [2025-10-20T19:22:01.335411] [DEBUG] [LOCAL ] INFA: Deleted primary data file. [2025-10-20T19:22:01.335468] [INFO ] [MAIN ] INFA: Starting full primary refresh. [2025-10-20T19:22:01.709114] [INFO ] [REMOTE ] INFA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:01.851736] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-01. [2025-10-20T19:22:01.885404] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:22:02.622914] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:22:02.651363] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-03. [2025-10-20T19:22:03.157569] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T19:22:03.451381] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-02. [2025-10-20T19:22:04.079997] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T19:22:04.251391] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-04. [2025-10-20T19:22:05.051358] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-04. [2025-10-20T19:22:05.134531] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T19:22:05.840461] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T19:22:05.851334] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-05. [2025-10-20T19:22:06.658846] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-10. [2025-10-20T19:22:06.664159] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:22:07.153474] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T19:22:07.451364] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-03. [2025-10-20T19:22:07.959598] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T19:22:07.964134] [INFO ] [REMOTE ] IHS: Full fetch complete (51728 records). [2025-10-20T19:22:07.964225] [DEBUG] [VALIDATOR ] IHS: Validating primary data. [2025-10-20T19:22:08.257099] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-05. [2025-10-20T19:22:08.351059] [DEBUG] [VALIDATOR ] IHS: All expected trading days are present. [2025-10-20T19:22:08.351130] [DEBUG] [VALIDATOR ] IHS: Primary data validation passed. [2025-10-20T19:22:08.351934] [INFO ] [MAIN ] IHS: Fetched primary data is valid. [2025-10-20T19:22:08.351995] [DEBUG] [LOCAL ] IHS: Saving primary data. [2025-10-20T19:22:08.353014] [DEBUG] [LOCAL ] IHS: Trimmed 51728 records to 37267 (keeping 200 days of data) [2025-10-20T19:22:08.556250] [DEBUG] [LOCAL ] IHS: Primary data saved (37267 records). [2025-10-20T19:22:08.556395] [INFO ] [MAIN ] IHS: Full history saved. [2025-10-20T19:22:08.556475] [INFO ] [MAIN ] Update Progress: 1103/2367 symbols completed. IHS took 50.4s. Estimated remaining: 2h 42m 48s. Rate: 7.8 symbols/min [2025-10-20T19:22:08.741178] [INFO ] [MAIN ] INFY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:08.792496] [DEBUG] [LOCAL ] INFY: Loaded 54376 existing records from primary storage. [2025-10-20T19:22:08.792877] [INFO ] [MAIN ] INFY: Loaded 54376 existing records. [2025-10-20T19:22:08.985399] [WARN ] [MAIN ] INFY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:08.985613] [INFO ] [MAIN ] INFY: Wiping existing primary data before full refresh. [2025-10-20T19:22:08.985690] [INFO ] [LOCAL ] INFY: Starting deletion of primary data. [2025-10-20T19:22:08.985910] [DEBUG] [LOCAL ] INFY: Deleted primary data file. [2025-10-20T19:22:08.985964] [INFO ] [MAIN ] INFY: Starting full primary refresh. [2025-10-20T19:22:09.070516] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-06. [2025-10-20T19:22:09.353673] [INFO ] [REMOTE ] INFY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:09.382073] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T19:22:09.723321] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T19:22:09.851349] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-06. [2025-10-20T19:22:10.285545] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T19:22:10.651377] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-04. [2025-10-20T19:22:11.291149] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T19:22:11.451369] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-07. [2025-10-20T19:22:12.070922] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T19:22:12.251405] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-05. [2025-10-20T19:22:12.593586] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:22:13.051433] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-08. [2025-10-20T19:22:13.663760] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T19:22:13.851490] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-01. [2025-10-20T19:22:14.545439] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T19:22:14.651353] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-09. [2025-10-20T19:22:15.451411] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-06. [2025-10-20T19:22:15.644737] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T19:22:16.162600] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T19:22:16.251364] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-02. [2025-10-20T19:22:16.897840] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T19:22:17.051368] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-10. [2025-10-20T19:22:17.863808] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-07. [2025-10-20T19:22:17.889368] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:22:18.643957] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T19:22:18.650633] [INFO ] [REMOTE ] IMAX: Full fetch complete (60919 records). [2025-10-20T19:22:18.650775] [DEBUG] [VALIDATOR ] IMAX: Validating primary data. [2025-10-20T19:22:18.651360] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-03. [2025-10-20T19:22:19.046889] [DEBUG] [VALIDATOR ] IMAX: All expected trading days are present. [2025-10-20T19:22:19.047092] [DEBUG] [VALIDATOR ] IMAX: Primary data validation passed. [2025-10-20T19:22:19.048059] [INFO ] [MAIN ] IMAX: Fetched primary data is valid. [2025-10-20T19:22:19.048125] [DEBUG] [LOCAL ] IMAX: Saving primary data. [2025-10-20T19:22:19.049473] [DEBUG] [LOCAL ] IMAX: Trimmed 60919 records to 43484 (keeping 200 days of data) [2025-10-20T19:22:19.333931] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:22:19.335074] [DEBUG] [LOCAL ] IMAX: Primary data saved (43484 records). [2025-10-20T19:22:19.341385] [INFO ] [MAIN ] IMAX: Full history saved. [2025-10-20T19:22:19.341726] [INFO ] [MAIN ] Update Progress: 1104/2367 symbols completed. IMAX took 22.7s. Estimated remaining: 2h 42m 44s. Rate: 7.8 symbols/min [2025-10-20T19:22:19.456619] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-07. [2025-10-20T19:22:19.518072] [INFO ] [MAIN ] ING: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:19.567355] [DEBUG] [LOCAL ] ING: Loaded 51494 existing records from primary storage. [2025-10-20T19:22:19.567743] [INFO ] [MAIN ] ING: Loaded 51494 existing records. [2025-10-20T19:22:19.754785] [WARN ] [MAIN ] ING: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:19.755052] [INFO ] [MAIN ] ING: Wiping existing primary data before full refresh. [2025-10-20T19:22:19.755106] [INFO ] [LOCAL ] ING: Starting deletion of primary data. [2025-10-20T19:22:19.755373] [DEBUG] [LOCAL ] ING: Deleted primary data file. [2025-10-20T19:22:19.755429] [INFO ] [MAIN ] ING: Starting full primary refresh. [2025-10-20T19:22:20.135851] [INFO ] [REMOTE ] ING: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:20.251447] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-01. [2025-10-20T19:22:20.643828] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T19:22:21.051375] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-08. [2025-10-20T19:22:21.185977] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T19:22:21.851400] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-04. [2025-10-20T19:22:22.039184] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T19:22:22.476082] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:22:22.651387] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-09. [2025-10-20T19:22:23.451409] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-02. [2025-10-20T19:22:23.708718] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:22:23.995878] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T19:22:24.251384] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-10. [2025-10-20T19:22:24.748080] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:22:25.051377] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-08. [2025-10-20T19:22:25.511374] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T19:22:25.517691] [INFO ] [REMOTE ] IIPR: Full fetch complete (43873 records). [2025-10-20T19:22:25.517845] [DEBUG] [VALIDATOR ] IIPR: Validating primary data. [2025-10-20T19:22:25.860514] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-03. [2025-10-20T19:22:25.914699] [DEBUG] [VALIDATOR ] IIPR: All expected trading days are present. [2025-10-20T19:22:25.914785] [DEBUG] [VALIDATOR ] IIPR: Primary data validation passed. [2025-10-20T19:22:25.915469] [INFO ] [MAIN ] IIPR: Fetched primary data is valid. [2025-10-20T19:22:25.915541] [DEBUG] [LOCAL ] IIPR: Saving primary data. [2025-10-20T19:22:25.916696] [DEBUG] [LOCAL ] IIPR: Trimmed 43873 records to 29071 (keeping 200 days of data) [2025-10-20T19:22:26.096455] [DEBUG] [LOCAL ] IIPR: Primary data saved (29071 records). [2025-10-20T19:22:26.096591] [INFO ] [MAIN ] IIPR: Full history saved. [2025-10-20T19:22:26.096689] [INFO ] [MAIN ] Update Progress: 1105/2367 symbols completed. IIPR took 32.1s. Estimated remaining: 2h 42m 35s. Rate: 7.8 symbols/min [2025-10-20T19:22:26.267695] [INFO ] [MAIN ] INGM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:26.295685] [DEBUG] [LOCAL ] INGM: Loaded 28706 existing records from primary storage. [2025-10-20T19:22:26.295950] [INFO ] [MAIN ] INGM: Loaded 28706 existing records. [2025-10-20T19:22:26.476566] [WARN ] [MAIN ] INGM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:26.476931] [INFO ] [MAIN ] INGM: Wiping existing primary data before full refresh. [2025-10-20T19:22:26.477022] [INFO ] [LOCAL ] INGM: Starting deletion of primary data. [2025-10-20T19:22:26.477257] [DEBUG] [LOCAL ] INGM: Deleted primary data file. [2025-10-20T19:22:26.477308] [INFO ] [MAIN ] INGM: Starting full primary refresh. [2025-10-20T19:22:26.651682] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-05. [2025-10-20T19:22:26.924471] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T19:22:26.994561] [INFO ] [REMOTE ] INGM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:27.451406] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-09. [2025-10-20T19:22:27.749232] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T19:22:28.251383] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-04. [2025-10-20T19:22:28.730012] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T19:22:28.985378] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T19:22:29.051551] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-01. [2025-10-20T19:22:29.715174] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T19:22:29.825941] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T19:22:29.851377] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-02. [2025-10-20T19:22:30.651394] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-10. [2025-10-20T19:22:30.959809] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:22:31.451382] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-06. [2025-10-20T19:22:32.251383] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-03. [2025-10-20T19:22:32.326841] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:22:32.335473] [INFO ] [REMOTE ] INFA: Full fetch complete (70880 records). [2025-10-20T19:22:32.335703] [DEBUG] [VALIDATOR ] INFA: Validating primary data. [2025-10-20T19:22:32.740159] [DEBUG] [VALIDATOR ] INFA: All expected trading days are present. [2025-10-20T19:22:32.740259] [DEBUG] [VALIDATOR ] INFA: Primary data validation passed. [2025-10-20T19:22:32.741428] [INFO ] [MAIN ] INFA: Fetched primary data is valid. [2025-10-20T19:22:32.741576] [DEBUG] [LOCAL ] INFA: Saving primary data. [2025-10-20T19:22:32.743032] [DEBUG] [LOCAL ] INFA: Trimmed 70880 records to 48813 (keeping 200 days of data) [2025-10-20T19:22:32.982827] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:22:33.061260] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-07. [2025-10-20T19:22:33.078324] [DEBUG] [LOCAL ] INFA: Primary data saved (48813 records). [2025-10-20T19:22:33.078464] [INFO ] [MAIN ] INFA: Full history saved. [2025-10-20T19:22:33.078543] [INFO ] [MAIN ] Update Progress: 1106/2367 symbols completed. INFA took 32.1s. Estimated remaining: 2h 42m 27s. Rate: 7.8 symbols/min [2025-10-20T19:22:33.264533] [INFO ] [MAIN ] INGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:33.299518] [DEBUG] [LOCAL ] INGR: Loaded 34266 existing records from primary storage. [2025-10-20T19:22:33.299870] [INFO ] [MAIN ] INGR: Loaded 34266 existing records. [2025-10-20T19:22:33.486353] [WARN ] [MAIN ] INGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:33.486792] [INFO ] [MAIN ] INGR: Wiping existing primary data before full refresh. [2025-10-20T19:22:33.486854] [INFO ] [LOCAL ] INGR: Starting deletion of primary data. [2025-10-20T19:22:33.487208] [DEBUG] [LOCAL ] INGR: Deleted primary data file. [2025-10-20T19:22:33.487269] [INFO ] [MAIN ] INGR: Starting full primary refresh. [2025-10-20T19:22:33.845416] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T19:22:33.852721] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-05. [2025-10-20T19:22:33.934714] [INFO ] [REMOTE ] INGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:34.651360] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-01. [2025-10-20T19:22:35.023810] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T19:22:35.451367] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-08. [2025-10-20T19:22:35.574297] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T19:22:35.810190] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:22:36.251419] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-06. [2025-10-20T19:22:37.051402] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-02. [2025-10-20T19:22:37.308667] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T19:22:37.851429] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-04. [2025-10-20T19:22:37.909877] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:22:38.081398] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:22:38.651378] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-07. [2025-10-20T19:22:39.255222] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:22:39.451384] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-05. [2025-10-20T19:22:40.251340] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-09. [2025-10-20T19:22:40.436451] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:22:40.724402] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T19:22:41.051414] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-03. [2025-10-20T19:22:41.851377] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-06. [2025-10-20T19:22:42.036887] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T19:22:42.531827] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T19:22:42.651408] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-08. [2025-10-20T19:22:42.965702] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T19:22:43.451421] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-07. [2025-10-20T19:22:44.251388] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-04. [2025-10-20T19:22:44.408333] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T19:22:44.701323] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T19:22:45.051370] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-09. [2025-10-20T19:22:45.651019] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:22:45.851349] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-05. [2025-10-20T19:22:46.529291] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T19:22:46.651411] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-10. [2025-10-20T19:22:47.205328] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T19:22:47.451436] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-06. [2025-10-20T19:22:48.251351] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-08. [2025-10-20T19:22:48.477242] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T19:22:48.706326] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T19:22:48.714591] [INFO ] [REMOTE ] INFY: Full fetch complete (77747 records). [2025-10-20T19:22:48.714768] [DEBUG] [VALIDATOR ] INFY: Validating primary data. [2025-10-20T19:22:49.058694] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-10. [2025-10-20T19:22:49.113383] [DEBUG] [VALIDATOR ] INFY: All expected trading days are present. [2025-10-20T19:22:49.113541] [DEBUG] [VALIDATOR ] INFY: Primary data validation passed. [2025-10-20T19:22:49.114700] [INFO ] [MAIN ] INFY: Fetched primary data is valid. [2025-10-20T19:22:49.114784] [DEBUG] [LOCAL ] INFY: Saving primary data. [2025-10-20T19:22:49.116451] [DEBUG] [LOCAL ] INFY: Trimmed 77747 records to 53597 (keeping 200 days of data) [2025-10-20T19:22:49.237789] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:22:49.463385] [DEBUG] [LOCAL ] INFY: Primary data saved (53597 records). [2025-10-20T19:22:49.463528] [INFO ] [MAIN ] INFY: Full history saved. [2025-10-20T19:22:49.463605] [INFO ] [MAIN ] Update Progress: 1107/2367 symbols completed. INFY took 40.9s. Estimated remaining: 2h 42m 29s. Rate: 7.8 symbols/min [2025-10-20T19:22:49.635293] [INFO ] [MAIN ] INN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:49.673252] [DEBUG] [LOCAL ] INN: Loaded 42375 existing records from primary storage. [2025-10-20T19:22:49.673559] [INFO ] [MAIN ] INN: Loaded 42375 existing records. [2025-10-20T19:22:49.848536] [WARN ] [MAIN ] INN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:49.848831] [INFO ] [MAIN ] INN: Wiping existing primary data before full refresh. [2025-10-20T19:22:49.848883] [INFO ] [LOCAL ] INN: Starting deletion of primary data. [2025-10-20T19:22:49.849095] [DEBUG] [LOCAL ] INN: Deleted primary data file. [2025-10-20T19:22:49.849151] [INFO ] [MAIN ] INN: Starting full primary refresh. [2025-10-20T19:22:49.860221] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-07. [2025-10-20T19:22:50.197866] [INFO ] [REMOTE ] INN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:50.651394] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-09. [2025-10-20T19:22:51.003610] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:22:51.011159] [INFO ] [REMOTE ] ING: Full fetch complete (73819 records). [2025-10-20T19:22:51.011276] [DEBUG] [VALIDATOR ] ING: Validating primary data. [2025-10-20T19:22:51.406682] [DEBUG] [VALIDATOR ] ING: All expected trading days are present. [2025-10-20T19:22:51.421636] [DEBUG] [VALIDATOR ] ING: Primary data validation passed. [2025-10-20T19:22:51.434130] [INFO ] [MAIN ] ING: Fetched primary data is valid. [2025-10-20T19:22:51.434218] [DEBUG] [LOCAL ] ING: Saving primary data. [2025-10-20T19:22:51.436582] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:22:51.445044] [DEBUG] [LOCAL ] ING: Trimmed 73819 records to 50721 (keeping 200 days of data) [2025-10-20T19:22:51.497295] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-01. [2025-10-20T19:22:51.751527] [DEBUG] [LOCAL ] ING: Primary data saved (50721 records). [2025-10-20T19:22:51.751703] [INFO ] [MAIN ] ING: Full history saved. [2025-10-20T19:22:51.751816] [INFO ] [MAIN ] Update Progress: 1108/2367 symbols completed. ING took 32.4s. Estimated remaining: 2h 42m 15s. Rate: 7.8 symbols/min [2025-10-20T19:22:51.925256] [INFO ] [MAIN ] INR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:51.925409] [DEBUG] [LOCAL ] INR: No existing data file found in primary storage. [2025-10-20T19:22:51.925445] [WARN ] [MAIN ] INR: No existing data found, performing full refresh instead. [2025-10-20T19:22:51.925549] [INFO ] [MAIN ] INR: Wiping existing primary data before full refresh. [2025-10-20T19:22:51.925589] [INFO ] [LOCAL ] INR: Starting deletion of primary data. [2025-10-20T19:22:51.925635] [DEBUG] [LOCAL ] INR: No primary data file found to delete. [2025-10-20T19:22:51.925721] [INFO ] [MAIN ] INR: Starting full primary refresh. [2025-10-20T19:22:52.253789] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-08. [2025-10-20T19:22:52.275209] [INFO ] [REMOTE ] INR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:53.051376] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-01. [2025-10-20T19:23:05.008921] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-01 (11.96 seconds). [2025-10-20T19:23:05.009970] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-02. [2025-10-20T19:23:05.276470] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-09 (14.63 seconds). [2025-10-20T19:23:05.282858] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-10. [2025-10-20T19:23:06.574798] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T19:23:06.579415] [INFO ] [REMOTE ] INGM: Full fetch complete (41766 records). [2025-10-20T19:23:06.579523] [DEBUG] [VALIDATOR ] INGM: Validating primary data. [2025-10-20T19:23:06.975221] [DEBUG] [VALIDATOR ] INGM: All expected trading days are present. [2025-10-20T19:23:06.975305] [DEBUG] [VALIDATOR ] INGM: Primary data validation passed. [2025-10-20T19:23:06.976013] [INFO ] [MAIN ] INGM: Fetched primary data is valid. [2025-10-20T19:23:06.976076] [DEBUG] [LOCAL ] INGM: Saving primary data. [2025-10-20T19:23:06.977189] [DEBUG] [LOCAL ] INGM: Trimmed 41766 records to 28268 (keeping 200 days of data) [2025-10-20T19:23:07.156699] [DEBUG] [LOCAL ] INGM: Primary data saved (28268 records). [2025-10-20T19:23:07.156851] [INFO ] [MAIN ] INGM: Full history saved. [2025-10-20T19:23:07.156935] [INFO ] [MAIN ] Update Progress: 1109/2367 symbols completed. INGM took 41.1s. Estimated remaining: 2h 42m 16s. Rate: 7.8 symbols/min [2025-10-20T19:23:07.333933] [INFO ] [MAIN ] INSP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:07.364516] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-01 (15.87 seconds). [2025-10-20T19:23:07.373314] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-02. [2025-10-20T19:23:07.408838] [DEBUG] [LOCAL ] INSP: Loaded 38922 existing records from primary storage. [2025-10-20T19:23:07.409179] [INFO ] [MAIN ] INSP: Loaded 38922 existing records. [2025-10-20T19:23:07.612350] [WARN ] [MAIN ] INSP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:07.612592] [INFO ] [MAIN ] INSP: Wiping existing primary data before full refresh. [2025-10-20T19:23:07.612639] [INFO ] [LOCAL ] INSP: Starting deletion of primary data. [2025-10-20T19:23:07.612867] [DEBUG] [LOCAL ] INSP: Deleted primary data file. [2025-10-20T19:23:07.612915] [INFO ] [MAIN ] INSP: Starting full primary refresh. [2025-10-20T19:23:07.975596] [INFO ] [REMOTE ] INSP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:07.976015] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-01. [2025-10-20T19:23:08.715253] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T19:23:08.726537] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-03. [2025-10-20T19:23:09.352140] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-08 (17.10 seconds). [2025-10-20T19:23:09.359720] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-09. [2025-10-20T19:23:09.394077] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:23:09.401764] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-02. [2025-10-20T19:23:10.201320] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T19:23:10.209481] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-04. [2025-10-20T19:23:10.391036] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T19:23:10.398395] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-03. [2025-10-20T19:23:10.482971] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-02 (5.47 seconds). [2025-10-20T19:23:10.492402] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-03. [2025-10-20T19:23:10.565222] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T19:23:10.574049] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-10. [2025-10-20T19:23:11.857248] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:23:11.865521] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-04. [2025-10-20T19:23:11.894817] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T19:23:11.904918] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-05. [2025-10-20T19:23:11.932396] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:23:11.941536] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-04. [2025-10-20T19:23:12.054432] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T19:23:12.060287] [INFO ] [REMOTE ] INGR: Full fetch complete (50728 records). [2025-10-20T19:23:12.060414] [DEBUG] [VALIDATOR ] INGR: Validating primary data. [2025-10-20T19:23:12.450933] [DEBUG] [VALIDATOR ] INGR: All expected trading days are present. [2025-10-20T19:23:12.451020] [DEBUG] [VALIDATOR ] INGR: Primary data validation passed. [2025-10-20T19:23:12.451794] [INFO ] [MAIN ] INGR: Fetched primary data is valid. [2025-10-20T19:23:12.451857] [DEBUG] [LOCAL ] INGR: Saving primary data. [2025-10-20T19:23:12.453022] [DEBUG] [LOCAL ] INGR: Trimmed 50728 records to 33774 (keeping 200 days of data) [2025-10-20T19:23:12.669164] [DEBUG] [LOCAL ] INGR: Primary data saved (33774 records). [2025-10-20T19:23:12.669293] [INFO ] [MAIN ] INGR: Full history saved. [2025-10-20T19:23:12.669367] [INFO ] [MAIN ] Update Progress: 1110/2367 symbols completed. INGR took 39.6s. Estimated remaining: 2h 42m 6s. Rate: 7.8 symbols/min [2025-10-20T19:23:12.842763] [INFO ] [MAIN ] INSW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:12.881547] [DEBUG] [LOCAL ] INSW: Loaded 40192 existing records from primary storage. [2025-10-20T19:23:12.882008] [INFO ] [MAIN ] INSW: Loaded 40192 existing records. [2025-10-20T19:23:13.088602] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T19:23:13.087558] [WARN ] [MAIN ] INSW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:13.113820] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-05. [2025-10-20T19:23:13.110919] [INFO ] [MAIN ] INSW: Wiping existing primary data before full refresh. [2025-10-20T19:23:13.115022] [INFO ] [LOCAL ] INSW: Starting deletion of primary data. [2025-10-20T19:23:13.115475] [DEBUG] [LOCAL ] INSW: Deleted primary data file. [2025-10-20T19:23:13.115997] [INFO ] [MAIN ] INSW: Starting full primary refresh. [2025-10-20T19:23:13.117044] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T19:23:13.194308] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-05. [2025-10-20T19:23:13.519219] [INFO ] [REMOTE ] INSW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:13.519616] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-01. [2025-10-20T19:23:13.550063] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:23:13.559423] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-06. [2025-10-20T19:23:14.113187] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T19:23:14.119395] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-06. [2025-10-20T19:23:14.507722] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:23:14.515395] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-06. [2025-10-20T19:23:14.797287] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T19:23:14.803518] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-07. [2025-10-20T19:23:15.070376] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T19:23:15.081974] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-07. [2025-10-20T19:23:15.187382] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T19:23:15.197113] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-02. [2025-10-20T19:23:15.703959] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:23:15.711925] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-08. [2025-10-20T19:23:15.870444] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T19:23:15.877992] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-07. [2025-10-20T19:23:16.188302] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T19:23:16.198464] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-08. [2025-10-20T19:23:16.253884] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T19:23:16.262543] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-03. [2025-10-20T19:23:17.063795] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:23:17.070598] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-09. [2025-10-20T19:23:17.181572] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T19:23:17.190436] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-08. [2025-10-20T19:23:17.349877] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T19:23:17.359379] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-09. [2025-10-20T19:23:17.842942] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:23:17.852097] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-04. [2025-10-20T19:23:17.964057] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T19:23:18.651490] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-10. [2025-10-20T19:23:18.710573] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:23:18.745465] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T19:23:19.451443] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-10. [2025-10-20T19:23:19.658990] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T19:23:19.989504] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T19:23:19.996244] [INFO ] [REMOTE ] INR: Full fetch complete (36887 records). [2025-10-20T19:23:19.996456] [DEBUG] [VALIDATOR ] INR: Validating primary data. [2025-10-20T19:23:20.245626] [WARN ] [VALIDATOR ] INR: Validation failed - insufficient historical data. Have 181 unique days, need 200 days [2025-10-20T19:23:20.246232] [FATAL] [MAIN ] INR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:23:20.246329] [WARN ] [MAIN ] INR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:23:20.246403] [INFO ] [LOCAL ] INR: Starting deletion of primary data. [2025-10-20T19:23:20.246475] [DEBUG] [LOCAL ] INR: No primary data file found to delete. [2025-10-20T19:23:20.246510] [INFO ] [LOCAL ] INR: Starting deletion of derivative data. [2025-10-20T19:23:20.246549] [DEBUG] [LOCAL ] INR: No derivative data file found to delete. [2025-10-20T19:23:20.246613] [INFO ] [MAIN ] Update Progress: 1111/2367 symbols completed. INR took 28.5s. Estimated remaining: 2h 41m 58s. Rate: 7.8 symbols/min [2025-10-20T19:23:20.252103] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-09. [2025-10-20T19:23:20.453505] [INFO ] [MAIN ] INVH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:20.504587] [DEBUG] [LOCAL ] INVH: Loaded 53973 existing records from primary storage. [2025-10-20T19:23:20.505032] [INFO ] [MAIN ] INVH: Loaded 53973 existing records. [2025-10-20T19:23:20.684700] [WARN ] [MAIN ] INVH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:20.684961] [INFO ] [MAIN ] INVH: Wiping existing primary data before full refresh. [2025-10-20T19:23:20.685078] [INFO ] [LOCAL ] INVH: Starting deletion of primary data. [2025-10-20T19:23:20.685408] [DEBUG] [LOCAL ] INVH: Deleted primary data file. [2025-10-20T19:23:20.685501] [INFO ] [MAIN ] INVH: Starting full primary refresh. [2025-10-20T19:23:20.889265] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T19:23:20.974017] [INFO ] [REMOTE ] INN: Full fetch complete (57765 records). [2025-10-20T19:23:20.995013] [DEBUG] [VALIDATOR ] INN: Validating primary data. [2025-10-20T19:23:21.057836] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-05. [2025-10-20T19:23:21.190293] [INFO ] [REMOTE ] INVH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:21.467706] [DEBUG] [VALIDATOR ] INN: All expected trading days are present. [2025-10-20T19:23:21.467775] [DEBUG] [VALIDATOR ] INN: Primary data validation passed. [2025-10-20T19:23:21.468793] [INFO ] [MAIN ] INN: Fetched primary data is valid. [2025-10-20T19:23:21.468856] [DEBUG] [LOCAL ] INN: Saving primary data. [2025-10-20T19:23:21.470179] [DEBUG] [LOCAL ] INN: Trimmed 57765 records to 41658 (keeping 200 days of data) [2025-10-20T19:23:21.710078] [DEBUG] [LOCAL ] INN: Primary data saved (41658 records). [2025-10-20T19:23:21.710194] [INFO ] [MAIN ] INN: Full history saved. [2025-10-20T19:23:21.710266] [INFO ] [MAIN ] Update Progress: 1112/2367 symbols completed. INN took 32.2s. Estimated remaining: 2h 41m 43s. Rate: 7.8 symbols/min [2025-10-20T19:23:21.856693] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-01. [2025-10-20T19:23:21.883088] [INFO ] [MAIN ] INVX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:21.884210] [DEBUG] [LOCAL ] INVX: No existing data file found in primary storage. [2025-10-20T19:23:21.884387] [WARN ] [MAIN ] INVX: No existing data found, performing full refresh instead. [2025-10-20T19:23:21.884549] [INFO ] [MAIN ] INVX: Wiping existing primary data before full refresh. [2025-10-20T19:23:21.884612] [INFO ] [LOCAL ] INVX: Starting deletion of primary data. [2025-10-20T19:23:21.884815] [DEBUG] [LOCAL ] INVX: No primary data file found to delete. [2025-10-20T19:23:21.884886] [INFO ] [MAIN ] INVX: Starting full primary refresh. [2025-10-20T19:23:22.179284] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T19:23:22.307146] [INFO ] [REMOTE ] INVX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:22.662329] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-10. [2025-10-20T19:23:22.666196] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:23:23.451373] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-01. [2025-10-20T19:23:23.638436] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:23:24.251370] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-02. [2025-10-20T19:23:24.578740] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T19:23:24.586201] [INFO ] [REMOTE ] INSP: Full fetch complete (53858 records). [2025-10-20T19:23:24.586326] [DEBUG] [VALIDATOR ] INSP: Validating primary data. [2025-10-20T19:23:24.852335] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T19:23:25.030016] [DEBUG] [VALIDATOR ] INSP: All expected trading days are present. [2025-10-20T19:23:25.030102] [DEBUG] [VALIDATOR ] INSP: Primary data validation passed. [2025-10-20T19:23:25.030936] [INFO ] [MAIN ] INSP: Fetched primary data is valid. [2025-10-20T19:23:25.031000] [DEBUG] [LOCAL ] INSP: Saving primary data. [2025-10-20T19:23:25.032237] [DEBUG] [LOCAL ] INSP: Trimmed 53858 records to 38552 (keeping 200 days of data) [2025-10-20T19:23:25.085286] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-06. [2025-10-20T19:23:25.268778] [DEBUG] [LOCAL ] INSP: Primary data saved (38552 records). [2025-10-20T19:23:25.268907] [INFO ] [MAIN ] INSP: Full history saved. [2025-10-20T19:23:25.268984] [INFO ] [MAIN ] Update Progress: 1113/2367 symbols completed. INSP took 18.1s. Estimated remaining: 2h 41m 30s. Rate: 7.8 symbols/min [2025-10-20T19:23:25.441231] [INFO ] [MAIN ] IONQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:25.495150] [DEBUG] [LOCAL ] IONQ: Loaded 54568 existing records from primary storage. [2025-10-20T19:23:25.495449] [INFO ] [MAIN ] IONQ: Loaded 54568 existing records. [2025-10-20T19:23:25.689001] [WARN ] [MAIN ] IONQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:25.689214] [INFO ] [MAIN ] IONQ: Wiping existing primary data before full refresh. [2025-10-20T19:23:25.689262] [INFO ] [LOCAL ] IONQ: Starting deletion of primary data. [2025-10-20T19:23:25.689443] [DEBUG] [LOCAL ] IONQ: Deleted primary data file. [2025-10-20T19:23:25.689481] [INFO ] [MAIN ] IONQ: Starting full primary refresh. [2025-10-20T19:23:25.855489] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-02. [2025-10-20T19:23:26.112204] [INFO ] [REMOTE ] IONQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:26.115368] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T19:23:26.650926] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:23:26.658911] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-03. [2025-10-20T19:23:27.226215] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T19:23:27.451406] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-01. [2025-10-20T19:23:28.251390] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-07. [2025-10-20T19:23:28.407505] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:23:29.051392] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-03. [2025-10-20T19:23:29.666743] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:23:29.851389] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-08. [2025-10-20T19:23:29.972372] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-01 (2.52 seconds). [2025-10-20T19:23:29.999524] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T19:23:30.651432] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-02. [2025-10-20T19:23:31.351854] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:23:31.451408] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-09. [2025-10-20T19:23:32.251421] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-04. [2025-10-20T19:23:32.868382] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:23:33.051374] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-04. [2025-10-20T19:23:33.119530] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-02 (2.47 seconds). [2025-10-20T19:23:33.662107] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:23:33.851422] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-05. [2025-10-20T19:23:34.535466] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T19:23:34.651432] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-10. [2025-10-20T19:23:35.159730] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:23:35.451392] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-03. [2025-10-20T19:23:35.957752] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:23:35.963378] [INFO ] [REMOTE ] INSW: Full fetch complete (59762 records). [2025-10-20T19:23:35.963523] [DEBUG] [VALIDATOR ] INSW: Validating primary data. [2025-10-20T19:23:36.260002] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-05. [2025-10-20T19:23:36.356996] [DEBUG] [VALIDATOR ] INSW: All expected trading days are present. [2025-10-20T19:23:36.357083] [DEBUG] [VALIDATOR ] INSW: Primary data validation passed. [2025-10-20T19:23:36.357922] [INFO ] [MAIN ] INSW: Fetched primary data is valid. [2025-10-20T19:23:36.357984] [DEBUG] [LOCAL ] INSW: Saving primary data. [2025-10-20T19:23:36.359406] [DEBUG] [LOCAL ] INSW: Trimmed 59762 records to 39541 (keeping 200 days of data) [2025-10-20T19:23:36.598131] [DEBUG] [LOCAL ] INSW: Primary data saved (39541 records). [2025-10-20T19:23:36.598235] [INFO ] [MAIN ] INSW: Full history saved. [2025-10-20T19:23:36.598313] [INFO ] [MAIN ] Update Progress: 1114/2367 symbols completed. INSW took 23.9s. Estimated remaining: 2h 41m 27s. Rate: 7.8 symbols/min [2025-10-20T19:23:36.767159] [INFO ] [MAIN ] IOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:36.818854] [DEBUG] [LOCAL ] IOT: Loaded 54260 existing records from primary storage. [2025-10-20T19:23:36.819157] [INFO ] [MAIN ] IOT: Loaded 54260 existing records. [2025-10-20T19:23:36.991513] [WARN ] [MAIN ] IOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:36.991742] [INFO ] [MAIN ] IOT: Wiping existing primary data before full refresh. [2025-10-20T19:23:36.991827] [INFO ] [LOCAL ] IOT: Starting deletion of primary data. [2025-10-20T19:23:36.992142] [DEBUG] [LOCAL ] IOT: Deleted primary data file. [2025-10-20T19:23:36.992179] [INFO ] [MAIN ] IOT: Starting full primary refresh. [2025-10-20T19:23:37.060099] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-06. [2025-10-20T19:23:37.350576] [INFO ] [REMOTE ] IOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:37.851417] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-01. [2025-10-20T19:23:38.150197] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T19:23:38.209822] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-03 (2.76 seconds). [2025-10-20T19:23:38.421729] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T19:23:38.651419] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-06. [2025-10-20T19:23:39.451436] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-04. [2025-10-20T19:23:39.621436] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T19:23:40.251400] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-02. [2025-10-20T19:23:40.354325] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:23:41.051432] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-07. [2025-10-20T19:23:41.851361] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-07. [2025-10-20T19:23:42.009618] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:23:42.055826] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-04 (2.60 seconds). [2025-10-20T19:23:42.651345] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-05. [2025-10-20T19:23:42.863044] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T19:23:42.895507] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T19:23:43.451368] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-03. [2025-10-20T19:23:44.251431] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-08. [2025-10-20T19:23:45.051372] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-08. [2025-10-20T19:23:45.670787] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-03 (2.22 seconds). [2025-10-20T19:23:45.697246] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T19:23:45.790300] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-05 (3.14 seconds). [2025-10-20T19:23:45.851339] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-06. [2025-10-20T19:23:46.546383] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:23:46.651368] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-09. [2025-10-20T19:23:47.451365] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-09. [2025-10-20T19:23:48.251347] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-04. [2025-10-20T19:23:48.446922] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:23:48.486684] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-06 (2.64 seconds). [2025-10-20T19:23:48.512117] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:23:49.051385] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-10. [2025-10-20T19:23:49.851386] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-07. [2025-10-20T19:23:50.044137] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T19:23:50.651354] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-10. [2025-10-20T19:23:50.883764] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T19:23:50.891536] [INFO ] [REMOTE ] INVH: Full fetch complete (76963 records). [2025-10-20T19:23:50.891693] [DEBUG] [VALIDATOR ] INVH: Validating primary data. [2025-10-20T19:23:51.277455] [DEBUG] [VALIDATOR ] INVH: All expected trading days are present. [2025-10-20T19:23:51.277533] [DEBUG] [VALIDATOR ] INVH: Primary data validation passed. [2025-10-20T19:23:51.278476] [INFO ] [MAIN ] INVH: Fetched primary data is valid. [2025-10-20T19:23:51.278540] [DEBUG] [LOCAL ] INVH: Saving primary data. [2025-10-20T19:23:51.279899] [DEBUG] [LOCAL ] INVH: Trimmed 76963 records to 53195 (keeping 200 days of data) [2025-10-20T19:23:51.458459] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-05. [2025-10-20T19:23:51.597476] [DEBUG] [LOCAL ] INVH: Primary data saved (53195 records). [2025-10-20T19:23:51.597618] [INFO ] [MAIN ] INVH: Full history saved. [2025-10-20T19:23:51.597870] [INFO ] [MAIN ] Update Progress: 1115/2367 symbols completed. INVH took 31.4s. Estimated remaining: 2h 41m 27s. Rate: 7.8 symbols/min [2025-10-20T19:23:51.664446] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:23:51.720640] [INFO ] [REMOTE ] INVX: Full fetch complete (42936 records). [2025-10-20T19:23:51.731274] [DEBUG] [VALIDATOR ] INVX: Validating primary data. [2025-10-20T19:23:51.898951] [INFO ] [MAIN ] IP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:52.204291] [DEBUG] [VALIDATOR ] INVX: All expected trading days are present. [2025-10-20T19:23:52.204467] [DEBUG] [VALIDATOR ] INVX: Primary data validation passed. [2025-10-20T19:23:52.219729] [INFO ] [MAIN ] INVX: Fetched primary data is valid. [2025-10-20T19:23:52.220681] [DEBUG] [LOCAL ] INVX: Saving primary data. [2025-10-20T19:23:52.222031] [DEBUG] [LOCAL ] IP: Loaded 54087 existing records from primary storage. [2025-10-20T19:23:52.222359] [INFO ] [MAIN ] IP: Loaded 54087 existing records. [2025-10-20T19:23:52.324908] [DEBUG] [LOCAL ] INVX: Trimmed 42936 records to 29241 (keeping 200 days of data) [2025-10-20T19:23:52.533527] [WARN ] [MAIN ] IP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:52.573705] [INFO ] [MAIN ] IP: Wiping existing primary data before full refresh. [2025-10-20T19:23:52.578603] [INFO ] [LOCAL ] IP: Starting deletion of primary data. [2025-10-20T19:23:52.598436] [DEBUG] [LOCAL ] IP: Deleted primary data file. [2025-10-20T19:23:52.598851] [INFO ] [MAIN ] IP: Starting full primary refresh. [2025-10-20T19:23:52.664101] [DEBUG] [LOCAL ] INVX: Primary data saved (29241 records). [2025-10-20T19:23:52.678093] [INFO ] [MAIN ] INVX: Full history saved. [2025-10-20T19:23:52.690311] [INFO ] [MAIN ] Update Progress: 1116/2367 symbols completed. INVX took 31.0s. Estimated remaining: 2h 41m 12s. Rate: 7.8 symbols/min [2025-10-20T19:23:53.050187] [INFO ] [MAIN ] IPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:53.177195] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-07 (3.33 seconds). [2025-10-20T19:23:53.203415] [INFO ] [REMOTE ] IP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:53.211551] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-01. [2025-10-20T19:23:53.218388] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-08. [2025-10-20T19:23:53.258014] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T19:23:53.303130] [DEBUG] [LOCAL ] IPG: Loaded 54111 existing records from primary storage. [2025-10-20T19:23:53.303449] [INFO ] [MAIN ] IPG: Loaded 54111 existing records. [2025-10-20T19:23:53.480929] [WARN ] [MAIN ] IPG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:53.481172] [INFO ] [MAIN ] IPG: Wiping existing primary data before full refresh. [2025-10-20T19:23:53.481230] [INFO ] [LOCAL ] IPG: Starting deletion of primary data. [2025-10-20T19:23:53.481472] [DEBUG] [LOCAL ] IPG: Deleted primary data file. [2025-10-20T19:23:53.481528] [INFO ] [MAIN ] IPG: Starting full primary refresh. [2025-10-20T19:23:53.831833] [INFO ] [REMOTE ] IPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:53.851334] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-01. [2025-10-20T19:23:54.651428] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-06. [2025-10-20T19:23:54.966410] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T19:23:55.451374] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-02. [2025-10-20T19:23:55.613879] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:23:55.920604] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-08 (2.70 seconds). [2025-10-20T19:23:56.251399] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-09. [2025-10-20T19:23:56.586030] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T19:23:56.743606] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:23:57.051376] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-07. [2025-10-20T19:23:57.851380] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-03. [2025-10-20T19:23:58.651356] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-02. [2025-10-20T19:23:58.886993] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-09 (2.64 seconds). [2025-10-20T19:23:59.019056] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T19:23:59.451382] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-08. [2025-10-20T19:23:59.644063] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T19:24:00.237982] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T19:24:00.251334] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-03. [2025-10-20T19:24:01.051375] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-10. [2025-10-20T19:24:01.733337] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-08 (2.28 seconds). [2025-10-20T19:24:01.851355] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-09. [2025-10-20T19:24:02.567876] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-03 (2.32 seconds). [2025-10-20T19:24:02.651383] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-04. [2025-10-20T19:24:03.451383] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-04. [2025-10-20T19:24:03.522448] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T19:24:03.999780] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-10 (2.95 seconds). [2025-10-20T19:24:04.007867] [INFO ] [REMOTE ] IONQ: Full fetch complete (77968 records). [2025-10-20T19:24:04.008028] [DEBUG] [VALIDATOR ] IONQ: Validating primary data. [2025-10-20T19:24:04.257877] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-10. [2025-10-20T19:24:04.423000] [DEBUG] [VALIDATOR ] IONQ: All expected trading days are present. [2025-10-20T19:24:04.423171] [DEBUG] [VALIDATOR ] IONQ: Primary data validation passed. [2025-10-20T19:24:04.424363] [INFO ] [MAIN ] IONQ: Fetched primary data is valid. [2025-10-20T19:24:04.424467] [DEBUG] [LOCAL ] IONQ: Saving primary data. [2025-10-20T19:24:04.426241] [DEBUG] [LOCAL ] IONQ: Trimmed 77968 records to 53788 (keeping 200 days of data) [2025-10-20T19:24:04.548562] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T19:24:04.784047] [DEBUG] [LOCAL ] IONQ: Primary data saved (53788 records). [2025-10-20T19:24:04.784180] [INFO ] [MAIN ] IONQ: Full history saved. [2025-10-20T19:24:04.785032] [INFO ] [MAIN ] Update Progress: 1117/2367 symbols completed. IONQ took 39.5s. Estimated remaining: 2h 41m 9s. Rate: 7.8 symbols/min [2025-10-20T19:24:05.051417] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-05. [2025-10-20T19:24:06.306224] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-04 (2.85 seconds). [2025-10-20T19:24:06.317208] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-05. [2025-10-20T19:24:07.984742] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-05 (2.93 seconds). [2025-10-20T19:24:07.996103] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-06. [2025-10-20T19:24:08.212622] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:24:08.223821] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-06. [2025-10-20T19:24:08.735256] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-10 (4.48 seconds). [2025-10-20T19:24:08.742941] [INFO ] [REMOTE ] IOT: Full fetch complete (77553 records). [2025-10-20T19:24:08.743056] [DEBUG] [VALIDATOR ] IOT: Validating primary data. [2025-10-20T19:24:09.132562] [DEBUG] [VALIDATOR ] IOT: All expected trading days are present. [2025-10-20T19:24:09.132699] [DEBUG] [VALIDATOR ] IOT: Primary data validation passed. [2025-10-20T19:24:09.133671] [INFO ] [MAIN ] IOT: Fetched primary data is valid. [2025-10-20T19:24:09.133732] [DEBUG] [LOCAL ] IOT: Saving primary data. [2025-10-20T19:24:09.135079] [DEBUG] [LOCAL ] IOT: Trimmed 77553 records to 53478 (keeping 200 days of data) [2025-10-20T19:24:09.443048] [DEBUG] [LOCAL ] IOT: Primary data saved (53478 records). [2025-10-20T19:24:09.443351] [INFO ] [MAIN ] IOT: Full history saved. [2025-10-20T19:24:09.443584] [INFO ] [MAIN ] Update Progress: 1118/2367 symbols completed. IOT took 32.8s. Estimated remaining: 2h 40m 58s. Rate: 7.8 symbols/min [2025-10-20T19:24:09.473826] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T19:24:09.485458] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-07. [2025-10-20T19:24:09.982574] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:24:09.993177] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-07. [2025-10-20T19:24:11.316683] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T19:24:11.328113] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-08. [2025-10-20T19:24:11.644211] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T19:24:11.657145] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-08. [2025-10-20T19:24:13.044455] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:24:13.060743] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-09. [2025-10-20T19:24:13.489918] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T19:24:13.501239] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-09. [2025-10-20T19:24:14.778010] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T19:24:14.790268] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-10. [2025-10-20T19:24:15.260740] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T19:24:15.272303] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-10. [2025-10-20T19:24:16.926808] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-10-20T19:24:16.938027] [INFO ] [REMOTE ] IPG: Full fetch complete (77434 records). [2025-10-20T19:24:16.938195] [DEBUG] [VALIDATOR ] IPG: Validating primary data. [2025-10-20T19:24:17.390242] [DEBUG] [VALIDATOR ] IPG: All expected trading days are present. [2025-10-20T19:24:17.390348] [DEBUG] [VALIDATOR ] IPG: Primary data validation passed. [2025-10-20T19:24:17.392176] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T19:24:17.393488] [INFO ] [MAIN ] IPG: Fetched primary data is valid. [2025-10-20T19:24:17.397376] [DEBUG] [LOCAL ] IPG: Saving primary data. [2025-10-20T19:24:17.405497] [INFO ] [REMOTE ] IP: Full fetch complete (77413 records). [2025-10-20T19:24:17.405845] [DEBUG] [VALIDATOR ] IP: Validating primary data. [2025-10-20T19:24:17.406098] [DEBUG] [LOCAL ] IPG: Trimmed 77434 records to 53330 (keeping 200 days of data) [2025-10-20T19:24:18.100868] [DEBUG] [VALIDATOR ] IP: All expected trading days are present. [2025-10-20T19:24:18.103164] [DEBUG] [VALIDATOR ] IP: Primary data validation passed. [2025-10-20T19:24:18.104518] [INFO ] [MAIN ] IP: Fetched primary data is valid. [2025-10-20T19:24:18.104888] [DEBUG] [LOCAL ] IP: Saving primary data. [2025-10-20T19:24:18.126120] [DEBUG] [LOCAL ] IP: Trimmed 77413 records to 53306 (keeping 200 days of data) [2025-10-20T19:24:18.317710] [DEBUG] [LOCAL ] IPG: Primary data saved (53330 records). [2025-10-20T19:24:18.329010] [INFO ] [MAIN ] IPG: Full history saved. [2025-10-20T19:24:18.334891] [INFO ] [MAIN ] Update Progress: 1119/2367 symbols completed. IPG took 25.6s. Estimated remaining: 2h 40m 52s. Rate: 7.8 symbols/min [2025-10-20T19:24:18.540955] [DEBUG] [LOCAL ] IP: Primary data saved (53306 records). [2025-10-20T19:24:18.541097] [INFO ] [MAIN ] IP: Full history saved. [2025-10-20T19:24:18.541175] [INFO ] [MAIN ] Update Progress: 1120/2367 symbols completed. IP took 26.9s. Estimated remaining: 2h 40m 35s. Rate: 7.8 symbols/min [2025-10-20T19:24:19.157139] [INFO ] [MAIN ] IPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:19.243418] [INFO ] [MAIN ] IQI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:19.265553] [INFO ] [MAIN ] IR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:19.269629] [INFO ] [MAIN ] IQV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:19.294431] [DEBUG] [LOCAL ] IQI: Loaded 18096 existing records from primary storage. [2025-10-20T19:24:19.294874] [INFO ] [MAIN ] IQI: Loaded 18096 existing records. [2025-10-20T19:24:19.337630] [DEBUG] [LOCAL ] IPI: Loaded 21783 existing records from primary storage. [2025-10-20T19:24:19.379796] [INFO ] [MAIN ] IPI: Loaded 21783 existing records. [2025-10-20T19:24:19.552756] [DEBUG] [LOCAL ] IR: Loaded 53548 existing records from primary storage. [2025-10-20T19:24:19.629298] [INFO ] [MAIN ] IR: Loaded 53548 existing records. [2025-10-20T19:24:19.709728] [DEBUG] [LOCAL ] IQV: Loaded 50750 existing records from primary storage. [2025-10-20T19:24:19.783313] [WARN ] [MAIN ] IQI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:19.821604] [INFO ] [MAIN ] IQV: Loaded 50750 existing records. [2025-10-20T19:24:19.868028] [INFO ] [MAIN ] IQI: Wiping existing primary data before full refresh. [2025-10-20T19:24:19.885267] [WARN ] [MAIN ] IPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:19.920730] [INFO ] [LOCAL ] IQI: Starting deletion of primary data. [2025-10-20T19:24:19.915490] [INFO ] [MAIN ] IPI: Wiping existing primary data before full refresh. [2025-10-20T19:24:19.982126] [INFO ] [LOCAL ] IPI: Starting deletion of primary data. [2025-10-20T19:24:19.982039] [DEBUG] [LOCAL ] IQI: Deleted primary data file. [2025-10-20T19:24:20.024783] [WARN ] [MAIN ] IR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:20.025236] [DEBUG] [LOCAL ] IPI: Deleted primary data file. [2025-10-20T19:24:20.025068] [INFO ] [MAIN ] IQI: Starting full primary refresh. [2025-10-20T19:24:20.032831] [INFO ] [MAIN ] IPI: Starting full primary refresh. [2025-10-20T19:24:20.041686] [INFO ] [MAIN ] IR: Wiping existing primary data before full refresh. [2025-10-20T19:24:20.112483] [INFO ] [LOCAL ] IR: Starting deletion of primary data. [2025-10-20T19:24:20.234414] [DEBUG] [LOCAL ] IR: Deleted primary data file. [2025-10-20T19:24:20.311636] [INFO ] [MAIN ] IR: Starting full primary refresh. [2025-10-20T19:24:20.531270] [WARN ] [MAIN ] IQV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:20.669499] [INFO ] [MAIN ] IQV: Wiping existing primary data before full refresh. [2025-10-20T19:24:20.697340] [INFO ] [LOCAL ] IQV: Starting deletion of primary data. [2025-10-20T19:24:20.901105] [DEBUG] [LOCAL ] IQV: Deleted primary data file. [2025-10-20T19:24:20.963573] [INFO ] [MAIN ] IQV: Starting full primary refresh. [2025-10-20T19:24:21.426949] [INFO ] [REMOTE ] IQI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:21.442342] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-01. [2025-10-20T19:24:21.436228] [INFO ] [REMOTE ] IPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:21.443829] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-01. [2025-10-20T19:24:21.629470] [INFO ] [REMOTE ] IR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:21.665771] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-01. [2025-10-20T19:24:21.773434] [INFO ] [REMOTE ] IQV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:21.773773] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-01. [2025-10-20T19:24:22.555803] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T19:24:22.559890] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-02. [2025-10-20T19:24:22.585472] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:24:22.589061] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-02. [2025-10-20T19:24:23.139117] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:24:23.148902] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-02. [2025-10-20T19:24:23.441296] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:24:23.451844] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-02. [2025-10-20T19:24:23.516736] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T19:24:23.520935] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-03. [2025-10-20T19:24:23.579969] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T19:24:23.585000] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-03. [2025-10-20T19:24:24.476729] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T19:24:24.486669] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-03. [2025-10-20T19:24:24.532253] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T19:24:24.536134] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-04. [2025-10-20T19:24:24.639847] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:24:24.645492] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-04. [2025-10-20T19:24:25.143135] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T19:24:25.153332] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-03. [2025-10-20T19:24:25.663628] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T19:24:25.693801] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T19:24:25.851383] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-05. [2025-10-20T19:24:25.978826] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:24:26.651348] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-04. [2025-10-20T19:24:26.882638] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:24:26.944215] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T19:24:27.451377] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-06. [2025-10-20T19:24:28.199487] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T19:24:28.251378] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-05. [2025-10-20T19:24:28.435593] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:24:29.051388] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-04. [2025-10-20T19:24:29.200188] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T19:24:29.851427] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-07. [2025-10-20T19:24:30.634922] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T19:24:30.651331] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-06. [2025-10-20T19:24:30.827837] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:24:31.451400] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-05. [2025-10-20T19:24:31.587123] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T19:24:32.251436] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-08. [2025-10-20T19:24:33.051391] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-07. [2025-10-20T19:24:33.151263] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:24:33.390393] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T19:24:33.851433] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-05. [2025-10-20T19:24:34.549910] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:24:34.651403] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-06. [2025-10-20T19:24:35.451346] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-08. [2025-10-20T19:24:35.659355] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T19:24:36.251398] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-06. [2025-10-20T19:24:36.466439] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T19:24:36.677598] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T19:24:37.051358] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-07. [2025-10-20T19:24:37.576594] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T19:24:37.851383] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-09. [2025-10-20T19:24:38.513071] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:24:38.651378] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-09. [2025-10-20T19:24:38.884708] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T19:24:39.451347] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-10. [2025-10-20T19:24:39.691395] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T19:24:40.251371] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-08. [2025-10-20T19:24:40.397306] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T19:24:40.400051] [INFO ] [REMOTE ] IQI: Full fetch complete (26402 records). [2025-10-20T19:24:40.400121] [DEBUG] [VALIDATOR ] IQI: Validating primary data. [2025-10-20T19:24:40.811931] [DEBUG] [VALIDATOR ] IQI: All expected trading days are present. [2025-10-20T19:24:40.812024] [DEBUG] [VALIDATOR ] IQI: Primary data validation passed. [2025-10-20T19:24:40.812552] [INFO ] [MAIN ] IQI: Fetched primary data is valid. [2025-10-20T19:24:40.812591] [DEBUG] [LOCAL ] IQI: Saving primary data. [2025-10-20T19:24:40.813229] [DEBUG] [LOCAL ] IQI: Trimmed 26402 records to 17832 (keeping 200 days of data) [2025-10-20T19:24:40.917214] [DEBUG] [LOCAL ] IQI: Primary data saved (17832 records). [2025-10-20T19:24:40.917342] [INFO ] [MAIN ] IQI: Full history saved. [2025-10-20T19:24:40.917444] [INFO ] [MAIN ] Update Progress: 1121/2367 symbols completed. IQI took 22.4s. Estimated remaining: 2h 40m 44s. Rate: 7.8 symbols/min [2025-10-20T19:24:41.057062] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-10. [2025-10-20T19:24:41.089625] [INFO ] [MAIN ] IRM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:41.134021] [DEBUG] [LOCAL ] IRM: Loaded 49820 existing records from primary storage. [2025-10-20T19:24:41.134348] [INFO ] [MAIN ] IRM: Loaded 49820 existing records. [2025-10-20T19:24:41.312386] [WARN ] [MAIN ] IRM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:41.312629] [INFO ] [MAIN ] IRM: Wiping existing primary data before full refresh. [2025-10-20T19:24:41.312728] [INFO ] [LOCAL ] IRM: Starting deletion of primary data. [2025-10-20T19:24:41.313217] [DEBUG] [LOCAL ] IRM: Deleted primary data file. [2025-10-20T19:24:41.313278] [INFO ] [MAIN ] IRM: Starting full primary refresh. [2025-10-20T19:24:41.672457] [INFO ] [REMOTE ] IRM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:41.851378] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-01. [2025-10-20T19:24:41.909718] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:24:42.288425] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:24:42.297335] [INFO ] [REMOTE ] IPI: Full fetch complete (30170 records). [2025-10-20T19:24:42.297519] [DEBUG] [VALIDATOR ] IPI: Validating primary data. [2025-10-20T19:24:42.659749] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-09. [2025-10-20T19:24:42.674170] [DEBUG] [VALIDATOR ] IPI: All expected trading days are present. [2025-10-20T19:24:42.674262] [DEBUG] [VALIDATOR ] IPI: Primary data validation passed. [2025-10-20T19:24:42.674920] [INFO ] [MAIN ] IPI: Fetched primary data is valid. [2025-10-20T19:24:42.675101] [DEBUG] [LOCAL ] IPI: Saving primary data. [2025-10-20T19:24:42.676065] [DEBUG] [LOCAL ] IPI: Trimmed 30170 records to 21513 (keeping 200 days of data) [2025-10-20T19:24:42.799445] [DEBUG] [LOCAL ] IPI: Primary data saved (21513 records). [2025-10-20T19:24:42.799574] [INFO ] [MAIN ] IPI: Full history saved. [2025-10-20T19:24:42.799669] [INFO ] [MAIN ] Update Progress: 1122/2367 symbols completed. IPI took 24.3s. Estimated remaining: 2h 40m 30s. Rate: 7.8 symbols/min [2025-10-20T19:24:42.964736] [INFO ] [MAIN ] IRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:42.987578] [DEBUG] [LOCAL ] IRS: Loaded 23427 existing records from primary storage. [2025-10-20T19:24:42.987851] [INFO ] [MAIN ] IRS: Loaded 23427 existing records. [2025-10-20T19:24:43.166584] [WARN ] [MAIN ] IRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:43.166833] [INFO ] [MAIN ] IRS: Wiping existing primary data before full refresh. [2025-10-20T19:24:43.166882] [INFO ] [LOCAL ] IRS: Starting deletion of primary data. [2025-10-20T19:24:43.167073] [DEBUG] [LOCAL ] IRS: Deleted primary data file. [2025-10-20T19:24:43.167129] [INFO ] [MAIN ] IRS: Starting full primary refresh. [2025-10-20T19:24:43.452907] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-07. [2025-10-20T19:24:43.588320] [INFO ] [REMOTE ] IRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:43.627164] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:24:44.251467] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-02. [2025-10-20T19:24:44.415465] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T19:24:45.051375] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-01. [2025-10-20T19:24:45.265524] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T19:24:45.851358] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-08. [2025-10-20T19:24:45.967342] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:24:46.222598] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T19:24:46.651392] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-02. [2025-10-20T19:24:47.451441] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-10. [2025-10-20T19:24:47.653350] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T19:24:47.677992] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:24:47.899610] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-10-20T19:24:47.906907] [INFO ] [REMOTE ] IQV: Full fetch complete (72475 records). [2025-10-20T19:24:47.907048] [DEBUG] [VALIDATOR ] IQV: Validating primary data. [2025-10-20T19:24:48.254557] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-03. [2025-10-20T19:24:48.293371] [DEBUG] [VALIDATOR ] IQV: All expected trading days are present. [2025-10-20T19:24:48.293467] [DEBUG] [VALIDATOR ] IQV: Primary data validation passed. [2025-10-20T19:24:48.294502] [INFO ] [MAIN ] IQV: Fetched primary data is valid. [2025-10-20T19:24:48.294683] [DEBUG] [LOCAL ] IQV: Saving primary data. [2025-10-20T19:24:48.296190] [DEBUG] [LOCAL ] IQV: Trimmed 72475 records to 50022 (keeping 200 days of data) [2025-10-20T19:24:48.590515] [DEBUG] [LOCAL ] IQV: Primary data saved (50022 records). [2025-10-20T19:24:48.590671] [INFO ] [MAIN ] IQV: Full history saved. [2025-10-20T19:24:48.590770] [INFO ] [MAIN ] Update Progress: 1123/2367 symbols completed. IQV took 30.0s. Estimated remaining: 2h 40m 20s. Rate: 7.8 symbols/min [2025-10-20T19:24:48.758865] [INFO ] [MAIN ] IRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:48.807120] [DEBUG] [LOCAL ] IRT: Loaded 51273 existing records from primary storage. [2025-10-20T19:24:48.807454] [INFO ] [MAIN ] IRT: Loaded 51273 existing records. [2025-10-20T19:24:48.979716] [WARN ] [MAIN ] IRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:48.979911] [INFO ] [MAIN ] IRT: Wiping existing primary data before full refresh. [2025-10-20T19:24:48.979959] [INFO ] [LOCAL ] IRT: Starting deletion of primary data. [2025-10-20T19:24:48.980163] [DEBUG] [LOCAL ] IRT: Deleted primary data file. [2025-10-20T19:24:48.980219] [INFO ] [MAIN ] IRT: Starting full primary refresh. [2025-10-20T19:24:49.056508] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-03. [2025-10-20T19:24:49.348355] [INFO ] [REMOTE ] IRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:49.361264] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T19:24:49.851415] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-01. [2025-10-20T19:24:50.651395] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-09. [2025-10-20T19:24:50.710773] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:24:51.065120] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T19:24:51.451390] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-04. [2025-10-20T19:24:52.116401] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:24:52.251365] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-10. [2025-10-20T19:24:52.484084] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T19:24:53.051404] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-05. [2025-10-20T19:24:53.851419] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-04. [2025-10-20T19:24:54.124421] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T19:24:54.139638] [INFO ] [REMOTE ] IR: Full fetch complete (76488 records). [2025-10-20T19:24:54.145515] [DEBUG] [VALIDATOR ] IR: Validating primary data. [2025-10-20T19:24:54.146584] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T19:24:54.611498] [DEBUG] [VALIDATOR ] IR: All expected trading days are present. [2025-10-20T19:24:54.611574] [DEBUG] [VALIDATOR ] IR: Primary data validation passed. [2025-10-20T19:24:54.612522] [INFO ] [MAIN ] IR: Fetched primary data is valid. [2025-10-20T19:24:54.612598] [DEBUG] [LOCAL ] IR: Saving primary data. [2025-10-20T19:24:54.614002] [DEBUG] [LOCAL ] IR: Trimmed 76488 records to 52765 (keeping 200 days of data) [2025-10-20T19:24:54.668082] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-06. [2025-10-20T19:24:54.934924] [DEBUG] [LOCAL ] IR: Primary data saved (52765 records). [2025-10-20T19:24:54.935055] [INFO ] [MAIN ] IR: Full history saved. [2025-10-20T19:24:54.935130] [INFO ] [MAIN ] Update Progress: 1124/2367 symbols completed. IR took 36.4s. Estimated remaining: 2h 40m 11s. Rate: 7.8 symbols/min [2025-10-20T19:24:55.101196] [INFO ] [MAIN ] ISD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:55.117491] [DEBUG] [LOCAL ] ISD: Loaded 16121 existing records from primary storage. [2025-10-20T19:24:55.117777] [INFO ] [MAIN ] ISD: Loaded 16121 existing records. [2025-10-20T19:24:55.297981] [WARN ] [MAIN ] ISD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:55.298196] [INFO ] [MAIN ] ISD: Wiping existing primary data before full refresh. [2025-10-20T19:24:55.298244] [INFO ] [LOCAL ] ISD: Starting deletion of primary data. [2025-10-20T19:24:55.298561] [DEBUG] [LOCAL ] ISD: Deleted primary data file. [2025-10-20T19:24:55.298628] [INFO ] [MAIN ] ISD: Starting full primary refresh. [2025-10-20T19:24:55.451979] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-02. [2025-10-20T19:24:55.672753] [INFO ] [REMOTE ] ISD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:55.707568] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:24:55.787464] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:24:56.251412] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-01. [2025-10-20T19:24:56.988252] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T19:24:57.051353] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-03. [2025-10-20T19:24:57.301957] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T19:24:57.851386] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-07. [2025-10-20T19:24:58.655949] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-02. [2025-10-20T19:24:58.661565] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T19:24:58.809138] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T19:24:59.451408] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-04. [2025-10-20T19:24:59.580737] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T19:25:00.251426] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-05. [2025-10-20T19:25:01.051395] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-03. [2025-10-20T19:25:01.592954] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T19:25:01.851361] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-05. [2025-10-20T19:25:01.976202] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-10-20T19:25:02.187282] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T19:25:02.651374] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-08. [2025-10-20T19:25:03.451342] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-04. [2025-10-20T19:25:03.723548] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T19:25:03.782099] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T19:25:04.251344] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-09. [2025-10-20T19:25:04.638452] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T19:25:05.051386] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-06. [2025-10-20T19:25:05.706536] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:25:05.851437] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-10. [2025-10-20T19:25:06.651383] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-05. [2025-10-20T19:25:06.871429] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T19:25:07.451417] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-06. [2025-10-20T19:25:07.591992] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:25:07.596395] [INFO ] [REMOTE ] IRS: Full fetch complete (32708 records). [2025-10-20T19:25:07.596483] [DEBUG] [VALIDATOR ] IRS: Validating primary data. [2025-10-20T19:25:07.728961] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:25:07.987604] [DEBUG] [VALIDATOR ] IRS: All expected trading days are present. [2025-10-20T19:25:07.987716] [DEBUG] [VALIDATOR ] IRS: Primary data validation passed. [2025-10-20T19:25:07.988342] [INFO ] [MAIN ] IRS: Fetched primary data is valid. [2025-10-20T19:25:07.988383] [DEBUG] [LOCAL ] IRS: Saving primary data. [2025-10-20T19:25:07.989147] [DEBUG] [LOCAL ] IRS: Trimmed 32708 records to 23243 (keeping 200 days of data) [2025-10-20T19:25:08.120971] [DEBUG] [LOCAL ] IRS: Primary data saved (23243 records). [2025-10-20T19:25:08.121095] [INFO ] [MAIN ] IRS: Full history saved. [2025-10-20T19:25:08.121169] [INFO ] [MAIN ] Update Progress: 1125/2367 symbols completed. IRS took 25.3s. Estimated remaining: 2h 40m 9s. Rate: 7.8 symbols/min [2025-10-20T19:25:08.262368] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-06. [2025-10-20T19:25:08.293731] [INFO ] [MAIN ] IT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:08.334477] [DEBUG] [LOCAL ] IT: Loaded 42892 existing records from primary storage. [2025-10-20T19:25:08.334787] [INFO ] [MAIN ] IT: Loaded 42892 existing records. [2025-10-20T19:25:08.510701] [WARN ] [MAIN ] IT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:08.510967] [INFO ] [MAIN ] IT: Wiping existing primary data before full refresh. [2025-10-20T19:25:08.511015] [INFO ] [LOCAL ] IT: Starting deletion of primary data. [2025-10-20T19:25:08.511199] [DEBUG] [LOCAL ] IT: Deleted primary data file. [2025-10-20T19:25:08.511254] [INFO ] [MAIN ] IT: Starting full primary refresh. [2025-10-20T19:25:08.870869] [INFO ] [REMOTE ] IT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:09.051368] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-01. [2025-10-20T19:25:09.215042] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:25:09.259059] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T19:25:09.851387] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-07. [2025-10-20T19:25:10.544537] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:25:10.651349] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-02. [2025-10-20T19:25:10.773214] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T19:25:11.451357] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-07. [2025-10-20T19:25:11.824863] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T19:25:12.251396] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-08. [2025-10-20T19:25:13.051379] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-03. [2025-10-20T19:25:13.157907] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T19:25:13.344956] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:25:13.851419] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-08. [2025-10-20T19:25:14.586027] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T19:25:14.651334] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-04. [2025-10-20T19:25:15.451527] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-09. [2025-10-20T19:25:15.521463] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T19:25:16.164463] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:25:16.251400] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-05. [2025-10-20T19:25:16.375146] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T19:25:17.051428] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-07. [2025-10-20T19:25:17.620972] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:25:17.851380] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-10. [2025-10-20T19:25:18.651361] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-09. [2025-10-20T19:25:18.814693] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:25:18.852497] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:25:18.856194] [INFO ] [REMOTE ] ISD: Full fetch complete (24181 records). [2025-10-20T19:25:18.856332] [DEBUG] [VALIDATOR ] ISD: Validating primary data. [2025-10-20T19:25:19.232593] [DEBUG] [VALIDATOR ] ISD: All expected trading days are present. [2025-10-20T19:25:19.232692] [DEBUG] [VALIDATOR ] ISD: Primary data validation passed. [2025-10-20T19:25:19.233198] [INFO ] [MAIN ] ISD: Fetched primary data is valid. [2025-10-20T19:25:19.233238] [DEBUG] [LOCAL ] ISD: Saving primary data. [2025-10-20T19:25:19.233878] [DEBUG] [LOCAL ] ISD: Trimmed 24181 records to 15782 (keeping 200 days of data) [2025-10-20T19:25:19.329408] [DEBUG] [LOCAL ] ISD: Primary data saved (15782 records). [2025-10-20T19:25:19.329510] [INFO ] [MAIN ] ISD: Full history saved. [2025-10-20T19:25:19.329590] [INFO ] [MAIN ] Update Progress: 1126/2367 symbols completed. ISD took 24.4s. Estimated remaining: 2h 40m 5s. Rate: 7.8 symbols/min [2025-10-20T19:25:19.456440] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-08. [2025-10-20T19:25:19.503842] [INFO ] [MAIN ] ITGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:19.536485] [DEBUG] [LOCAL ] ITGR: Loaded 31722 existing records from primary storage. [2025-10-20T19:25:19.536772] [INFO ] [MAIN ] ITGR: Loaded 31722 existing records. [2025-10-20T19:25:19.715986] [WARN ] [MAIN ] ITGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:19.716246] [INFO ] [MAIN ] ITGR: Wiping existing primary data before full refresh. [2025-10-20T19:25:19.716301] [INFO ] [LOCAL ] ITGR: Starting deletion of primary data. [2025-10-20T19:25:19.716487] [DEBUG] [LOCAL ] ITGR: Deleted primary data file. [2025-10-20T19:25:19.716526] [INFO ] [MAIN ] ITGR: Starting full primary refresh. [2025-10-20T19:25:20.076324] [INFO ] [REMOTE ] ITGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:20.251411] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-06. [2025-10-20T19:25:20.362145] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:25:21.051343] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-10. [2025-10-20T19:25:21.164471] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T19:25:21.668845] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T19:25:21.851407] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-07. [2025-10-20T19:25:22.573925] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T19:25:22.581399] [INFO ] [REMOTE ] IRT: Full fetch complete (72912 records). [2025-10-20T19:25:22.581547] [DEBUG] [VALIDATOR ] IRT: Validating primary data. [2025-10-20T19:25:22.656568] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-01. [2025-10-20T19:25:22.970970] [DEBUG] [VALIDATOR ] IRT: All expected trading days are present. [2025-10-20T19:25:22.971046] [DEBUG] [VALIDATOR ] IRT: Primary data validation passed. [2025-10-20T19:25:22.971996] [INFO ] [MAIN ] IRT: Fetched primary data is valid. [2025-10-20T19:25:22.972056] [DEBUG] [LOCAL ] IRT: Saving primary data. [2025-10-20T19:25:22.973383] [DEBUG] [LOCAL ] IRT: Trimmed 72912 records to 50527 (keeping 200 days of data) [2025-10-20T19:25:23.273031] [DEBUG] [LOCAL ] IRT: Primary data saved (50527 records). [2025-10-20T19:25:23.273153] [INFO ] [MAIN ] IRT: Full history saved. [2025-10-20T19:25:23.273229] [INFO ] [MAIN ] Update Progress: 1127/2367 symbols completed. IRT took 34.7s. Estimated remaining: 2h 39m 53s. Rate: 7.8 symbols/min [2025-10-20T19:25:23.461365] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-09. [2025-10-20T19:25:23.464606] [INFO ] [MAIN ] ITT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:23.491974] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T19:25:23.530447] [DEBUG] [LOCAL ] ITT: Loaded 33177 existing records from primary storage. [2025-10-20T19:25:23.530729] [INFO ] [MAIN ] ITT: Loaded 33177 existing records. [2025-10-20T19:25:23.703555] [WARN ] [MAIN ] ITT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:23.703747] [INFO ] [MAIN ] ITT: Wiping existing primary data before full refresh. [2025-10-20T19:25:23.703795] [INFO ] [LOCAL ] ITT: Starting deletion of primary data. [2025-10-20T19:25:23.703981] [DEBUG] [LOCAL ] ITT: Deleted primary data file. [2025-10-20T19:25:23.704035] [INFO ] [MAIN ] ITT: Starting full primary refresh. [2025-10-20T19:25:23.825403] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T19:25:24.071624] [INFO ] [REMOTE ] ITT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:24.251412] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-01. [2025-10-20T19:25:25.051367] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-08. [2025-10-20T19:25:25.187094] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:25:25.740044] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:25:25.851339] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-02. [2025-10-20T19:25:26.651470] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-10. [2025-10-20T19:25:26.774726] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:25:26.841436] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T19:25:27.451354] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-09. [2025-10-20T19:25:28.251406] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-03. [2025-10-20T19:25:28.445352] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T19:25:28.452526] [INFO ] [REMOTE ] IRM: Full fetch complete (72238 records). [2025-10-20T19:25:28.452709] [DEBUG] [VALIDATOR ] IRM: Validating primary data. [2025-10-20T19:25:28.836818] [DEBUG] [VALIDATOR ] IRM: All expected trading days are present. [2025-10-20T19:25:28.837003] [DEBUG] [VALIDATOR ] IRM: Primary data validation passed. [2025-10-20T19:25:28.837993] [INFO ] [MAIN ] IRM: Fetched primary data is valid. [2025-10-20T19:25:28.838109] [DEBUG] [LOCAL ] IRM: Saving primary data. [2025-10-20T19:25:28.839584] [DEBUG] [LOCAL ] IRM: Trimmed 72238 records to 49015 (keeping 200 days of data) [2025-10-20T19:25:29.052929] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-02. [2025-10-20T19:25:29.142139] [DEBUG] [LOCAL ] IRM: Primary data saved (49015 records). [2025-10-20T19:25:29.142277] [INFO ] [MAIN ] IRM: Full history saved. [2025-10-20T19:25:29.142356] [INFO ] [MAIN ] Update Progress: 1128/2367 symbols completed. IRM took 48.2s. Estimated remaining: 2h 39m 43s. Rate: 7.8 symbols/min [2025-10-20T19:25:29.346716] [INFO ] [MAIN ] ITUB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:29.354043] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-09 (1.90 seconds). [2025-10-20T19:25:29.415106] [DEBUG] [LOCAL ] ITUB: Loaded 54103 existing records from primary storage. [2025-10-20T19:25:29.415455] [INFO ] [MAIN ] ITUB: Loaded 54103 existing records. [2025-10-20T19:25:29.599435] [WARN ] [MAIN ] ITUB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:29.599752] [INFO ] [MAIN ] ITUB: Wiping existing primary data before full refresh. [2025-10-20T19:25:29.599813] [INFO ] [LOCAL ] ITUB: Starting deletion of primary data. [2025-10-20T19:25:29.600392] [DEBUG] [LOCAL ] ITUB: Deleted primary data file. [2025-10-20T19:25:29.600528] [INFO ] [MAIN ] ITUB: Starting full primary refresh. [2025-10-20T19:25:29.801575] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:25:29.856459] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-10. [2025-10-20T19:25:30.022543] [INFO ] [REMOTE ] ITUB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:30.233916] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T19:25:30.651429] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-03. [2025-10-20T19:25:31.451376] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-04. [2025-10-20T19:25:31.729285] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T19:25:31.736470] [INFO ] [REMOTE ] IT: Full fetch complete (57726 records). [2025-10-20T19:25:31.736613] [DEBUG] [VALIDATOR ] IT: Validating primary data. [2025-10-20T19:25:32.130948] [DEBUG] [VALIDATOR ] IT: All expected trading days are present. [2025-10-20T19:25:32.131143] [DEBUG] [VALIDATOR ] IT: Primary data validation passed. [2025-10-20T19:25:32.132031] [INFO ] [MAIN ] IT: Fetched primary data is valid. [2025-10-20T19:25:32.138604] [DEBUG] [LOCAL ] IT: Saving primary data. [2025-10-20T19:25:32.152585] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:25:32.160630] [DEBUG] [LOCAL ] IT: Trimmed 57726 records to 42199 (keeping 200 days of data) [2025-10-20T19:25:32.252439] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-04. [2025-10-20T19:25:32.409940] [DEBUG] [LOCAL ] IT: Primary data saved (42199 records). [2025-10-20T19:25:32.410066] [INFO ] [MAIN ] IT: Full history saved. [2025-10-20T19:25:32.410138] [INFO ] [MAIN ] Update Progress: 1129/2367 symbols completed. IT took 24.3s. Estimated remaining: 2h 39m 31s. Rate: 7.8 symbols/min [2025-10-20T19:25:32.580555] [INFO ] [MAIN ] ITW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:32.624296] [DEBUG] [LOCAL ] ITW: Loaded 46398 existing records from primary storage. [2025-10-20T19:25:32.624586] [INFO ] [MAIN ] ITW: Loaded 46398 existing records. [2025-10-20T19:25:32.812870] [WARN ] [MAIN ] ITW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:32.813175] [INFO ] [MAIN ] ITW: Wiping existing primary data before full refresh. [2025-10-20T19:25:32.813252] [INFO ] [LOCAL ] ITW: Starting deletion of primary data. [2025-10-20T19:25:32.813499] [DEBUG] [LOCAL ] ITW: Deleted primary data file. [2025-10-20T19:25:32.813573] [INFO ] [MAIN ] ITW: Starting full primary refresh. [2025-10-20T19:25:33.057121] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-01. [2025-10-20T19:25:33.176245] [INFO ] [REMOTE ] ITW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:33.205784] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:25:33.851873] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-01. [2025-10-20T19:25:33.851721] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T19:25:34.651364] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-05. [2025-10-20T19:25:34.862537] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T19:25:35.451382] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-02. [2025-10-20T19:25:35.501762] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T19:25:35.962048] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:25:36.251381] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-05. [2025-10-20T19:25:37.051357] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-02. [2025-10-20T19:25:37.131021] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T19:25:37.782568] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:25:37.851349] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-06. [2025-10-20T19:25:38.651400] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-03. [2025-10-20T19:25:38.711082] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:25:39.129712] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T19:25:39.451363] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-07. [2025-10-20T19:25:40.251409] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-06. [2025-10-20T19:25:40.466377] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T19:25:40.672997] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T19:25:41.051419] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-04. [2025-10-20T19:25:41.388107] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:25:41.851379] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-03. [2025-10-20T19:25:42.651403] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-07. [2025-10-20T19:25:42.743975] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T19:25:43.136343] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T19:25:43.451377] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-08. [2025-10-20T19:25:44.087716] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T19:25:44.251511] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-05. [2025-10-20T19:25:44.684277] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T19:25:45.051454] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-08. [2025-10-20T19:25:45.851363] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-09. [2025-10-20T19:25:46.131350] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T19:25:46.180751] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T19:25:46.651379] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-09. [2025-10-20T19:25:47.124714] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T19:25:47.451344] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-10. [2025-10-20T19:25:47.613778] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T19:25:48.251374] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-06. [2025-10-20T19:25:48.509716] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T19:25:48.514733] [INFO ] [REMOTE ] ITT: Full fetch complete (47495 records). [2025-10-20T19:25:48.514829] [DEBUG] [VALIDATOR ] ITT: Validating primary data. [2025-10-20T19:25:48.946789] [DEBUG] [VALIDATOR ] ITT: All expected trading days are present. [2025-10-20T19:25:48.946888] [DEBUG] [VALIDATOR ] ITT: Primary data validation passed. [2025-10-20T19:25:48.947698] [INFO ] [MAIN ] ITT: Fetched primary data is valid. [2025-10-20T19:25:48.947761] [DEBUG] [LOCAL ] ITT: Saving primary data. [2025-10-20T19:25:48.948747] [DEBUG] [LOCAL ] ITT: Trimmed 47495 records to 32539 (keeping 200 days of data) [2025-10-20T19:25:49.056482] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-04. [2025-10-20T19:25:49.164172] [DEBUG] [LOCAL ] ITT: Primary data saved (32539 records). [2025-10-20T19:25:49.164328] [INFO ] [MAIN ] ITT: Full history saved. [2025-10-20T19:25:49.164417] [INFO ] [MAIN ] Update Progress: 1130/2367 symbols completed. ITT took 25.9s. Estimated remaining: 2h 39m 33s. Rate: 7.8 symbols/min [2025-10-20T19:25:49.353844] [INFO ] [MAIN ] IVR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:49.399819] [DEBUG] [LOCAL ] IVR: Loaded 48761 existing records from primary storage. [2025-10-20T19:25:49.400146] [INFO ] [MAIN ] IVR: Loaded 48761 existing records. [2025-10-20T19:25:49.582251] [WARN ] [MAIN ] IVR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:49.582446] [INFO ] [MAIN ] IVR: Wiping existing primary data before full refresh. [2025-10-20T19:25:49.582494] [INFO ] [LOCAL ] IVR: Starting deletion of primary data. [2025-10-20T19:25:49.582697] [DEBUG] [LOCAL ] IVR: Deleted primary data file. [2025-10-20T19:25:49.582754] [INFO ] [MAIN ] IVR: Starting full primary refresh. [2025-10-20T19:25:49.855843] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-10. [2025-10-20T19:25:49.938050] [INFO ] [REMOTE ] IVR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:49.990050] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T19:25:50.651340] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-07. [2025-10-20T19:25:50.845360] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T19:25:51.240386] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T19:25:51.244990] [INFO ] [REMOTE ] ITGR: Full fetch complete (42955 records). [2025-10-20T19:25:51.245098] [DEBUG] [VALIDATOR ] ITGR: Validating primary data. [2025-10-20T19:25:51.459373] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-05. [2025-10-20T19:25:51.615874] [DEBUG] [VALIDATOR ] ITGR: All expected trading days are present. [2025-10-20T19:25:51.615981] [DEBUG] [VALIDATOR ] ITGR: Primary data validation passed. [2025-10-20T19:25:51.616737] [INFO ] [MAIN ] ITGR: Fetched primary data is valid. [2025-10-20T19:25:51.616797] [DEBUG] [LOCAL ] ITGR: Saving primary data. [2025-10-20T19:25:51.617774] [DEBUG] [LOCAL ] ITGR: Trimmed 42955 records to 31097 (keeping 200 days of data) [2025-10-20T19:25:51.797685] [DEBUG] [LOCAL ] ITGR: Primary data saved (31097 records). [2025-10-20T19:25:51.797821] [INFO ] [MAIN ] ITGR: Full history saved. [2025-10-20T19:25:51.797910] [INFO ] [MAIN ] Update Progress: 1131/2367 symbols completed. ITGR took 32.5s. Estimated remaining: 2h 39m 19s. Rate: 7.8 symbols/min [2025-10-20T19:25:51.964341] [INFO ] [MAIN ] IVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:51.993619] [DEBUG] [LOCAL ] IVT: Loaded 31836 existing records from primary storage. [2025-10-20T19:25:51.993902] [INFO ] [MAIN ] IVT: Loaded 31836 existing records. [2025-10-20T19:25:52.165938] [WARN ] [MAIN ] IVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:52.166148] [INFO ] [MAIN ] IVT: Wiping existing primary data before full refresh. [2025-10-20T19:25:52.166192] [INFO ] [LOCAL ] IVT: Starting deletion of primary data. [2025-10-20T19:25:52.166407] [DEBUG] [LOCAL ] IVT: Deleted primary data file. [2025-10-20T19:25:52.166461] [INFO ] [MAIN ] IVT: Starting full primary refresh. [2025-10-20T19:25:52.261787] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-01. [2025-10-20T19:25:52.523234] [INFO ] [REMOTE ] IVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:52.556387] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T19:25:53.051397] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-01. [2025-10-20T19:25:53.177719] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T19:25:53.851435] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-06. [2025-10-20T19:25:54.025280] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:25:54.089294] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:25:54.651385] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-02. [2025-10-20T19:25:55.459442] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-02. [2025-10-20T19:25:55.482150] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T19:25:55.753189] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T19:25:56.251369] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-03. [2025-10-20T19:25:56.778597] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T19:25:57.051403] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-03. [2025-10-20T19:25:57.814531] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:25:57.851385] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-08. [2025-10-20T19:25:58.415046] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:25:58.651419] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-04. [2025-10-20T19:25:59.451352] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-07. [2025-10-20T19:25:59.560575] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T19:26:00.251355] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-04. [2025-10-20T19:26:00.504334] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:26:01.051367] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-05. [2025-10-20T19:26:01.599009] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-07 (2.15 seconds). [2025-10-20T19:26:01.851388] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-09. [2025-10-20T19:26:01.965084] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:26:02.651423] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-08. [2025-10-20T19:26:02.893340] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T19:26:03.173696] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T19:26:03.451381] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-10. [2025-10-20T19:26:04.251392] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-06. [2025-10-20T19:26:04.295853] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T19:26:05.051388] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-05. [2025-10-20T19:26:05.452070] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T19:26:05.460169] [INFO ] [REMOTE ] ITUB: Full fetch complete (77321 records). [2025-10-20T19:26:05.460308] [DEBUG] [VALIDATOR ] ITUB: Validating primary data. [2025-10-20T19:26:05.859207] [DEBUG] [VALIDATOR ] ITUB: All expected trading days are present. [2025-10-20T19:26:05.859456] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-09. [2025-10-20T19:26:05.859669] [DEBUG] [VALIDATOR ] ITUB: Primary data validation passed. [2025-10-20T19:26:05.861702] [INFO ] [MAIN ] ITUB: Fetched primary data is valid. [2025-10-20T19:26:05.861823] [DEBUG] [LOCAL ] ITUB: Saving primary data. [2025-10-20T19:26:05.886725] [DEBUG] [LOCAL ] ITUB: Trimmed 77321 records to 53327 (keeping 200 days of data) [2025-10-20T19:26:05.889142] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:26:06.000333] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T19:26:06.234379] [DEBUG] [LOCAL ] ITUB: Primary data saved (53327 records). [2025-10-20T19:26:06.234519] [INFO ] [MAIN ] ITUB: Full history saved. [2025-10-20T19:26:06.234597] [INFO ] [MAIN ] Update Progress: 1132/2367 symbols completed. ITUB took 37.1s. Estimated remaining: 2h 39m 19s. Rate: 7.8 symbols/min [2025-10-20T19:26:06.405909] [INFO ] [MAIN ] IVZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:06.455476] [DEBUG] [LOCAL ] IVZ: Loaded 53895 existing records from primary storage. [2025-10-20T19:26:06.455808] [INFO ] [MAIN ] IVZ: Loaded 53895 existing records. [2025-10-20T19:26:06.634395] [WARN ] [MAIN ] IVZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:06.634667] [INFO ] [MAIN ] IVZ: Wiping existing primary data before full refresh. [2025-10-20T19:26:06.634735] [INFO ] [LOCAL ] IVZ: Starting deletion of primary data. [2025-10-20T19:26:06.635106] [DEBUG] [LOCAL ] IVZ: Deleted primary data file. [2025-10-20T19:26:06.635166] [INFO ] [MAIN ] IVZ: Starting full primary refresh. [2025-10-20T19:26:06.657995] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-06. [2025-10-20T19:26:06.986895] [INFO ] [REMOTE ] IVZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:07.451337] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-07. [2025-10-20T19:26:07.636459] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:26:07.958731] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T19:26:08.251371] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-10. [2025-10-20T19:26:09.051392] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-01. [2025-10-20T19:26:09.170513] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T19:26:09.851388] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-07. [2025-10-20T19:26:09.951080] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T19:26:09.957842] [INFO ] [REMOTE ] ITW: Full fetch complete (66742 records). [2025-10-20T19:26:09.958002] [DEBUG] [VALIDATOR ] ITW: Validating primary data. [2025-10-20T19:26:10.344283] [DEBUG] [VALIDATOR ] ITW: All expected trading days are present. [2025-10-20T19:26:10.344400] [DEBUG] [VALIDATOR ] ITW: Primary data validation passed. [2025-10-20T19:26:10.345383] [INFO ] [MAIN ] ITW: Fetched primary data is valid. [2025-10-20T19:26:10.345502] [DEBUG] [LOCAL ] ITW: Saving primary data. [2025-10-20T19:26:10.347041] [DEBUG] [LOCAL ] ITW: Trimmed 66742 records to 45637 (keeping 200 days of data) [2025-10-20T19:26:10.652000] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-08. [2025-10-20T19:26:10.678118] [DEBUG] [LOCAL ] ITW: Primary data saved (45637 records). [2025-10-20T19:26:10.684199] [INFO ] [MAIN ] ITW: Full history saved. [2025-10-20T19:26:10.683987] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:26:10.685582] [INFO ] [MAIN ] Update Progress: 1133/2367 symbols completed. ITW took 38.3s. Estimated remaining: 2h 39m 8s. Rate: 7.8 symbols/min [2025-10-20T19:26:10.865347] [INFO ] [MAIN ] IX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:10.887457] [DEBUG] [LOCAL ] IX: Loaded 22572 existing records from primary storage. [2025-10-20T19:26:10.887727] [INFO ] [MAIN ] IX: Loaded 22572 existing records. [2025-10-20T19:26:11.064549] [WARN ] [MAIN ] IX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:11.064867] [INFO ] [MAIN ] IX: Wiping existing primary data before full refresh. [2025-10-20T19:26:11.064948] [INFO ] [LOCAL ] IX: Starting deletion of primary data. [2025-10-20T19:26:11.065375] [DEBUG] [LOCAL ] IX: Deleted primary data file. [2025-10-20T19:26:11.065447] [INFO ] [MAIN ] IX: Starting full primary refresh. [2025-10-20T19:26:11.380734] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T19:26:11.458998] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-02. [2025-10-20T19:26:11.463486] [INFO ] [REMOTE ] IX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:12.251404] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-08. [2025-10-20T19:26:12.349514] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T19:26:12.606097] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T19:26:13.051399] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-01. [2025-10-20T19:26:13.461082] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T19:26:13.851419] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-09. [2025-10-20T19:26:13.903068] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T19:26:14.651444] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-02. [2025-10-20T19:26:15.436950] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T19:26:15.451411] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-03. [2025-10-20T19:26:15.501007] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T19:26:16.251425] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-10. [2025-10-20T19:26:16.832855] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T19:26:17.051421] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-04. [2025-10-20T19:26:17.851347] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-03. [2025-10-20T19:26:18.182403] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-10-20T19:26:18.191072] [INFO ] [REMOTE ] IVR: Full fetch complete (69131 records). [2025-10-20T19:26:18.191317] [DEBUG] [VALIDATOR ] IVR: Validating primary data. [2025-10-20T19:26:18.569439] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:26:18.660376] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-09. [2025-10-20T19:26:18.679565] [DEBUG] [VALIDATOR ] IVR: All expected trading days are present. [2025-10-20T19:26:18.679749] [DEBUG] [VALIDATOR ] IVR: Primary data validation passed. [2025-10-20T19:26:18.681013] [INFO ] [MAIN ] IVR: Fetched primary data is valid. [2025-10-20T19:26:18.681088] [DEBUG] [LOCAL ] IVR: Saving primary data. [2025-10-20T19:26:18.682807] [DEBUG] [LOCAL ] IVR: Trimmed 69131 records to 47946 (keeping 200 days of data) [2025-10-20T19:26:18.982121] [DEBUG] [LOCAL ] IVR: Primary data saved (47946 records). [2025-10-20T19:26:18.982261] [INFO ] [MAIN ] IVR: Full history saved. [2025-10-20T19:26:18.982343] [INFO ] [MAIN ] Update Progress: 1134/2367 symbols completed. IVR took 29.8s. Estimated remaining: 2h 39m 1s. Rate: 7.8 symbols/min [2025-10-20T19:26:19.177718] [INFO ] [MAIN ] J: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:19.221982] [DEBUG] [LOCAL ] J: Loaded 41066 existing records from primary storage. [2025-10-20T19:26:19.222334] [INFO ] [MAIN ] J: Loaded 41066 existing records. [2025-10-20T19:26:19.406479] [WARN ] [MAIN ] J: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:19.406829] [INFO ] [MAIN ] J: Wiping existing primary data before full refresh. [2025-10-20T19:26:19.406937] [INFO ] [LOCAL ] J: Starting deletion of primary data. [2025-10-20T19:26:19.407420] [DEBUG] [LOCAL ] J: Deleted primary data file. [2025-10-20T19:26:19.407549] [INFO ] [MAIN ] J: Starting full primary refresh. [2025-10-20T19:26:19.461575] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-05. [2025-10-20T19:26:19.589044] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T19:26:19.833383] [INFO ] [REMOTE ] J: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:20.023238] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:26:20.251409] [INFO ] [REMOTE ] J: Fetching primary data for 2025-01. [2025-10-20T19:26:20.705578] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:26:21.051422] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-06. [2025-10-20T19:26:21.408968] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:26:21.800971] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T19:26:21.851422] [INFO ] [REMOTE ] J: Fetching primary data for 2025-02. [2025-10-20T19:26:22.651384] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-07. [2025-10-20T19:26:23.415511] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T19:26:23.451346] [INFO ] [REMOTE ] J: Fetching primary data for 2025-03. [2025-10-20T19:26:23.470727] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T19:26:24.251473] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-10. [2025-10-20T19:26:25.051574] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-04. [2025-10-20T19:26:25.055341] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:26:25.691135] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T19:26:25.696753] [INFO ] [REMOTE ] IVT: Full fetch complete (45873 records). [2025-10-20T19:26:25.696852] [DEBUG] [VALIDATOR ] IVT: Validating primary data. [2025-10-20T19:26:25.859579] [INFO ] [REMOTE ] J: Fetching primary data for 2025-04. [2025-10-20T19:26:26.085691] [DEBUG] [VALIDATOR ] IVT: All expected trading days are present. [2025-10-20T19:26:26.085772] [DEBUG] [VALIDATOR ] IVT: Primary data validation passed. [2025-10-20T19:26:26.086491] [INFO ] [MAIN ] IVT: Fetched primary data is valid. [2025-10-20T19:26:26.086531] [DEBUG] [LOCAL ] IVT: Saving primary data. [2025-10-20T19:26:26.087419] [DEBUG] [LOCAL ] IVT: Trimmed 45873 records to 31523 (keeping 200 days of data) [2025-10-20T19:26:26.276483] [DEBUG] [LOCAL ] IVT: Primary data saved (31523 records). [2025-10-20T19:26:26.276632] [INFO ] [MAIN ] IVT: Full history saved. [2025-10-20T19:26:26.276760] [INFO ] [MAIN ] Update Progress: 1135/2367 symbols completed. IVT took 34.5s. Estimated remaining: 2h 38m 52s. Rate: 7.8 symbols/min [2025-10-20T19:26:26.443564] [INFO ] [MAIN ] JBGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:26.482966] [DEBUG] [LOCAL ] JBGS: Loaded 42660 existing records from primary storage. [2025-10-20T19:26:26.483242] [INFO ] [MAIN ] JBGS: Loaded 42660 existing records. [2025-10-20T19:26:26.658256] [WARN ] [MAIN ] JBGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:26.658545] [INFO ] [MAIN ] JBGS: Wiping existing primary data before full refresh. [2025-10-20T19:26:26.654349] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-08. [2025-10-20T19:26:26.658834] [INFO ] [LOCAL ] JBGS: Starting deletion of primary data. [2025-10-20T19:26:26.659105] [DEBUG] [LOCAL ] JBGS: Deleted primary data file. [2025-10-20T19:26:26.659857] [INFO ] [MAIN ] JBGS: Starting full primary refresh. [2025-10-20T19:26:27.019375] [INFO ] [REMOTE ] JBGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:27.048922] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T19:26:27.451443] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-01. [2025-10-20T19:26:27.570514] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:26:27.717975] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-10-20T19:26:28.251446] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-09. [2025-10-20T19:26:28.751788] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T19:26:29.051349] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-05. [2025-10-20T19:26:29.220134] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:26:29.851434] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-10. [2025-10-20T19:26:30.616897] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T19:26:30.646260] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T19:26:30.649914] [INFO ] [REMOTE ] IX: Full fetch complete (28472 records). [2025-10-20T19:26:30.650085] [DEBUG] [VALIDATOR ] IX: Validating primary data. [2025-10-20T19:26:30.656536] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-06. [2025-10-20T19:26:31.085409] [DEBUG] [VALIDATOR ] IX: All expected trading days are present. [2025-10-20T19:26:31.085505] [DEBUG] [VALIDATOR ] IX: Primary data validation passed. [2025-10-20T19:26:31.086179] [INFO ] [MAIN ] IX: Fetched primary data is valid. [2025-10-20T19:26:31.086245] [DEBUG] [LOCAL ] IX: Saving primary data. [2025-10-20T19:26:31.087035] [DEBUG] [LOCAL ] IX: Trimmed 28472 records to 22282 (keeping 200 days of data) [2025-10-20T19:26:31.221837] [DEBUG] [LOCAL ] IX: Primary data saved (22282 records). [2025-10-20T19:26:31.222015] [INFO ] [MAIN ] IX: Full history saved. [2025-10-20T19:26:31.222126] [INFO ] [MAIN ] Update Progress: 1136/2367 symbols completed. IX took 20.5s. Estimated remaining: 2h 38m 42s. Rate: 7.8 symbols/min [2025-10-20T19:26:31.394565] [INFO ] [MAIN ] JBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:31.437699] [DEBUG] [LOCAL ] JBI: Loaded 43558 existing records from primary storage. [2025-10-20T19:26:31.438025] [INFO ] [MAIN ] JBI: Loaded 43558 existing records. [2025-10-20T19:26:31.458395] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-02. [2025-10-20T19:26:31.624821] [WARN ] [MAIN ] JBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:31.625059] [INFO ] [MAIN ] JBI: Wiping existing primary data before full refresh. [2025-10-20T19:26:31.625105] [INFO ] [LOCAL ] JBI: Starting deletion of primary data. [2025-10-20T19:26:31.625321] [DEBUG] [LOCAL ] JBI: Deleted primary data file. [2025-10-20T19:26:31.625376] [INFO ] [MAIN ] JBI: Starting full primary refresh. [2025-10-20T19:26:31.979032] [INFO ] [REMOTE ] JBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:32.010874] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T19:26:32.251372] [INFO ] [REMOTE ] J: Fetching primary data for 2025-05. [2025-10-20T19:26:32.923080] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:26:33.051383] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-03. [2025-10-20T19:26:33.833871] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T19:26:33.851357] [INFO ] [REMOTE ] J: Fetching primary data for 2025-06. [2025-10-20T19:26:34.349579] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:26:34.651380] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-04. [2025-10-20T19:26:34.858232] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T19:26:35.451442] [INFO ] [REMOTE ] J: Fetching primary data for 2025-07. [2025-10-20T19:26:35.879692] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T19:26:36.251444] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-05. [2025-10-20T19:26:36.521119] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T19:26:37.051377] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-01. [2025-10-20T19:26:37.800196] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:26:37.851350] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-06. [2025-10-20T19:26:38.302112] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T19:26:38.651493] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-07. [2025-10-20T19:26:39.308245] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:26:39.451412] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-07. [2025-10-20T19:26:40.251389] [INFO ] [REMOTE ] J: Fetching primary data for 2025-08. [2025-10-20T19:26:40.406222] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T19:26:40.915279] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:26:41.051406] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-08. [2025-10-20T19:26:41.639091] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T19:26:41.851366] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-02. [2025-10-20T19:26:42.651418] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-08. [2025-10-20T19:26:42.801582] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T19:26:43.144705] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:26:43.451370] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-03. [2025-10-20T19:26:43.997527] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:26:44.251463] [INFO ] [REMOTE ] J: Fetching primary data for 2025-09. [2025-10-20T19:26:44.949414] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:26:45.051370] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-04. [2025-10-20T19:26:45.634934] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T19:26:45.851396] [INFO ] [REMOTE ] J: Fetching primary data for 2025-10. [2025-10-20T19:26:46.519305] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:26:46.651338] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-05. [2025-10-20T19:26:47.384514] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T19:26:47.391243] [INFO ] [REMOTE ] J: Full fetch complete (59687 records). [2025-10-20T19:26:47.391385] [DEBUG] [VALIDATOR ] J: Validating primary data. [2025-10-20T19:26:47.456905] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-09. [2025-10-20T19:26:47.793890] [DEBUG] [VALIDATOR ] J: All expected trading days are present. [2025-10-20T19:26:47.793991] [DEBUG] [VALIDATOR ] J: Primary data validation passed. [2025-10-20T19:26:47.794838] [INFO ] [MAIN ] J: Fetched primary data is valid. [2025-10-20T19:26:47.794989] [DEBUG] [LOCAL ] J: Saving primary data. [2025-10-20T19:26:47.821961] [DEBUG] [LOCAL ] J: Trimmed 59687 records to 40502 (keeping 200 days of data) [2025-10-20T19:26:47.823832] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T19:26:48.081850] [DEBUG] [LOCAL ] J: Primary data saved (40502 records). [2025-10-20T19:26:48.081990] [INFO ] [MAIN ] J: Full history saved. [2025-10-20T19:26:48.082078] [INFO ] [MAIN ] Update Progress: 1137/2367 symbols completed. J took 29.1s. Estimated remaining: 2h 38m 44s. Rate: 7.7 symbols/min [2025-10-20T19:26:48.249747] [INFO ] [MAIN ] JBL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:48.251490] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-09. [2025-10-20T19:26:48.294784] [DEBUG] [LOCAL ] JBL: Loaded 49045 existing records from primary storage. [2025-10-20T19:26:48.295078] [INFO ] [MAIN ] JBL: Loaded 49045 existing records. [2025-10-20T19:26:48.475139] [WARN ] [MAIN ] JBL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:48.475339] [INFO ] [MAIN ] JBL: Wiping existing primary data before full refresh. [2025-10-20T19:26:48.475387] [INFO ] [LOCAL ] JBL: Starting deletion of primary data. [2025-10-20T19:26:48.476437] [DEBUG] [LOCAL ] JBL: Deleted primary data file. [2025-10-20T19:26:48.476497] [INFO ] [MAIN ] JBL: Starting full primary refresh. [2025-10-20T19:26:48.852082] [INFO ] [REMOTE ] JBL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:48.876548] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:26:49.051381] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-01. [2025-10-20T19:26:49.651778] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T19:26:49.851377] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-10. [2025-10-20T19:26:50.651378] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-10. [2025-10-20T19:26:50.837028] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T19:26:51.451439] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-06. [2025-10-20T19:26:51.522435] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:26:51.528034] [INFO ] [REMOTE ] JBGS: Full fetch complete (59219 records). [2025-10-20T19:26:51.528224] [DEBUG] [VALIDATOR ] JBGS: Validating primary data. [2025-10-20T19:26:51.920229] [DEBUG] [VALIDATOR ] JBGS: All expected trading days are present. [2025-10-20T19:26:51.920312] [DEBUG] [VALIDATOR ] JBGS: Primary data validation passed. [2025-10-20T19:26:51.921149] [INFO ] [MAIN ] JBGS: Fetched primary data is valid. [2025-10-20T19:26:51.921211] [DEBUG] [LOCAL ] JBGS: Saving primary data. [2025-10-20T19:26:51.922342] [DEBUG] [LOCAL ] JBGS: Trimmed 59219 records to 41953 (keeping 200 days of data) [2025-10-20T19:26:52.170050] [DEBUG] [LOCAL ] JBGS: Primary data saved (41953 records). [2025-10-20T19:26:52.170175] [INFO ] [MAIN ] JBGS: Full history saved. [2025-10-20T19:26:52.170248] [INFO ] [MAIN ] Update Progress: 1138/2367 symbols completed. JBGS took 25.9s. Estimated remaining: 2h 38m 32s. Rate: 7.8 symbols/min [2025-10-20T19:26:52.258883] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-02. [2025-10-20T19:26:52.348975] [INFO ] [MAIN ] JBND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:52.380757] [DEBUG] [LOCAL ] JBND: Loaded 27442 existing records from primary storage. [2025-10-20T19:26:52.381112] [INFO ] [MAIN ] JBND: Loaded 27442 existing records. [2025-10-20T19:26:52.589285] [WARN ] [MAIN ] JBND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:52.591731] [INFO ] [MAIN ] JBND: Wiping existing primary data before full refresh. [2025-10-20T19:26:52.591467] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:26:52.599392] [INFO ] [LOCAL ] JBND: Starting deletion of primary data. [2025-10-20T19:26:52.600089] [DEBUG] [LOCAL ] JBND: Deleted primary data file. [2025-10-20T19:26:52.600154] [INFO ] [MAIN ] JBND: Starting full primary refresh. [2025-10-20T19:26:52.636827] [INFO ] [REMOTE ] IVZ: Full fetch complete (77029 records). [2025-10-20T19:26:52.669534] [DEBUG] [VALIDATOR ] IVZ: Validating primary data. [2025-10-20T19:26:52.875446] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T19:26:53.079309] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-07. [2025-10-20T19:26:53.315580] [INFO ] [REMOTE ] JBND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:53.407920] [DEBUG] [VALIDATOR ] IVZ: All expected trading days are present. [2025-10-20T19:26:53.408075] [DEBUG] [VALIDATOR ] IVZ: Primary data validation passed. [2025-10-20T19:26:53.409073] [INFO ] [MAIN ] IVZ: Fetched primary data is valid. [2025-10-20T19:26:53.409134] [DEBUG] [LOCAL ] IVZ: Saving primary data. [2025-10-20T19:26:53.410479] [DEBUG] [LOCAL ] IVZ: Trimmed 77029 records to 53129 (keeping 200 days of data) [2025-10-20T19:26:53.756459] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T19:26:53.764532] [DEBUG] [LOCAL ] IVZ: Primary data saved (53129 records). [2025-10-20T19:26:53.774018] [INFO ] [MAIN ] IVZ: Full history saved. [2025-10-20T19:26:53.774144] [INFO ] [MAIN ] Update Progress: 1139/2367 symbols completed. IVZ took 47.5s. Estimated remaining: 2h 38m 18s. Rate: 7.8 symbols/min [2025-10-20T19:26:53.856494] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-03. [2025-10-20T19:26:53.945620] [INFO ] [MAIN ] JBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:53.945786] [DEBUG] [LOCAL ] JBS: No existing data file found in primary storage. [2025-10-20T19:26:53.945827] [WARN ] [MAIN ] JBS: No existing data found, performing full refresh instead. [2025-10-20T19:26:53.945967] [INFO ] [MAIN ] JBS: Wiping existing primary data before full refresh. [2025-10-20T19:26:53.946063] [INFO ] [LOCAL ] JBS: Starting deletion of primary data. [2025-10-20T19:26:53.946128] [DEBUG] [LOCAL ] JBS: No primary data file found to delete. [2025-10-20T19:26:53.946162] [INFO ] [MAIN ] JBS: Starting full primary refresh. [2025-10-20T19:26:54.290200] [INFO ] [REMOTE ] JBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:54.651356] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-01. [2025-10-20T19:26:54.731581] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T19:26:54.847457] [DEBUG] [REMOTE ] JBS: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:26:54.847956] [WARN ] [REMOTE ] JBS: No data returned for month 2025-01 [2025-10-20T19:26:54.848059] [FATAL] [REMOTE ] JBS: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:26:54.848110] [FATAL] [MAIN ] JBS: Failed to fetch primary data. [2025-10-20T19:26:54.848146] [WARN ] [MAIN ] JBS: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:26:54.848190] [INFO ] [LOCAL ] JBS: Starting deletion of primary data. [2025-10-20T19:26:54.848253] [DEBUG] [LOCAL ] JBS: No primary data file found to delete. [2025-10-20T19:26:54.848290] [INFO ] [LOCAL ] JBS: Starting deletion of derivative data. [2025-10-20T19:26:54.848334] [DEBUG] [LOCAL ] JBS: No derivative data file found to delete. [2025-10-20T19:26:54.848386] [INFO ] [MAIN ] Update Progress: 1140/2367 symbols completed. JBS took 1.1s. Estimated remaining: 2h 38m 3s. Rate: 7.8 symbols/min [2025-10-20T19:26:55.072054] [INFO ] [MAIN ] JBTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:55.072319] [DEBUG] [LOCAL ] JBTM: No existing data file found in primary storage. [2025-10-20T19:26:55.072385] [WARN ] [MAIN ] JBTM: No existing data found, performing full refresh instead. [2025-10-20T19:26:55.072526] [INFO ] [MAIN ] JBTM: Wiping existing primary data before full refresh. [2025-10-20T19:26:55.072582] [INFO ] [LOCAL ] JBTM: Starting deletion of primary data. [2025-10-20T19:26:55.072675] [DEBUG] [LOCAL ] JBTM: No primary data file found to delete. [2025-10-20T19:26:55.072745] [INFO ] [MAIN ] JBTM: Starting full primary refresh. [2025-10-20T19:26:55.430630] [INFO ] [REMOTE ] JBTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:55.451334] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-01. [2025-10-20T19:26:55.802266] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T19:26:56.251384] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-01. [2025-10-20T19:26:56.775030] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T19:26:57.051399] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-02. [2025-10-20T19:26:57.194866] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T19:26:57.851401] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-04. [2025-10-20T19:26:58.156963] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:26:58.651382] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-03. [2025-10-20T19:26:59.451383] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-02. [2025-10-20T19:26:59.508669] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T19:26:59.871895] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T19:27:00.251427] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-08. [2025-10-20T19:27:00.383911] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T19:27:01.051385] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-05. [2025-10-20T19:27:01.851416] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-03. [2025-10-20T19:27:02.080477] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T19:27:02.651384] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-04. [2025-10-20T19:27:02.814952] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T19:27:03.059397] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T19:27:03.451369] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-04. [2025-10-20T19:27:04.172209] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:27:04.251370] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-06. [2025-10-20T19:27:04.526173] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T19:27:05.051370] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-05. [2025-10-20T19:27:05.858215] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-09. [2025-10-20T19:27:05.898114] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:27:06.052848] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:27:06.651374] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-05. [2025-10-20T19:27:07.092814] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T19:27:07.451396] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-07. [2025-10-20T19:27:07.759982] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:27:08.251368] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-06. [2025-10-20T19:27:09.051383] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-10. [2025-10-20T19:27:09.329436] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T19:27:09.394557] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:27:09.851384] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-07. [2025-10-20T19:27:10.455264] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:27:10.461207] [INFO ] [REMOTE ] JBI: Full fetch complete (63764 records). [2025-10-20T19:27:10.461339] [DEBUG] [VALIDATOR ] JBI: Validating primary data. [2025-10-20T19:27:10.661362] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-08. [2025-10-20T19:27:10.870261] [DEBUG] [VALIDATOR ] JBI: All expected trading days are present. [2025-10-20T19:27:10.870349] [DEBUG] [VALIDATOR ] JBI: Primary data validation passed. [2025-10-20T19:27:10.871598] [INFO ] [MAIN ] JBI: Fetched primary data is valid. [2025-10-20T19:27:10.871708] [DEBUG] [LOCAL ] JBI: Saving primary data. [2025-10-20T19:27:10.873079] [DEBUG] [LOCAL ] JBI: Trimmed 63764 records to 42783 (keeping 200 days of data) [2025-10-20T19:27:11.123532] [DEBUG] [LOCAL ] JBI: Primary data saved (42783 records). [2025-10-20T19:27:11.123770] [INFO ] [MAIN ] JBI: Full history saved. [2025-10-20T19:27:11.124807] [INFO ] [MAIN ] Update Progress: 1141/2367 symbols completed. JBI took 39.9s. Estimated remaining: 2h 38m 4s. Rate: 7.8 symbols/min [2025-10-20T19:27:11.314251] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:27:11.316486] [INFO ] [MAIN ] JCE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:11.332391] [DEBUG] [LOCAL ] JCE: Loaded 7507 existing records from primary storage. [2025-10-20T19:27:11.332603] [INFO ] [MAIN ] JCE: Loaded 7507 existing records. [2025-10-20T19:27:11.456451] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-06. [2025-10-20T19:27:11.509063] [WARN ] [MAIN ] JCE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:11.509259] [INFO ] [MAIN ] JCE: Wiping existing primary data before full refresh. [2025-10-20T19:27:11.509306] [INFO ] [LOCAL ] JCE: Starting deletion of primary data. [2025-10-20T19:27:11.509472] [DEBUG] [LOCAL ] JCE: Deleted primary data file. [2025-10-20T19:27:11.509509] [INFO ] [MAIN ] JCE: Starting full primary refresh. [2025-10-20T19:27:11.868760] [INFO ] [REMOTE ] JCE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:12.251356] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-08. [2025-10-20T19:27:12.620630] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T19:27:12.648062] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T19:27:13.051362] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-09. [2025-10-20T19:27:13.129372] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T19:27:13.851391] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-01. [2025-10-20T19:27:14.265258] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-01 (0.41 seconds). [2025-10-20T19:27:14.538591] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:27:14.651348] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-07. [2025-10-20T19:27:15.451390] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-10. [2025-10-20T19:27:15.889161] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T19:27:16.251436] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-09. [2025-10-20T19:27:17.051366] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-02. [2025-10-20T19:27:17.435238] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T19:27:17.523427] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-10-20T19:27:17.530972] [INFO ] [REMOTE ] JBL: Full fetch complete (70633 records). [2025-10-20T19:27:17.531140] [DEBUG] [VALIDATOR ] JBL: Validating primary data. [2025-10-20T19:27:17.655079] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T19:27:17.856491] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-08. [2025-10-20T19:27:17.936865] [DEBUG] [VALIDATOR ] JBL: All expected trading days are present. [2025-10-20T19:27:17.936969] [DEBUG] [VALIDATOR ] JBL: Primary data validation passed. [2025-10-20T19:27:17.937893] [INFO ] [MAIN ] JBL: Fetched primary data is valid. [2025-10-20T19:27:17.937954] [DEBUG] [LOCAL ] JBL: Saving primary data. [2025-10-20T19:27:17.939347] [DEBUG] [LOCAL ] JBL: Trimmed 70633 records to 48383 (keeping 200 days of data) [2025-10-20T19:27:18.244109] [DEBUG] [LOCAL ] JBL: Primary data saved (48383 records). [2025-10-20T19:27:18.244308] [INFO ] [MAIN ] JBL: Full history saved. [2025-10-20T19:27:18.244399] [INFO ] [MAIN ] Update Progress: 1142/2367 symbols completed. JBL took 30.2s. Estimated remaining: 2h 37m 56s. Rate: 7.8 symbols/min [2025-10-20T19:27:18.412217] [INFO ] [MAIN ] JCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:18.468102] [DEBUG] [LOCAL ] JCI: Loaded 54306 existing records from primary storage. [2025-10-20T19:27:18.468484] [INFO ] [MAIN ] JCI: Loaded 54306 existing records. [2025-10-20T19:27:18.643445] [WARN ] [MAIN ] JCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:18.643664] [INFO ] [MAIN ] JCI: Wiping existing primary data before full refresh. [2025-10-20T19:27:18.643724] [INFO ] [LOCAL ] JCI: Starting deletion of primary data. [2025-10-20T19:27:18.643936] [DEBUG] [LOCAL ] JCI: Deleted primary data file. [2025-10-20T19:27:18.644009] [INFO ] [MAIN ] JCI: Starting full primary refresh. [2025-10-20T19:27:18.651376] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-03. [2025-10-20T19:27:19.057106] [INFO ] [REMOTE ] JCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:19.250898] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T19:27:19.316936] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T19:27:19.451423] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-10. [2025-10-20T19:27:20.251380] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-04. [2025-10-20T19:27:20.940484] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:27:20.946515] [INFO ] [REMOTE ] JBTM: Full fetch complete (46931 records). [2025-10-20T19:27:20.946600] [DEBUG] [VALIDATOR ] JBTM: Validating primary data. [2025-10-20T19:27:21.036532] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:27:21.056602] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-01. [2025-10-20T19:27:21.150941] [WARN ] [VALIDATOR ] JBTM: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:27:21.151455] [FATAL] [MAIN ] JBTM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:27:21.151525] [WARN ] [MAIN ] JBTM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:27:21.151598] [INFO ] [LOCAL ] JBTM: Starting deletion of primary data. [2025-10-20T19:27:21.151687] [DEBUG] [LOCAL ] JBTM: No primary data file found to delete. [2025-10-20T19:27:21.151770] [INFO ] [LOCAL ] JBTM: Starting deletion of derivative data. [2025-10-20T19:27:21.151839] [DEBUG] [LOCAL ] JBTM: No derivative data file found to delete. [2025-10-20T19:27:21.151898] [INFO ] [MAIN ] Update Progress: 1143/2367 symbols completed. JBTM took 26.3s. Estimated remaining: 2h 37m 43s. Rate: 7.8 symbols/min [2025-10-20T19:27:21.318614] [INFO ] [MAIN ] JEF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:21.364543] [DEBUG] [LOCAL ] JEF: Loaded 50950 existing records from primary storage. [2025-10-20T19:27:21.364858] [INFO ] [MAIN ] JEF: Loaded 50950 existing records. [2025-10-20T19:27:21.539287] [WARN ] [MAIN ] JEF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:21.539461] [INFO ] [MAIN ] JEF: Wiping existing primary data before full refresh. [2025-10-20T19:27:21.539506] [INFO ] [LOCAL ] JEF: Starting deletion of primary data. [2025-10-20T19:27:21.539738] [DEBUG] [LOCAL ] JEF: Deleted primary data file. [2025-10-20T19:27:21.539796] [INFO ] [MAIN ] JEF: Starting full primary refresh. [2025-10-20T19:27:21.858513] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-09. [2025-10-20T19:27:21.886947] [INFO ] [REMOTE ] JEF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:22.651424] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-05. [2025-10-20T19:27:22.806361] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T19:27:23.245133] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T19:27:23.259824] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T19:27:23.451402] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-06. [2025-10-20T19:27:24.192943] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T19:27:24.251407] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-10. [2025-10-20T19:27:25.051461] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-07. [2025-10-20T19:27:25.586738] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T19:27:25.591986] [INFO ] [REMOTE ] JBND: Full fetch complete (35860 records). [2025-10-20T19:27:25.592068] [DEBUG] [VALIDATOR ] JBND: Validating primary data. [2025-10-20T19:27:25.856515] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-02. [2025-10-20T19:27:25.892075] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T19:27:25.987147] [DEBUG] [VALIDATOR ] JBND: All expected trading days are present. [2025-10-20T19:27:25.987234] [DEBUG] [VALIDATOR ] JBND: Primary data validation passed. [2025-10-20T19:27:25.988060] [INFO ] [MAIN ] JBND: Fetched primary data is valid. [2025-10-20T19:27:25.988127] [DEBUG] [LOCAL ] JBND: Saving primary data. [2025-10-20T19:27:25.992120] [DEBUG] [LOCAL ] JBND: Trimmed 35860 records to 27243 (keeping 200 days of data) [2025-10-20T19:27:26.160104] [DEBUG] [LOCAL ] JBND: Primary data saved (27243 records). [2025-10-20T19:27:26.160258] [INFO ] [MAIN ] JBND: Full history saved. [2025-10-20T19:27:26.160340] [INFO ] [MAIN ] Update Progress: 1144/2367 symbols completed. JBND took 34.0s. Estimated remaining: 2h 37m 32s. Rate: 7.8 symbols/min [2025-10-20T19:27:26.339343] [INFO ] [MAIN ] JELD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:26.380473] [DEBUG] [LOCAL ] JELD: Loaded 46697 existing records from primary storage. [2025-10-20T19:27:26.380793] [INFO ] [MAIN ] JELD: Loaded 46697 existing records. [2025-10-20T19:27:26.555681] [WARN ] [MAIN ] JELD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:26.555895] [INFO ] [MAIN ] JELD: Wiping existing primary data before full refresh. [2025-10-20T19:27:26.555944] [INFO ] [LOCAL ] JELD: Starting deletion of primary data. [2025-10-20T19:27:26.556116] [DEBUG] [LOCAL ] JELD: Deleted primary data file. [2025-10-20T19:27:26.556156] [INFO ] [MAIN ] JELD: Starting full primary refresh. [2025-10-20T19:27:26.656520] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-08. [2025-10-20T19:27:26.914213] [INFO ] [REMOTE ] JELD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:27.302731] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T19:27:27.451376] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-09. [2025-10-20T19:27:27.666323] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T19:27:28.230601] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:27:28.251351] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-10. [2025-10-20T19:27:29.051381] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-01. [2025-10-20T19:27:29.163751] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T19:27:29.166178] [INFO ] [REMOTE ] JCE: Full fetch complete (9739 records). [2025-10-20T19:27:29.166253] [DEBUG] [VALIDATOR ] JCE: Validating primary data. [2025-10-20T19:27:29.529580] [DEBUG] [VALIDATOR ] JCE: All expected trading days are present. [2025-10-20T19:27:29.529687] [DEBUG] [VALIDATOR ] JCE: Primary data validation passed. [2025-10-20T19:27:29.529947] [INFO ] [MAIN ] JCE: Fetched primary data is valid. [2025-10-20T19:27:29.529989] [DEBUG] [LOCAL ] JCE: Saving primary data. [2025-10-20T19:27:29.530403] [DEBUG] [LOCAL ] JCE: Trimmed 9739 records to 7308 (keeping 200 days of data) [2025-10-20T19:27:29.575005] [DEBUG] [LOCAL ] JCE: Primary data saved (7308 records). [2025-10-20T19:27:29.575154] [INFO ] [MAIN ] JCE: Full history saved. [2025-10-20T19:27:29.575236] [INFO ] [MAIN ] Update Progress: 1145/2367 symbols completed. JCE took 18.5s. Estimated remaining: 2h 37m 20s. Rate: 7.8 symbols/min [2025-10-20T19:27:29.750778] [INFO ] [MAIN ] JFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:29.782861] [DEBUG] [LOCAL ] JFR: Loaded 31519 existing records from primary storage. [2025-10-20T19:27:29.783144] [INFO ] [MAIN ] JFR: Loaded 31519 existing records. [2025-10-20T19:27:29.860471] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-03. [2025-10-20T19:27:29.966710] [WARN ] [MAIN ] JFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:29.966887] [INFO ] [MAIN ] JFR: Wiping existing primary data before full refresh. [2025-10-20T19:27:29.966934] [INFO ] [LOCAL ] JFR: Starting deletion of primary data. [2025-10-20T19:27:29.967108] [DEBUG] [LOCAL ] JFR: Deleted primary data file. [2025-10-20T19:27:29.967146] [INFO ] [MAIN ] JFR: Starting full primary refresh. [2025-10-20T19:27:30.333177] [INFO ] [REMOTE ] JFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:30.651378] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-01. [2025-10-20T19:27:30.750072] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T19:27:31.451424] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-01. [2025-10-20T19:27:31.702851] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T19:27:31.768764] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T19:27:32.251368] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-04. [2025-10-20T19:27:32.783348] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T19:27:33.051446] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-02. [2025-10-20T19:27:33.851434] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-02. [2025-10-20T19:27:34.118026] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T19:27:34.494334] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T19:27:34.651393] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-03. [2025-10-20T19:27:35.438397] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T19:27:35.451340] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-02. [2025-10-20T19:27:35.732206] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T19:27:36.251375] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-04. [2025-10-20T19:27:36.493048] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T19:27:37.051392] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-03. [2025-10-20T19:27:37.779262] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T19:27:37.851416] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-05. [2025-10-20T19:27:38.651376] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-03. [2025-10-20T19:27:38.758980] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T19:27:39.451379] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-04. [2025-10-20T19:27:39.543499] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:27:39.646164] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T19:27:40.251445] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-06. [2025-10-20T19:27:40.970889] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:27:41.051338] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-05. [2025-10-20T19:27:41.632441] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:27:41.851391] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-05. [2025-10-20T19:27:42.604424] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:27:42.651328] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-06. [2025-10-20T19:27:43.456508] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-07. [2025-10-20T19:27:43.479305] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T19:27:44.031398] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:27:44.251376] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-06. [2025-10-20T19:27:45.039862] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:27:45.051336] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-08. [2025-10-20T19:27:45.851424] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-04. [2025-10-20T19:27:45.926009] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:27:46.235013] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T19:27:46.651533] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-07. [2025-10-20T19:27:47.153512] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T19:27:47.451378] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-05. [2025-10-20T19:27:48.251350] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-09. [2025-10-20T19:27:48.348587] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:27:48.639747] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T19:27:49.051440] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-07. [2025-10-20T19:27:49.829712] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T19:27:49.851356] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-10. [2025-10-20T19:27:50.651369] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-08. [2025-10-20T19:27:50.917010] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T19:27:51.283119] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T19:27:51.290100] [INFO ] [REMOTE ] JELD: Full fetch complete (64565 records). [2025-10-20T19:27:51.290247] [DEBUG] [VALIDATOR ] JELD: Validating primary data. [2025-10-20T19:27:51.456709] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-08. [2025-10-20T19:27:51.729169] [DEBUG] [VALIDATOR ] JELD: All expected trading days are present. [2025-10-20T19:27:51.729264] [DEBUG] [VALIDATOR ] JELD: Primary data validation passed. [2025-10-20T19:27:51.730300] [INFO ] [MAIN ] JELD: Fetched primary data is valid. [2025-10-20T19:27:51.730373] [DEBUG] [LOCAL ] JELD: Saving primary data. [2025-10-20T19:27:51.731792] [DEBUG] [LOCAL ] JELD: Trimmed 64565 records to 46009 (keeping 200 days of data) [2025-10-20T19:27:51.989974] [DEBUG] [LOCAL ] JELD: Primary data saved (46009 records). [2025-10-20T19:27:51.990128] [INFO ] [MAIN ] JELD: Full history saved. [2025-10-20T19:27:51.990219] [INFO ] [MAIN ] Update Progress: 1146/2367 symbols completed. JELD took 25.8s. Estimated remaining: 2h 37m 28s. Rate: 7.8 symbols/min [2025-10-20T19:27:52.166768] [INFO ] [MAIN ] JGH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:52.198609] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T19:27:52.219573] [DEBUG] [LOCAL ] JGH: Loaded 12828 existing records from primary storage. [2025-10-20T19:27:52.219846] [INFO ] [MAIN ] JGH: Loaded 12828 existing records. [2025-10-20T19:27:52.256524] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-06. [2025-10-20T19:27:52.405175] [WARN ] [MAIN ] JGH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:52.405433] [INFO ] [MAIN ] JGH: Wiping existing primary data before full refresh. [2025-10-20T19:27:52.405481] [INFO ] [LOCAL ] JGH: Starting deletion of primary data. [2025-10-20T19:27:52.405697] [DEBUG] [LOCAL ] JGH: Deleted primary data file. [2025-10-20T19:27:52.405758] [INFO ] [MAIN ] JGH: Starting full primary refresh. [2025-10-20T19:27:52.771468] [INFO ] [REMOTE ] JGH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:53.051391] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-01. [2025-10-20T19:27:53.350446] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T19:27:53.507800] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T19:27:53.783510] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:27:53.851396] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-02. [2025-10-20T19:27:54.440495] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T19:27:54.651368] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-03. [2025-10-20T19:27:55.417713] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T19:27:55.451360] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-04. [2025-10-20T19:27:56.251417] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-09. [2025-10-20T19:27:56.274564] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T19:27:57.051395] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-07. [2025-10-20T19:27:57.851387] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-09. [2025-10-20T19:27:57.899801] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T19:27:58.466800] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:27:58.651412] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-08. [2025-10-20T19:27:59.451363] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-05. [2025-10-20T19:27:59.688053] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T19:27:59.717892] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T19:28:00.220003] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T19:28:00.251457] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-09. [2025-10-20T19:28:01.051374] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-06. [2025-10-20T19:28:01.836580] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T19:28:01.852603] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-07. [2025-10-20T19:28:02.016813] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:28:02.598569] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-10-20T19:28:02.651366] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-08. [2025-10-20T19:28:03.451353] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-10. [2025-10-20T19:28:03.481518] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T19:28:04.251370] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-09. [2025-10-20T19:28:04.908741] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:28:04.914079] [INFO ] [REMOTE ] JFR: Full fetch complete (48310 records). [2025-10-20T19:28:04.914182] [DEBUG] [VALIDATOR ] JFR: Validating primary data. [2025-10-20T19:28:05.056410] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-10. [2025-10-20T19:28:05.132946] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T19:28:05.318258] [DEBUG] [VALIDATOR ] JFR: All expected trading days are present. [2025-10-20T19:28:05.318387] [DEBUG] [VALIDATOR ] JFR: Primary data validation passed. [2025-10-20T19:28:05.319340] [INFO ] [MAIN ] JFR: Fetched primary data is valid. [2025-10-20T19:28:05.319412] [DEBUG] [LOCAL ] JFR: Saving primary data. [2025-10-20T19:28:05.320487] [DEBUG] [LOCAL ] JFR: Trimmed 48310 records to 30952 (keeping 200 days of data) [2025-10-20T19:28:05.499522] [DEBUG] [LOCAL ] JFR: Primary data saved (30952 records). [2025-10-20T19:28:05.499639] [INFO ] [MAIN ] JFR: Full history saved. [2025-10-20T19:28:05.499754] [INFO ] [MAIN ] Update Progress: 1147/2367 symbols completed. JFR took 35.9s. Estimated remaining: 2h 37m 26s. Rate: 7.7 symbols/min [2025-10-20T19:28:05.669011] [INFO ] [MAIN ] JHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:05.712019] [DEBUG] [LOCAL ] JHG: Loaded 48298 existing records from primary storage. [2025-10-20T19:28:05.712329] [INFO ] [MAIN ] JHG: Loaded 48298 existing records. [2025-10-20T19:28:05.857998] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-10. [2025-10-20T19:28:05.889053] [WARN ] [MAIN ] JHG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:05.889221] [INFO ] [MAIN ] JHG: Wiping existing primary data before full refresh. [2025-10-20T19:28:05.889267] [INFO ] [LOCAL ] JHG: Starting deletion of primary data. [2025-10-20T19:28:05.889516] [DEBUG] [LOCAL ] JHG: Deleted primary data file. [2025-10-20T19:28:05.889575] [INFO ] [MAIN ] JHG: Starting full primary refresh. [2025-10-20T19:28:06.244692] [INFO ] [REMOTE ] JHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:06.651422] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-10. [2025-10-20T19:28:06.805676] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T19:28:06.813409] [INFO ] [REMOTE ] JCI: Full fetch complete (77644 records). [2025-10-20T19:28:06.813541] [DEBUG] [VALIDATOR ] JCI: Validating primary data. [2025-10-20T19:28:07.176440] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T19:28:07.221159] [INFO ] [REMOTE ] JEF: Full fetch complete (73337 records). [2025-10-20T19:28:07.240634] [DEBUG] [VALIDATOR ] JEF: Validating primary data. [2025-10-20T19:28:07.244395] [DEBUG] [VALIDATOR ] JCI: All expected trading days are present. [2025-10-20T19:28:07.244498] [DEBUG] [VALIDATOR ] JCI: Primary data validation passed. [2025-10-20T19:28:07.245572] [INFO ] [MAIN ] JCI: Fetched primary data is valid. [2025-10-20T19:28:07.245674] [DEBUG] [LOCAL ] JCI: Saving primary data. [2025-10-20T19:28:07.357303] [DEBUG] [LOCAL ] JCI: Trimmed 77644 records to 53523 (keeping 200 days of data) [2025-10-20T19:28:07.460516] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-01. [2025-10-20T19:28:07.653127] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:28:07.681936] [INFO ] [REMOTE ] JGH: Full fetch complete (16405 records). [2025-10-20T19:28:07.690147] [DEBUG] [VALIDATOR ] JGH: Validating primary data. [2025-10-20T19:28:08.001761] [DEBUG] [VALIDATOR ] JEF: All expected trading days are present. [2025-10-20T19:28:08.018928] [DEBUG] [VALIDATOR ] JEF: Primary data validation passed. [2025-10-20T19:28:08.033176] [INFO ] [MAIN ] JEF: Fetched primary data is valid. [2025-10-20T19:28:08.048753] [DEBUG] [LOCAL ] JEF: Saving primary data. [2025-10-20T19:28:08.193342] [DEBUG] [LOCAL ] JEF: Trimmed 73337 records to 50184 (keeping 200 days of data) [2025-10-20T19:28:08.371638] [DEBUG] [LOCAL ] JCI: Primary data saved (53523 records). [2025-10-20T19:28:08.397893] [INFO ] [MAIN ] JCI: Full history saved. [2025-10-20T19:28:08.410084] [INFO ] [MAIN ] Update Progress: 1148/2367 symbols completed. JCI took 50.2s. Estimated remaining: 2h 37m 13s. Rate: 7.8 symbols/min [2025-10-20T19:28:08.717323] [DEBUG] [VALIDATOR ] JGH: All expected trading days are present. [2025-10-20T19:28:08.759178] [DEBUG] [VALIDATOR ] JGH: Primary data validation passed. [2025-10-20T19:28:08.760180] [INFO ] [MAIN ] JGH: Fetched primary data is valid. [2025-10-20T19:28:08.774709] [DEBUG] [LOCAL ] JGH: Saving primary data. [2025-10-20T19:28:08.825562] [INFO ] [MAIN ] JHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:08.839397] [DEBUG] [LOCAL ] JGH: Trimmed 16405 records to 12697 (keeping 200 days of data) [2025-10-20T19:28:08.911531] [DEBUG] [LOCAL ] JHI: Loaded 4176 existing records from primary storage. [2025-10-20T19:28:08.952043] [INFO ] [MAIN ] JHI: Loaded 4176 existing records. [2025-10-20T19:28:09.018466] [DEBUG] [LOCAL ] JGH: Primary data saved (12697 records). [2025-10-20T19:28:09.051480] [INFO ] [MAIN ] JGH: Full history saved. [2025-10-20T19:28:09.069317] [INFO ] [MAIN ] Update Progress: 1149/2367 symbols completed. JGH took 17.1s. Estimated remaining: 2h 36m 58s. Rate: 7.8 symbols/min [2025-10-20T19:28:09.111458] [DEBUG] [LOCAL ] JEF: Primary data saved (50184 records). [2025-10-20T19:28:09.132284] [INFO ] [MAIN ] JEF: Full history saved. [2025-10-20T19:28:09.193266] [INFO ] [MAIN ] Update Progress: 1150/2367 symbols completed. JEF took 48.0s. Estimated remaining: 2h 36m 42s. Rate: 7.8 symbols/min [2025-10-20T19:28:09.287555] [WARN ] [MAIN ] JHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:09.343435] [INFO ] [MAIN ] JHI: Wiping existing primary data before full refresh. [2025-10-20T19:28:09.391416] [INFO ] [LOCAL ] JHI: Starting deletion of primary data. [2025-10-20T19:28:09.455954] [DEBUG] [LOCAL ] JHI: Deleted primary data file. [2025-10-20T19:28:09.458476] [INFO ] [MAIN ] JHI: Starting full primary refresh. [2025-10-20T19:28:09.480987] [INFO ] [MAIN ] JHS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:09.677161] [DEBUG] [LOCAL ] JHS: Loaded 2952 existing records from primary storage. [2025-10-20T19:28:09.669525] [INFO ] [MAIN ] JHX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:09.689109] [INFO ] [MAIN ] JHS: Loaded 2952 existing records. [2025-10-20T19:28:09.702108] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-01 (2.24 seconds). [2025-10-20T19:28:09.760485] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-02. [2025-10-20T19:28:09.998996] [WARN ] [MAIN ] JHS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:10.038368] [INFO ] [MAIN ] JHS: Wiping existing primary data before full refresh. [2025-10-20T19:28:10.064913] [INFO ] [LOCAL ] JHS: Starting deletion of primary data. [2025-10-20T19:28:10.086071] [DEBUG] [LOCAL ] JHS: Deleted primary data file. [2025-10-20T19:28:10.089498] [INFO ] [MAIN ] JHS: Starting full primary refresh. [2025-10-20T19:28:10.280216] [INFO ] [REMOTE ] JHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:10.291110] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-01. [2025-10-20T19:28:10.294673] [DEBUG] [LOCAL ] JHX: Loaded 53939 existing records from primary storage. [2025-10-20T19:28:10.311200] [INFO ] [MAIN ] JHX: Loaded 53939 existing records. [2025-10-20T19:28:10.781173] [WARN ] [MAIN ] JHX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:10.786526] [INFO ] [MAIN ] JHX: Wiping existing primary data before full refresh. [2025-10-20T19:28:10.792658] [INFO ] [LOCAL ] JHX: Starting deletion of primary data. [2025-10-20T19:28:10.793945] [DEBUG] [LOCAL ] JHX: Deleted primary data file. [2025-10-20T19:28:10.809432] [INFO ] [MAIN ] JHX: Starting full primary refresh. [2025-10-20T19:28:10.813121] [INFO ] [REMOTE ] JHS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:10.844402] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-01. [2025-10-20T19:28:10.944909] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T19:28:10.958532] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-02. [2025-10-20T19:28:11.184120] [INFO ] [REMOTE ] JHX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:11.233039] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-10-20T19:28:11.261525] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T19:28:11.451403] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-01. [2025-10-20T19:28:11.471823] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T19:28:12.251402] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-03. [2025-10-20T19:28:12.438168] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T19:28:13.051460] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-02. [2025-10-20T19:28:13.689601] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T19:28:13.851428] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-03. [2025-10-20T19:28:13.919971] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:28:14.400329] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T19:28:14.651388] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-04. [2025-10-20T19:28:15.254839] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T19:28:15.451379] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-05. [2025-10-20T19:28:15.936382] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T19:28:16.251431] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-02. [2025-10-20T19:28:16.839976] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T19:28:17.051371] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-06. [2025-10-20T19:28:17.583352] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T19:28:17.851377] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-07. [2025-10-20T19:28:18.377144] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T19:28:18.651387] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-08. [2025-10-20T19:28:19.290638] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T19:28:19.451382] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-09. [2025-10-20T19:28:20.027559] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T19:28:20.251414] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-04. [2025-10-20T19:28:21.051430] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-10. [2025-10-20T19:28:21.789098] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T19:28:21.790848] [INFO ] [REMOTE ] JHI: Full fetch complete (5775 records). [2025-10-20T19:28:21.790945] [DEBUG] [VALIDATOR ] JHI: Validating primary data. [2025-10-20T19:28:21.856526] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-03. [2025-10-20T19:28:22.107559] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:28:22.197820] [DEBUG] [VALIDATOR ] JHI: All expected trading days are present. [2025-10-20T19:28:22.197918] [DEBUG] [VALIDATOR ] JHI: Primary data validation passed. [2025-10-20T19:28:22.198131] [INFO ] [MAIN ] JHI: Fetched primary data is valid. [2025-10-20T19:28:22.198187] [DEBUG] [LOCAL ] JHI: Saving primary data. [2025-10-20T19:28:22.198551] [DEBUG] [LOCAL ] JHI: Trimmed 5775 records to 4157 (keeping 200 days of data) [2025-10-20T19:28:22.223409] [DEBUG] [LOCAL ] JHI: Primary data saved (4157 records). [2025-10-20T19:28:22.223503] [INFO ] [MAIN ] JHI: Full history saved. [2025-10-20T19:28:22.223576] [INFO ] [MAIN ] Update Progress: 1151/2367 symbols completed. JHI took 13.8s. Estimated remaining: 2h 36m 40s. Rate: 7.8 symbols/min [2025-10-20T19:28:22.388062] [INFO ] [MAIN ] JILL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:22.400598] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T19:28:22.407491] [DEBUG] [LOCAL ] JILL: Loaded 16049 existing records from primary storage. [2025-10-20T19:28:22.407682] [INFO ] [MAIN ] JILL: Loaded 16049 existing records. [2025-10-20T19:28:22.579691] [WARN ] [MAIN ] JILL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:22.579912] [INFO ] [MAIN ] JILL: Wiping existing primary data before full refresh. [2025-10-20T19:28:22.579958] [INFO ] [LOCAL ] JILL: Starting deletion of primary data. [2025-10-20T19:28:22.580207] [DEBUG] [LOCAL ] JILL: Deleted primary data file. [2025-10-20T19:28:22.580262] [INFO ] [MAIN ] JILL: Starting full primary refresh. [2025-10-20T19:28:22.656540] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-04. [2025-10-20T19:28:22.943493] [INFO ] [REMOTE ] JILL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:23.160392] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T19:28:23.451411] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-01. [2025-10-20T19:28:24.251389] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-05. [2025-10-20T19:28:24.335129] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T19:28:24.706610] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T19:28:25.051379] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-02. [2025-10-20T19:28:25.837608] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T19:28:25.851337] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-03. [2025-10-20T19:28:26.651349] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-06. [2025-10-20T19:28:26.860892] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:28:27.089334] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T19:28:27.451378] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-04. [2025-10-20T19:28:28.251746] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-05. [2025-10-20T19:28:28.252730] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T19:28:29.051409] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-03. [2025-10-20T19:28:29.851358] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-05. [2025-10-20T19:28:30.044724] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:28:30.103291] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:28:30.651347] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-07. [2025-10-20T19:28:30.792483] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T19:28:31.225272] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T19:28:31.451389] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-08. [2025-10-20T19:28:31.981334] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T19:28:32.251376] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-04. [2025-10-20T19:28:33.051360] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-06. [2025-10-20T19:28:33.851413] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-06. [2025-10-20T19:28:33.955699] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:28:34.496709] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T19:28:34.630625] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:28:34.651343] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-07. [2025-10-20T19:28:35.451472] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-07. [2025-10-20T19:28:36.215492] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T19:28:36.251363] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-08. [2025-10-20T19:28:36.406899] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T19:28:37.051396] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-05. [2025-10-20T19:28:37.813331] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T19:28:37.851340] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-09. [2025-10-20T19:28:38.651374] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-08. [2025-10-20T19:28:38.795900] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T19:28:39.370844] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:28:39.451343] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-10. [2025-10-20T19:28:39.666432] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T19:28:40.251372] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-06. [2025-10-20T19:28:41.051355] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-09. [2025-10-20T19:28:41.117514] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:28:41.124622] [INFO ] [REMOTE ] JHG: Full fetch complete (68630 records). [2025-10-20T19:28:41.124772] [DEBUG] [VALIDATOR ] JHG: Validating primary data. [2025-10-20T19:28:41.523387] [DEBUG] [VALIDATOR ] JHG: All expected trading days are present. [2025-10-20T19:28:41.523491] [DEBUG] [VALIDATOR ] JHG: Primary data validation passed. [2025-10-20T19:28:41.524527] [INFO ] [MAIN ] JHG: Fetched primary data is valid. [2025-10-20T19:28:41.524599] [DEBUG] [LOCAL ] JHG: Saving primary data. [2025-10-20T19:28:41.526108] [DEBUG] [LOCAL ] JHG: Trimmed 68630 records to 47569 (keeping 200 days of data) [2025-10-20T19:28:41.644322] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:28:41.827517] [DEBUG] [LOCAL ] JHG: Primary data saved (47569 records). [2025-10-20T19:28:41.827712] [INFO ] [MAIN ] JHG: Full history saved. [2025-10-20T19:28:41.828967] [INFO ] [MAIN ] Update Progress: 1152/2367 symbols completed. JHG took 36.3s. Estimated remaining: 2h 36m 45s. Rate: 7.8 symbols/min [2025-10-20T19:28:41.861175] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-10. [2025-10-20T19:28:42.004278] [INFO ] [MAIN ] JKS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:42.033483] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T19:28:42.079924] [DEBUG] [LOCAL ] JKS: Loaded 40154 existing records from primary storage. [2025-10-20T19:28:42.080223] [INFO ] [MAIN ] JKS: Loaded 40154 existing records. [2025-10-20T19:28:42.257602] [WARN ] [MAIN ] JKS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:42.257841] [INFO ] [MAIN ] JKS: Wiping existing primary data before full refresh. [2025-10-20T19:28:42.257904] [INFO ] [LOCAL ] JKS: Starting deletion of primary data. [2025-10-20T19:28:42.258264] [DEBUG] [LOCAL ] JKS: Deleted primary data file. [2025-10-20T19:28:42.258303] [INFO ] [MAIN ] JKS: Starting full primary refresh. [2025-10-20T19:28:42.613164] [INFO ] [REMOTE ] JKS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:42.651339] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-01. [2025-10-20T19:28:42.681576] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T19:28:42.682460] [INFO ] [REMOTE ] JHS: Full fetch complete (4446 records). [2025-10-20T19:28:42.682525] [DEBUG] [VALIDATOR ] JHS: Validating primary data. [2025-10-20T19:28:43.041780] [DEBUG] [VALIDATOR ] JHS: All expected trading days are present. [2025-10-20T19:28:43.041853] [DEBUG] [VALIDATOR ] JHS: Primary data validation passed. [2025-10-20T19:28:43.041986] [INFO ] [MAIN ] JHS: Fetched primary data is valid. [2025-10-20T19:28:43.042023] [DEBUG] [LOCAL ] JHS: Saving primary data. [2025-10-20T19:28:43.042357] [DEBUG] [LOCAL ] JHS: Trimmed 4446 records to 2931 (keeping 200 days of data) [2025-10-20T19:28:43.059852] [DEBUG] [LOCAL ] JHS: Primary data saved (2931 records). [2025-10-20T19:28:43.059919] [INFO ] [MAIN ] JHS: Full history saved. [2025-10-20T19:28:43.059992] [INFO ] [MAIN ] Update Progress: 1153/2367 symbols completed. JHS took 33.9s. Estimated remaining: 2h 36m 31s. Rate: 7.8 symbols/min [2025-10-20T19:28:43.230180] [INFO ] [MAIN ] JLL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:43.261466] [DEBUG] [LOCAL ] JLL: Loaded 32147 existing records from primary storage. [2025-10-20T19:28:43.261745] [INFO ] [MAIN ] JLL: Loaded 32147 existing records. [2025-10-20T19:28:43.447934] [WARN ] [MAIN ] JLL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:43.448138] [INFO ] [MAIN ] JLL: Wiping existing primary data before full refresh. [2025-10-20T19:28:43.448185] [INFO ] [LOCAL ] JLL: Starting deletion of primary data. [2025-10-20T19:28:43.448534] [DEBUG] [LOCAL ] JLL: Deleted primary data file. [2025-10-20T19:28:43.448573] [INFO ] [MAIN ] JLL: Starting full primary refresh. [2025-10-20T19:28:43.460307] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-09. [2025-10-20T19:28:43.801096] [INFO ] [REMOTE ] JLL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:44.256845] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-01. [2025-10-20T19:28:44.270329] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:28:44.520234] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:28:45.051373] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-10. [2025-10-20T19:28:45.565019] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T19:28:45.851384] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-02. [2025-10-20T19:28:46.013544] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T19:28:46.015996] [INFO ] [REMOTE ] JILL: Full fetch complete (22419 records). [2025-10-20T19:28:46.016096] [DEBUG] [VALIDATOR ] JILL: Validating primary data. [2025-10-20T19:28:46.393579] [DEBUG] [VALIDATOR ] JILL: All expected trading days are present. [2025-10-20T19:28:46.393691] [DEBUG] [VALIDATOR ] JILL: Primary data validation passed. [2025-10-20T19:28:46.394241] [INFO ] [MAIN ] JILL: Fetched primary data is valid. [2025-10-20T19:28:46.394283] [DEBUG] [LOCAL ] JILL: Saving primary data. [2025-10-20T19:28:46.394981] [DEBUG] [LOCAL ] JILL: Trimmed 22419 records to 15604 (keeping 200 days of data) [2025-10-20T19:28:46.490054] [DEBUG] [LOCAL ] JILL: Primary data saved (15604 records). [2025-10-20T19:28:46.490187] [INFO ] [MAIN ] JILL: Full history saved. [2025-10-20T19:28:46.490267] [INFO ] [MAIN ] Update Progress: 1154/2367 symbols completed. JILL took 24.3s. Estimated remaining: 2h 36m 18s. Rate: 7.8 symbols/min [2025-10-20T19:28:46.661960] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-02. [2025-10-20T19:28:46.664483] [INFO ] [MAIN ] JLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:46.670886] [DEBUG] [LOCAL ] JLS: Loaded 4259 existing records from primary storage. [2025-10-20T19:28:46.671047] [INFO ] [MAIN ] JLS: Loaded 4259 existing records. [2025-10-20T19:28:46.846454] [WARN ] [MAIN ] JLS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:46.846760] [INFO ] [MAIN ] JLS: Wiping existing primary data before full refresh. [2025-10-20T19:28:46.846811] [INFO ] [LOCAL ] JLS: Starting deletion of primary data. [2025-10-20T19:28:46.846987] [DEBUG] [LOCAL ] JLS: Deleted primary data file. [2025-10-20T19:28:46.847044] [INFO ] [MAIN ] JLS: Starting full primary refresh. [2025-10-20T19:28:47.204847] [INFO ] [REMOTE ] JLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:47.237960] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T19:28:47.451374] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-03. [2025-10-20T19:28:48.221031] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T19:28:48.251342] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-03. [2025-10-20T19:28:48.468946] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T19:28:49.051377] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-04. [2025-10-20T19:28:49.851402] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-07. [2025-10-20T19:28:49.916953] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:28:50.201888] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:28:50.651380] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-04. [2025-10-20T19:28:51.451385] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-05. [2025-10-20T19:28:51.566479] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T19:28:52.251576] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-01. [2025-10-20T19:28:52.352746] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:28:52.782253] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T19:28:52.815004] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:28:53.051362] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-08. [2025-10-20T19:28:53.851398] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-06. [2025-10-20T19:28:54.651437] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-05. [2025-10-20T19:28:54.972468] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T19:28:55.168006] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T19:28:55.451400] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-09. [2025-10-20T19:28:56.130160] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T19:28:56.251386] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-07. [2025-10-20T19:28:57.051363] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-06. [2025-10-20T19:28:57.295816] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T19:28:57.567085] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T19:28:57.851396] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-02. [2025-10-20T19:28:58.155991] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T19:28:58.480408] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T19:28:58.651358] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-07. [2025-10-20T19:28:59.451376] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-08. [2025-10-20T19:29:00.216709] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:29:00.251376] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-08. [2025-10-20T19:29:00.947523] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:29:01.051415] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-03. [2025-10-20T19:29:01.614402] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T19:29:01.701543] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T19:29:01.851387] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-10. [2025-10-20T19:29:02.651383] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-09. [2025-10-20T19:29:03.451348] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-09. [2025-10-20T19:29:03.861337] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T19:29:03.869911] [INFO ] [REMOTE ] JHX: Full fetch complete (61997 records). [2025-10-20T19:29:03.870141] [DEBUG] [VALIDATOR ] JHX: Validating primary data. [2025-10-20T19:29:04.259894] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-04. [2025-10-20T19:29:04.282963] [DEBUG] [VALIDATOR ] JHX: All expected trading days are present. [2025-10-20T19:29:04.283116] [DEBUG] [VALIDATOR ] JHX: Primary data validation passed. [2025-10-20T19:29:04.284993] [INFO ] [MAIN ] JHX: Fetched primary data is valid. [2025-10-20T19:29:04.285366] [DEBUG] [LOCAL ] JHX: Saving primary data. [2025-10-20T19:29:04.287371] [DEBUG] [LOCAL ] JHX: Trimmed 61997 records to 53195 (keeping 200 days of data) [2025-10-20T19:29:04.538001] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T19:29:04.678220] [DEBUG] [LOCAL ] JHX: Primary data saved (53195 records). [2025-10-20T19:29:04.678380] [INFO ] [MAIN ] JHX: Full history saved. [2025-10-20T19:29:04.678464] [INFO ] [MAIN ] Update Progress: 1155/2367 symbols completed. JHX took 55.4s. Estimated remaining: 2h 36m 21s. Rate: 7.8 symbols/min [2025-10-20T19:29:04.850758] [INFO ] [MAIN ] JMIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:04.869288] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:29:04.924480] [DEBUG] [LOCAL ] JMIA: Loaded 51831 existing records from primary storage. [2025-10-20T19:29:04.924766] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T19:29:04.926695] [INFO ] [MAIN ] JMIA: Loaded 51831 existing records. [2025-10-20T19:29:05.063865] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-10. [2025-10-20T19:29:05.104435] [WARN ] [MAIN ] JMIA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:05.104865] [INFO ] [MAIN ] JMIA: Wiping existing primary data before full refresh. [2025-10-20T19:29:05.104937] [INFO ] [LOCAL ] JMIA: Starting deletion of primary data. [2025-10-20T19:29:05.105187] [DEBUG] [LOCAL ] JMIA: Deleted primary data file. [2025-10-20T19:29:05.105227] [INFO ] [MAIN ] JMIA: Starting full primary refresh. [2025-10-20T19:29:05.453231] [INFO ] [REMOTE ] JMIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:05.851391] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-01. [2025-10-20T19:29:06.541004] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T19:29:06.545173] [INFO ] [REMOTE ] JLL: Full fetch complete (45164 records). [2025-10-20T19:29:06.545263] [DEBUG] [VALIDATOR ] JLL: Validating primary data. [2025-10-20T19:29:06.656572] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-05. [2025-10-20T19:29:06.917736] [DEBUG] [VALIDATOR ] JLL: All expected trading days are present. [2025-10-20T19:29:06.917814] [DEBUG] [VALIDATOR ] JLL: Primary data validation passed. [2025-10-20T19:29:06.918520] [INFO ] [MAIN ] JLL: Fetched primary data is valid. [2025-10-20T19:29:06.918559] [DEBUG] [LOCAL ] JLL: Saving primary data. [2025-10-20T19:29:06.919466] [DEBUG] [LOCAL ] JLL: Trimmed 45164 records to 31753 (keeping 200 days of data) [2025-10-20T19:29:07.106732] [DEBUG] [LOCAL ] JLL: Primary data saved (31753 records). [2025-10-20T19:29:07.106851] [INFO ] [MAIN ] JLL: Full history saved. [2025-10-20T19:29:07.106922] [INFO ] [MAIN ] Update Progress: 1156/2367 symbols completed. JLL took 24.0s. Estimated remaining: 2h 36m 8s. Rate: 7.8 symbols/min [2025-10-20T19:29:07.309068] [INFO ] [MAIN ] JMM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:07.313321] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T19:29:07.313461] [DEBUG] [LOCAL ] JMM: No existing data file found in primary storage. [2025-10-20T19:29:07.315281] [WARN ] [MAIN ] JMM: No existing data found, performing full refresh instead. [2025-10-20T19:29:07.315423] [INFO ] [MAIN ] JMM: Wiping existing primary data before full refresh. [2025-10-20T19:29:07.315490] [INFO ] [LOCAL ] JMM: Starting deletion of primary data. [2025-10-20T19:29:07.315544] [DEBUG] [LOCAL ] JMM: No primary data file found to delete. [2025-10-20T19:29:07.315577] [INFO ] [MAIN ] JMM: Starting full primary refresh. [2025-10-20T19:29:07.456500] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-10. [2025-10-20T19:29:07.710965] [INFO ] [REMOTE ] JMM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:07.742825] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T19:29:08.251456] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-02. [2025-10-20T19:29:09.051435] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-01. [2025-10-20T19:29:09.137600] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:29:09.143758] [INFO ] [REMOTE ] JKS: Full fetch complete (58805 records). [2025-10-20T19:29:09.143896] [DEBUG] [VALIDATOR ] JKS: Validating primary data. [2025-10-20T19:29:09.437267] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-10-20T19:29:09.540692] [DEBUG] [VALIDATOR ] JKS: All expected trading days are present. [2025-10-20T19:29:09.540789] [DEBUG] [VALIDATOR ] JKS: Primary data validation passed. [2025-10-20T19:29:09.541619] [INFO ] [MAIN ] JKS: Fetched primary data is valid. [2025-10-20T19:29:09.541690] [DEBUG] [LOCAL ] JKS: Saving primary data. [2025-10-20T19:29:09.542910] [DEBUG] [LOCAL ] JKS: Trimmed 58805 records to 39426 (keeping 200 days of data) [2025-10-20T19:29:09.775488] [DEBUG] [LOCAL ] JKS: Primary data saved (39426 records). [2025-10-20T19:29:09.775765] [INFO ] [MAIN ] JKS: Full history saved. [2025-10-20T19:29:09.775865] [INFO ] [MAIN ] Update Progress: 1157/2367 symbols completed. JKS took 27.9s. Estimated remaining: 2h 35m 55s. Rate: 7.8 symbols/min [2025-10-20T19:29:09.851344] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-06. [2025-10-20T19:29:09.961582] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T19:29:10.188974] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-10-20T19:29:10.651388] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-07. [2025-10-20T19:29:11.217163] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T19:29:11.451425] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-03. [2025-10-20T19:29:12.251402] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-08. [2025-10-20T19:29:12.842351] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T19:29:13.051360] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-02. [2025-10-20T19:29:13.317777] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T19:29:13.387995] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-10-20T19:29:13.851370] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-03. [2025-10-20T19:29:14.198188] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T19:29:14.651385] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-04. [2025-10-20T19:29:15.451419] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-09. [2025-10-20T19:29:16.040159] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:29:16.251379] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-10. [2025-10-20T19:29:16.283504] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T19:29:16.930280] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T19:29:16.931838] [INFO ] [REMOTE ] JLS: Full fetch complete (5810 records). [2025-10-20T19:29:16.931928] [DEBUG] [VALIDATOR ] JLS: Validating primary data. [2025-10-20T19:29:17.056763] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-05. [2025-10-20T19:29:17.313888] [DEBUG] [VALIDATOR ] JLS: All expected trading days are present. [2025-10-20T19:29:17.313965] [DEBUG] [VALIDATOR ] JLS: Primary data validation passed. [2025-10-20T19:29:17.314140] [INFO ] [MAIN ] JLS: Fetched primary data is valid. [2025-10-20T19:29:17.314179] [DEBUG] [LOCAL ] JLS: Saving primary data. [2025-10-20T19:29:17.314540] [DEBUG] [LOCAL ] JLS: Trimmed 5810 records to 4066 (keeping 200 days of data) [2025-10-20T19:29:17.338767] [DEBUG] [LOCAL ] JLS: Primary data saved (4066 records). [2025-10-20T19:29:17.338830] [INFO ] [MAIN ] JLS: Full history saved. [2025-10-20T19:29:17.338894] [INFO ] [MAIN ] Update Progress: 1158/2367 symbols completed. JLS took 30.8s. Estimated remaining: 2h 35m 47s. Rate: 7.8 symbols/min [2025-10-20T19:29:17.851447] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-04. [2025-10-20T19:29:18.289981] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-10-20T19:29:18.651347] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-05. [2025-10-20T19:29:18.923738] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T19:29:19.118495] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T19:29:19.451421] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-06. [2025-10-20T19:29:19.879540] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T19:29:20.251376] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-07. [2025-10-20T19:29:20.717371] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T19:29:21.051365] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-08. [2025-10-20T19:29:21.497848] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:29:21.851422] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-06. [2025-10-20T19:29:22.651440] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-09. [2025-10-20T19:29:23.085051] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-10-20T19:29:23.451367] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-10. [2025-10-20T19:29:23.499759] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:29:24.004099] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T19:29:24.004849] [INFO ] [REMOTE ] JMM: Full fetch complete (2298 records). [2025-10-20T19:29:24.004918] [DEBUG] [VALIDATOR ] JMM: Validating primary data. [2025-10-20T19:29:24.184025] [WARN ] [VALIDATOR ] JMM: Validation failed - insufficient historical data. Have 194 unique days, need 200 days [2025-10-20T19:29:24.184207] [FATAL] [MAIN ] JMM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:29:24.184286] [WARN ] [MAIN ] JMM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:29:24.184333] [INFO ] [LOCAL ] JMM: Starting deletion of primary data. [2025-10-20T19:29:24.184386] [DEBUG] [LOCAL ] JMM: No primary data file found to delete. [2025-10-20T19:29:24.184421] [INFO ] [LOCAL ] JMM: Starting deletion of derivative data. [2025-10-20T19:29:24.184458] [DEBUG] [LOCAL ] JMM: No derivative data file found to delete. [2025-10-20T19:29:24.184512] [INFO ] [MAIN ] Update Progress: 1159/2367 symbols completed. JMM took 17.1s. Estimated remaining: 2h 35m 39s. Rate: 7.8 symbols/min [2025-10-20T19:29:24.251394] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-07. [2025-10-20T19:29:26.231737] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T19:29:26.242494] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-08. [2025-10-20T19:29:28.213040] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T19:29:28.224339] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-09. [2025-10-20T19:29:30.215809] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T19:29:30.226923] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-10. [2025-10-20T19:29:32.717620] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-10 (2.49 seconds). [2025-10-20T19:29:32.725226] [INFO ] [REMOTE ] JMIA: Full fetch complete (73466 records). [2025-10-20T19:29:32.725374] [DEBUG] [VALIDATOR ] JMIA: Validating primary data. [2025-10-20T19:29:33.135134] [DEBUG] [VALIDATOR ] JMIA: All expected trading days are present. [2025-10-20T19:29:33.135262] [DEBUG] [VALIDATOR ] JMIA: Primary data validation passed. [2025-10-20T19:29:33.136280] [INFO ] [MAIN ] JMIA: Fetched primary data is valid. [2025-10-20T19:29:33.136343] [DEBUG] [LOCAL ] JMIA: Saving primary data. [2025-10-20T19:29:33.137756] [DEBUG] [LOCAL ] JMIA: Trimmed 73466 records to 51222 (keeping 200 days of data) [2025-10-20T19:29:33.422096] [DEBUG] [LOCAL ] JMIA: Primary data saved (51222 records). [2025-10-20T19:29:33.422251] [INFO ] [MAIN ] JMIA: Full history saved. [2025-10-20T19:29:33.422332] [INFO ] [MAIN ] Update Progress: 1160/2367 symbols completed. JMIA took 28.7s. Estimated remaining: 2h 35m 32s. Rate: 7.8 symbols/min [2025-10-20T19:29:33.997242] [INFO ] [MAIN ] JNJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:34.093414] [INFO ] [MAIN ] JOE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:34.136853] [INFO ] [MAIN ] JOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:34.156569] [INFO ] [MAIN ] JOBY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:34.170185] [DEBUG] [LOCAL ] JOF: Loaded 6969 existing records from primary storage. [2025-10-20T19:29:34.170576] [INFO ] [MAIN ] JOF: Loaded 6969 existing records. [2025-10-20T19:29:34.222275] [DEBUG] [LOCAL ] JOE: Loaded 24168 existing records from primary storage. [2025-10-20T19:29:34.246420] [INFO ] [MAIN ] JOE: Loaded 24168 existing records. [2025-10-20T19:29:34.561513] [WARN ] [MAIN ] JOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:34.603221] [DEBUG] [LOCAL ] JNJ: Loaded 54533 existing records from primary storage. [2025-10-20T19:29:34.601796] [WARN ] [MAIN ] JOE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:34.604397] [INFO ] [MAIN ] JOE: Wiping existing primary data before full refresh. [2025-10-20T19:29:34.603390] [INFO ] [MAIN ] JOF: Wiping existing primary data before full refresh. [2025-10-20T19:29:34.604748] [INFO ] [LOCAL ] JOE: Starting deletion of primary data. [2025-10-20T19:29:34.605078] [INFO ] [MAIN ] JNJ: Loaded 54533 existing records. [2025-10-20T19:29:34.604848] [INFO ] [LOCAL ] JOF: Starting deletion of primary data. [2025-10-20T19:29:34.632329] [DEBUG] [LOCAL ] JOE: Deleted primary data file. [2025-10-20T19:29:34.653665] [INFO ] [MAIN ] JOE: Starting full primary refresh. [2025-10-20T19:29:34.659558] [DEBUG] [LOCAL ] JOF: Deleted primary data file. [2025-10-20T19:29:34.659488] [DEBUG] [LOCAL ] JOBY: Loaded 54588 existing records from primary storage. [2025-10-20T19:29:34.690439] [INFO ] [MAIN ] JOBY: Loaded 54588 existing records. [2025-10-20T19:29:34.681205] [INFO ] [MAIN ] JOF: Starting full primary refresh. [2025-10-20T19:29:35.298156] [WARN ] [MAIN ] JNJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:35.395373] [INFO ] [MAIN ] JNJ: Wiping existing primary data before full refresh. [2025-10-20T19:29:35.391005] [WARN ] [MAIN ] JOBY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:35.411661] [INFO ] [MAIN ] JOBY: Wiping existing primary data before full refresh. [2025-10-20T19:29:35.411405] [INFO ] [LOCAL ] JNJ: Starting deletion of primary data. [2025-10-20T19:29:35.446451] [INFO ] [LOCAL ] JOBY: Starting deletion of primary data. [2025-10-20T19:29:35.460798] [DEBUG] [LOCAL ] JOBY: Deleted primary data file. [2025-10-20T19:29:35.497902] [INFO ] [MAIN ] JOBY: Starting full primary refresh. [2025-10-20T19:29:35.516948] [DEBUG] [LOCAL ] JNJ: Deleted primary data file. [2025-10-20T19:29:35.531599] [INFO ] [MAIN ] JNJ: Starting full primary refresh. [2025-10-20T19:29:36.015235] [INFO ] [REMOTE ] JOE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:36.122199] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-01. [2025-10-20T19:29:36.205940] [INFO ] [REMOTE ] JOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:36.241891] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-01. [2025-10-20T19:29:36.524669] [INFO ] [REMOTE ] JNJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:36.543214] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-01. [2025-10-20T19:29:36.542981] [INFO ] [REMOTE ] JOBY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:36.544312] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-01. [2025-10-20T19:29:36.913892] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T19:29:36.915431] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-02. [2025-10-20T19:29:37.374364] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T19:29:37.380274] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-02. [2025-10-20T19:29:37.522184] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T19:29:37.523744] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-03. [2025-10-20T19:29:38.105202] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T19:29:38.107467] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-04. [2025-10-20T19:29:38.457694] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T19:29:38.462439] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-03. [2025-10-20T19:29:38.760548] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-01 (2.22 seconds). [2025-10-20T19:29:38.771006] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-02. [2025-10-20T19:29:38.868851] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T19:29:38.871773] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-05. [2025-10-20T19:29:39.071330] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-01 (2.53 seconds). [2025-10-20T19:29:39.081775] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-02. [2025-10-20T19:29:39.252993] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-10-20T19:29:39.254776] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-06. [2025-10-20T19:29:39.338064] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T19:29:39.344024] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-04. [2025-10-20T19:29:39.835614] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T19:29:39.837560] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-07. [2025-10-20T19:29:40.286139] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T19:29:40.293222] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-05. [2025-10-20T19:29:40.523354] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-10-20T19:29:40.526504] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-08. [2025-10-20T19:29:40.610694] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T19:29:41.051414] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-03. [2025-10-20T19:29:41.249389] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-10-20T19:29:41.302882] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-02 (2.22 seconds). [2025-10-20T19:29:41.414273] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T19:29:41.851395] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-06. [2025-10-20T19:29:42.651405] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-03. [2025-10-20T19:29:42.991069] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:29:43.078032] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T19:29:43.451441] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-07. [2025-10-20T19:29:44.251402] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-09. [2025-10-20T19:29:44.613050] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T19:29:44.956465] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T19:29:45.051342] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-10. [2025-10-20T19:29:45.104553] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-03 (2.45 seconds). [2025-10-20T19:29:45.764212] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T19:29:45.765733] [INFO ] [REMOTE ] JOF: Full fetch complete (9416 records). [2025-10-20T19:29:45.765800] [DEBUG] [VALIDATOR ] JOF: Validating primary data. [2025-10-20T19:29:45.856604] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-08. [2025-10-20T19:29:46.140493] [DEBUG] [VALIDATOR ] JOF: All expected trading days are present. [2025-10-20T19:29:46.140575] [DEBUG] [VALIDATOR ] JOF: Primary data validation passed. [2025-10-20T19:29:46.140886] [INFO ] [MAIN ] JOF: Fetched primary data is valid. [2025-10-20T19:29:46.140949] [DEBUG] [LOCAL ] JOF: Saving primary data. [2025-10-20T19:29:46.141400] [DEBUG] [LOCAL ] JOF: Trimmed 9416 records to 6925 (keeping 200 days of data) [2025-10-20T19:29:46.183597] [DEBUG] [LOCAL ] JOF: Primary data saved (6925 records). [2025-10-20T19:29:46.183765] [INFO ] [MAIN ] JOF: Full history saved. [2025-10-20T19:29:46.183852] [INFO ] [MAIN ] Update Progress: 1161/2367 symbols completed. JOF took 12.7s. Estimated remaining: 2h 35m 30s. Rate: 7.8 symbols/min [2025-10-20T19:29:46.367086] [INFO ] [MAIN ] JPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:46.401677] [DEBUG] [LOCAL ] JPC: Loaded 37747 existing records from primary storage. [2025-10-20T19:29:46.401956] [INFO ] [MAIN ] JPC: Loaded 37747 existing records. [2025-10-20T19:29:46.576220] [WARN ] [MAIN ] JPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:46.576443] [INFO ] [MAIN ] JPC: Wiping existing primary data before full refresh. [2025-10-20T19:29:46.576507] [INFO ] [LOCAL ] JPC: Starting deletion of primary data. [2025-10-20T19:29:46.576788] [DEBUG] [LOCAL ] JPC: Deleted primary data file. [2025-10-20T19:29:46.576863] [INFO ] [MAIN ] JPC: Starting full primary refresh. [2025-10-20T19:29:46.656463] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-04. [2025-10-20T19:29:46.965555] [INFO ] [REMOTE ] JPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:46.978342] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T19:29:47.451422] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-09. [2025-10-20T19:29:48.251357] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-01. [2025-10-20T19:29:48.472831] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:29:49.052682] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-10. [2025-10-20T19:29:49.058565] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-04 (2.40 seconds). [2025-10-20T19:29:49.572306] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T19:29:49.851355] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-05. [2025-10-20T19:29:50.169158] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T19:29:50.174467] [INFO ] [REMOTE ] JOE: Full fetch complete (35625 records). [2025-10-20T19:29:50.174580] [DEBUG] [VALIDATOR ] JOE: Validating primary data. [2025-10-20T19:29:50.547675] [DEBUG] [VALIDATOR ] JOE: All expected trading days are present. [2025-10-20T19:29:50.547778] [DEBUG] [VALIDATOR ] JOE: Primary data validation passed. [2025-10-20T19:29:50.548450] [INFO ] [MAIN ] JOE: Fetched primary data is valid. [2025-10-20T19:29:50.548499] [DEBUG] [LOCAL ] JOE: Saving primary data. [2025-10-20T19:29:50.549514] [DEBUG] [LOCAL ] JOE: Trimmed 35625 records to 23890 (keeping 200 days of data) [2025-10-20T19:29:50.656016] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-02. [2025-10-20T19:29:50.690893] [DEBUG] [LOCAL ] JOE: Primary data saved (23890 records). [2025-10-20T19:29:50.691006] [INFO ] [MAIN ] JOE: Full history saved. [2025-10-20T19:29:50.691078] [INFO ] [MAIN ] Update Progress: 1162/2367 symbols completed. JOE took 17.3s. Estimated remaining: 2h 35m 19s. Rate: 7.8 symbols/min [2025-10-20T19:29:50.868114] [INFO ] [MAIN ] JPM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:50.920491] [DEBUG] [LOCAL ] JPM: Loaded 54574 existing records from primary storage. [2025-10-20T19:29:50.920873] [INFO ] [MAIN ] JPM: Loaded 54574 existing records. [2025-10-20T19:29:51.108318] [WARN ] [MAIN ] JPM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:51.108545] [INFO ] [MAIN ] JPM: Wiping existing primary data before full refresh. [2025-10-20T19:29:51.108602] [INFO ] [LOCAL ] JPM: Starting deletion of primary data. [2025-10-20T19:29:51.108921] [DEBUG] [LOCAL ] JPM: Deleted primary data file. [2025-10-20T19:29:51.108978] [INFO ] [MAIN ] JPM: Starting full primary refresh. [2025-10-20T19:29:51.452139] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-04. [2025-10-20T19:29:51.470977] [INFO ] [REMOTE ] JPM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:52.030512] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T19:29:52.239965] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-05 (2.39 seconds). [2025-10-20T19:29:52.251335] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-06. [2025-10-20T19:29:53.051422] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-01. [2025-10-20T19:29:53.774418] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-04 (2.32 seconds). [2025-10-20T19:29:53.851343] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-05. [2025-10-20T19:29:54.314131] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T19:29:54.651369] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-07. [2025-10-20T19:29:55.331980] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-01 (2.28 seconds). [2025-10-20T19:29:55.451351] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-02. [2025-10-20T19:29:56.022267] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T19:29:56.251380] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-03. [2025-10-20T19:29:56.363600] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T19:29:57.051372] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-06. [2025-10-20T19:29:57.751019] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:29:57.802065] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-10-20T19:29:57.851375] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-04. [2025-10-20T19:29:58.651432] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-08. [2025-10-20T19:29:59.189285] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T19:29:59.451404] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-05. [2025-10-20T19:29:59.507339] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-06 (2.46 seconds). [2025-10-20T19:30:00.251358] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-07. [2025-10-20T19:30:00.729861] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T19:30:00.897418] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-10-20T19:30:01.051369] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-06. [2025-10-20T19:30:01.851356] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-09. [2025-10-20T19:30:02.352975] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T19:30:02.651379] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-03. [2025-10-20T19:30:03.211989] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-07 (2.96 seconds). [2025-10-20T19:30:03.451447] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-08. [2025-10-20T19:30:04.252408] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-07. [2025-10-20T19:30:04.262255] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-09 (2.41 seconds). [2025-10-20T19:30:05.055055] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-10. [2025-10-20T19:30:05.060629] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-03 (2.41 seconds). [2025-10-20T19:30:05.851362] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-04. [2025-10-20T19:30:05.965296] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T19:30:06.288363] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-08 (2.84 seconds). [2025-10-20T19:30:06.651384] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-08. [2025-10-20T19:30:07.451459] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-09. [2025-10-20T19:30:07.508046] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-10 (2.45 seconds). [2025-10-20T19:30:07.516256] [INFO ] [REMOTE ] JNJ: Full fetch complete (77933 records). [2025-10-20T19:30:07.516426] [DEBUG] [VALIDATOR ] JNJ: Validating primary data. [2025-10-20T19:30:07.931531] [DEBUG] [VALIDATOR ] JNJ: All expected trading days are present. [2025-10-20T19:30:07.931622] [DEBUG] [VALIDATOR ] JNJ: Primary data validation passed. [2025-10-20T19:30:07.932632] [INFO ] [MAIN ] JNJ: Fetched primary data is valid. [2025-10-20T19:30:07.932726] [DEBUG] [LOCAL ] JNJ: Saving primary data. [2025-10-20T19:30:07.934217] [DEBUG] [LOCAL ] JNJ: Trimmed 77933 records to 53753 (keeping 200 days of data) [2025-10-20T19:30:08.299453] [DEBUG] [LOCAL ] JNJ: Primary data saved (53753 records). [2025-10-20T19:30:08.299589] [INFO ] [MAIN ] JNJ: Full history saved. [2025-10-20T19:30:08.299691] [INFO ] [MAIN ] Update Progress: 1163/2367 symbols completed. JNJ took 34.9s. Estimated remaining: 2h 35m 21s. Rate: 7.7 symbols/min [2025-10-20T19:30:08.490985] [INFO ] [MAIN ] JQC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:08.492867] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T19:30:08.501769] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-09. [2025-10-20T19:30:08.532410] [DEBUG] [LOCAL ] JQC: Loaded 34272 existing records from primary storage. [2025-10-20T19:30:08.532697] [INFO ] [MAIN ] JQC: Loaded 34272 existing records. [2025-10-20T19:30:08.713925] [WARN ] [MAIN ] JQC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:08.714191] [INFO ] [MAIN ] JQC: Wiping existing primary data before full refresh. [2025-10-20T19:30:08.714287] [INFO ] [LOCAL ] JQC: Starting deletion of primary data. [2025-10-20T19:30:08.714564] [DEBUG] [LOCAL ] JQC: Deleted primary data file. [2025-10-20T19:30:08.714668] [INFO ] [MAIN ] JQC: Starting full primary refresh. [2025-10-20T19:30:08.994229] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-04 (3.14 seconds). [2025-10-20T19:30:09.056448] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-05. [2025-10-20T19:30:09.126975] [INFO ] [REMOTE ] JQC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:09.851372] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-01. [2025-10-20T19:30:09.901494] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T19:30:09.949455] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-09 (2.50 seconds). [2025-10-20T19:30:10.651377] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-10. [2025-10-20T19:30:11.307215] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T19:30:11.451391] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-02. [2025-10-20T19:30:11.582430] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-05 (2.53 seconds). [2025-10-20T19:30:12.251356] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-06. [2025-10-20T19:30:12.818359] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T19:30:13.051435] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-03. [2025-10-20T19:30:13.448398] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-10 (2.80 seconds). [2025-10-20T19:30:13.456585] [INFO ] [REMOTE ] JOBY: Full fetch complete (77974 records). [2025-10-20T19:30:13.456804] [DEBUG] [VALIDATOR ] JOBY: Validating primary data. [2025-10-20T19:30:13.852186] [DEBUG] [VALIDATOR ] JOBY: All expected trading days are present. [2025-10-20T19:30:13.851959] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-10. [2025-10-20T19:30:13.852387] [DEBUG] [VALIDATOR ] JOBY: Primary data validation passed. [2025-10-20T19:30:13.853677] [INFO ] [MAIN ] JOBY: Fetched primary data is valid. [2025-10-20T19:30:13.854391] [DEBUG] [LOCAL ] JOBY: Saving primary data. [2025-10-20T19:30:13.855995] [DEBUG] [LOCAL ] JOBY: Trimmed 77974 records to 53809 (keeping 200 days of data) [2025-10-20T19:30:14.191548] [DEBUG] [LOCAL ] JOBY: Primary data saved (53809 records). [2025-10-20T19:30:14.191800] [INFO ] [MAIN ] JOBY: Full history saved. [2025-10-20T19:30:14.191955] [INFO ] [MAIN ] Update Progress: 1164/2367 symbols completed. JOBY took 40.7s. Estimated remaining: 2h 35m 12s. Rate: 7.8 symbols/min [2025-10-20T19:30:14.369746] [INFO ] [MAIN ] JRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:14.407735] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-06 (2.16 seconds). [2025-10-20T19:30:14.429075] [DEBUG] [LOCAL ] JRI: Loaded 15593 existing records from primary storage. [2025-10-20T19:30:14.429518] [INFO ] [MAIN ] JRI: Loaded 15593 existing records. [2025-10-20T19:30:14.597524] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:30:14.652854] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-04. [2025-10-20T19:30:14.652276] [WARN ] [MAIN ] JRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:14.654002] [INFO ] [MAIN ] JRI: Wiping existing primary data before full refresh. [2025-10-20T19:30:14.654071] [INFO ] [LOCAL ] JRI: Starting deletion of primary data. [2025-10-20T19:30:14.654320] [DEBUG] [LOCAL ] JRI: Deleted primary data file. [2025-10-20T19:30:14.654387] [INFO ] [MAIN ] JRI: Starting full primary refresh. [2025-10-20T19:30:15.032728] [INFO ] [REMOTE ] JRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:15.054374] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T19:30:15.059947] [INFO ] [REMOTE ] JPC: Full fetch complete (55065 records). [2025-10-20T19:30:15.060034] [DEBUG] [VALIDATOR ] JPC: Validating primary data. [2025-10-20T19:30:15.452111] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-01. [2025-10-20T19:30:15.470082] [DEBUG] [VALIDATOR ] JPC: All expected trading days are present. [2025-10-20T19:30:15.470166] [DEBUG] [VALIDATOR ] JPC: Primary data validation passed. [2025-10-20T19:30:15.471114] [INFO ] [MAIN ] JPC: Fetched primary data is valid. [2025-10-20T19:30:15.471321] [DEBUG] [LOCAL ] JPC: Saving primary data. [2025-10-20T19:30:15.472562] [DEBUG] [LOCAL ] JPC: Trimmed 55065 records to 37073 (keeping 200 days of data) [2025-10-20T19:30:15.682369] [DEBUG] [LOCAL ] JPC: Primary data saved (37073 records). [2025-10-20T19:30:15.682509] [INFO ] [MAIN ] JPC: Full history saved. [2025-10-20T19:30:15.682586] [INFO ] [MAIN ] Update Progress: 1165/2367 symbols completed. JPC took 29.5s. Estimated remaining: 2h 34m 57s. Rate: 7.8 symbols/min [2025-10-20T19:30:15.856757] [INFO ] [MAIN ] JRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:15.869542] [DEBUG] [LOCAL ] JRS: Loaded 11802 existing records from primary storage. [2025-10-20T19:30:15.869925] [INFO ] [MAIN ] JRS: Loaded 11802 existing records. [2025-10-20T19:30:16.079488] [WARN ] [MAIN ] JRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:16.079804] [INFO ] [MAIN ] JRS: Wiping existing primary data before full refresh. [2025-10-20T19:30:16.079946] [INFO ] [LOCAL ] JRS: Starting deletion of primary data. [2025-10-20T19:30:16.080115] [DEBUG] [LOCAL ] JRS: Deleted primary data file. [2025-10-20T19:30:16.080156] [INFO ] [MAIN ] JRS: Starting full primary refresh. [2025-10-20T19:30:16.275727] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-07. [2025-10-20T19:30:16.434420] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T19:30:16.504069] [INFO ] [REMOTE ] JRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:16.592705] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:30:17.051413] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-01. [2025-10-20T19:30:17.851470] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-02. [2025-10-20T19:30:17.865416] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T19:30:18.379097] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T19:30:18.651400] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-02. [2025-10-20T19:30:18.856716] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-07 (2.58 seconds). [2025-10-20T19:30:19.451415] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-08. [2025-10-20T19:30:19.471700] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T19:30:20.251398] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-03. [2025-10-20T19:30:21.051445] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-05. [2025-10-20T19:30:21.077197] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T19:30:21.702246] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-10-20T19:30:21.851423] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-04. [2025-10-20T19:30:22.052913] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:30:22.651381] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-09. [2025-10-20T19:30:22.661278] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T19:30:23.451408] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-06. [2025-10-20T19:30:24.251423] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-05. [2025-10-20T19:30:24.570018] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:30:24.714462] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T19:30:25.051361] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-03. [2025-10-20T19:30:25.097131] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-09 (2.45 seconds). [2025-10-20T19:30:25.851394] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-06. [2025-10-20T19:30:26.060177] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:30:26.648327] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T19:30:26.652128] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-04. [2025-10-20T19:30:27.451338] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-10. [2025-10-20T19:30:27.641325] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T19:30:28.251446] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-07. [2025-10-20T19:30:29.051390] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-07. [2025-10-20T19:30:29.235754] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:30:29.851392] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-05. [2025-10-20T19:30:30.056906] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-10 (2.61 seconds). [2025-10-20T19:30:30.065021] [INFO ] [REMOTE ] JPM: Full fetch complete (77973 records). [2025-10-20T19:30:30.065141] [DEBUG] [VALIDATOR ] JPM: Validating primary data. [2025-10-20T19:30:30.481154] [DEBUG] [VALIDATOR ] JPM: All expected trading days are present. [2025-10-20T19:30:30.481225] [DEBUG] [VALIDATOR ] JPM: Primary data validation passed. [2025-10-20T19:30:30.482300] [INFO ] [MAIN ] JPM: Fetched primary data is valid. [2025-10-20T19:30:30.482364] [DEBUG] [LOCAL ] JPM: Saving primary data. [2025-10-20T19:30:30.483864] [DEBUG] [LOCAL ] JPM: Trimmed 77973 records to 53794 (keeping 200 days of data) [2025-10-20T19:30:30.598869] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:30:30.695696] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-08. [2025-10-20T19:30:30.779819] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T19:30:30.868741] [DEBUG] [LOCAL ] JPM: Primary data saved (53794 records). [2025-10-20T19:30:30.868901] [INFO ] [MAIN ] JPM: Full history saved. [2025-10-20T19:30:30.868982] [INFO ] [MAIN ] Update Progress: 1166/2367 symbols completed. JPM took 40.2s. Estimated remaining: 2h 34m 57s. Rate: 7.8 symbols/min [2025-10-20T19:30:31.037102] [INFO ] [MAIN ] JXN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:31.077850] [DEBUG] [LOCAL ] JXN: Loaded 45301 existing records from primary storage. [2025-10-20T19:30:31.078179] [INFO ] [MAIN ] JXN: Loaded 45301 existing records. [2025-10-20T19:30:31.255051] [WARN ] [MAIN ] JXN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:31.255253] [INFO ] [MAIN ] JXN: Wiping existing primary data before full refresh. [2025-10-20T19:30:31.255321] [INFO ] [LOCAL ] JXN: Starting deletion of primary data. [2025-10-20T19:30:31.255537] [DEBUG] [LOCAL ] JXN: Deleted primary data file. [2025-10-20T19:30:31.255594] [INFO ] [MAIN ] JXN: Starting full primary refresh. [2025-10-20T19:30:31.461153] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-06. [2025-10-20T19:30:31.611945] [INFO ] [REMOTE ] JXN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:32.115190] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:30:32.251345] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-09. [2025-10-20T19:30:32.420659] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T19:30:33.051373] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-07. [2025-10-20T19:30:33.571228] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T19:30:33.851351] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-10. [2025-10-20T19:30:34.158808] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T19:30:34.651436] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-08. [2025-10-20T19:30:35.150263] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T19:30:35.156379] [INFO ] [REMOTE ] JQC: Full fetch complete (49835 records). [2025-10-20T19:30:35.156524] [DEBUG] [VALIDATOR ] JQC: Validating primary data. [2025-10-20T19:30:35.457097] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-01. [2025-10-20T19:30:35.471181] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T19:30:35.562021] [DEBUG] [VALIDATOR ] JQC: All expected trading days are present. [2025-10-20T19:30:35.562103] [DEBUG] [VALIDATOR ] JQC: Primary data validation passed. [2025-10-20T19:30:35.562877] [INFO ] [MAIN ] JQC: Fetched primary data is valid. [2025-10-20T19:30:35.562940] [DEBUG] [LOCAL ] JQC: Saving primary data. [2025-10-20T19:30:35.564084] [DEBUG] [LOCAL ] JQC: Trimmed 49835 records to 33824 (keeping 200 days of data) [2025-10-20T19:30:35.760674] [DEBUG] [LOCAL ] JQC: Primary data saved (33824 records). [2025-10-20T19:30:35.760824] [INFO ] [MAIN ] JQC: Full history saved. [2025-10-20T19:30:35.760902] [INFO ] [MAIN ] Update Progress: 1167/2367 symbols completed. JQC took 27.5s. Estimated remaining: 2h 34m 47s. Rate: 7.8 symbols/min [2025-10-20T19:30:35.930724] [INFO ] [MAIN ] K: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:35.979004] [DEBUG] [LOCAL ] K: Loaded 51154 existing records from primary storage. [2025-10-20T19:30:35.979284] [INFO ] [MAIN ] K: Loaded 51154 existing records. [2025-10-20T19:30:36.155405] [WARN ] [MAIN ] K: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:36.155613] [INFO ] [MAIN ] K: Wiping existing primary data before full refresh. [2025-10-20T19:30:36.155704] [INFO ] [LOCAL ] K: Starting deletion of primary data. [2025-10-20T19:30:36.155892] [DEBUG] [LOCAL ] K: Deleted primary data file. [2025-10-20T19:30:36.155931] [INFO ] [MAIN ] K: Starting full primary refresh. [2025-10-20T19:30:36.257316] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-08. [2025-10-20T19:30:36.507753] [INFO ] [REMOTE ] K: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:37.051339] [INFO ] [REMOTE ] K: Fetching primary data for 2025-01. [2025-10-20T19:30:37.113677] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T19:30:37.115896] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T19:30:37.851423] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-09. [2025-10-20T19:30:38.651345] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-09. [2025-10-20T19:30:38.760290] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T19:30:38.853226] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:30:39.451351] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-10. [2025-10-20T19:30:39.504869] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T19:30:40.251398] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-02. [2025-10-20T19:30:40.389506] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:30:40.392461] [INFO ] [REMOTE ] JRI: Full fetch complete (23130 records). [2025-10-20T19:30:40.392542] [DEBUG] [VALIDATOR ] JRI: Validating primary data. [2025-10-20T19:30:40.770545] [DEBUG] [VALIDATOR ] JRI: All expected trading days are present. [2025-10-20T19:30:40.770622] [DEBUG] [VALIDATOR ] JRI: Primary data validation passed. [2025-10-20T19:30:40.771114] [INFO ] [MAIN ] JRI: Fetched primary data is valid. [2025-10-20T19:30:40.771165] [DEBUG] [LOCAL ] JRI: Saving primary data. [2025-10-20T19:30:40.771752] [DEBUG] [LOCAL ] JRI: Trimmed 23130 records to 15423 (keeping 200 days of data) [2025-10-20T19:30:40.863810] [DEBUG] [LOCAL ] JRI: Primary data saved (15423 records). [2025-10-20T19:30:40.863956] [INFO ] [MAIN ] JRI: Full history saved. [2025-10-20T19:30:40.864034] [INFO ] [MAIN ] Update Progress: 1168/2367 symbols completed. JRI took 26.7s. Estimated remaining: 2h 34m 36s. Rate: 7.8 symbols/min [2025-10-20T19:30:41.032774] [INFO ] [MAIN ] KAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:41.047058] [DEBUG] [LOCAL ] KAI: Loaded 13783 existing records from primary storage. [2025-10-20T19:30:41.047248] [INFO ] [MAIN ] KAI: Loaded 13783 existing records. [2025-10-20T19:30:41.058281] [INFO ] [REMOTE ] K: Fetching primary data for 2025-02. [2025-10-20T19:30:41.224289] [WARN ] [MAIN ] KAI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:41.224529] [INFO ] [MAIN ] KAI: Wiping existing primary data before full refresh. [2025-10-20T19:30:41.224583] [INFO ] [LOCAL ] KAI: Starting deletion of primary data. [2025-10-20T19:30:41.224835] [DEBUG] [LOCAL ] KAI: Deleted primary data file. [2025-10-20T19:30:41.224892] [INFO ] [MAIN ] KAI: Starting full primary refresh. [2025-10-20T19:30:41.581547] [INFO ] [REMOTE ] KAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:41.606505] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T19:30:41.851375] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-01. [2025-10-20T19:30:42.625845] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T19:30:42.651341] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-10. [2025-10-20T19:30:42.703133] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:30:43.451357] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-02. [2025-10-20T19:30:43.524715] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T19:30:43.527047] [INFO ] [REMOTE ] JRS: Full fetch complete (16227 records). [2025-10-20T19:30:43.527120] [DEBUG] [VALIDATOR ] JRS: Validating primary data. [2025-10-20T19:30:43.877787] [DEBUG] [VALIDATOR ] JRS: All expected trading days are present. [2025-10-20T19:30:43.877907] [DEBUG] [VALIDATOR ] JRS: Primary data validation passed. [2025-10-20T19:30:43.878251] [INFO ] [MAIN ] JRS: Fetched primary data is valid. [2025-10-20T19:30:43.878290] [DEBUG] [LOCAL ] JRS: Saving primary data. [2025-10-20T19:30:43.878877] [DEBUG] [LOCAL ] JRS: Trimmed 16227 records to 11623 (keeping 200 days of data) [2025-10-20T19:30:43.943455] [DEBUG] [LOCAL ] JRS: Primary data saved (11623 records). [2025-10-20T19:30:43.943578] [INFO ] [MAIN ] JRS: Full history saved. [2025-10-20T19:30:43.943673] [INFO ] [MAIN ] Update Progress: 1169/2367 symbols completed. JRS took 28.3s. Estimated remaining: 2h 34m 24s. Rate: 7.8 symbols/min [2025-10-20T19:30:44.112819] [INFO ] [MAIN ] KAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:44.149234] [DEBUG] [LOCAL ] KAR: Loaded 40601 existing records from primary storage. [2025-10-20T19:30:44.149508] [INFO ] [MAIN ] KAR: Loaded 40601 existing records. [2025-10-20T19:30:44.256575] [INFO ] [REMOTE ] K: Fetching primary data for 2025-03. [2025-10-20T19:30:44.329777] [WARN ] [MAIN ] KAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:44.330011] [INFO ] [MAIN ] KAR: Wiping existing primary data before full refresh. [2025-10-20T19:30:44.330205] [INFO ] [LOCAL ] KAR: Starting deletion of primary data. [2025-10-20T19:30:44.330388] [DEBUG] [LOCAL ] KAR: Deleted primary data file. [2025-10-20T19:30:44.330427] [INFO ] [MAIN ] KAR: Starting full primary refresh. [2025-10-20T19:30:44.513416] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T19:30:44.693637] [INFO ] [REMOTE ] KAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:45.051350] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-03. [2025-10-20T19:30:45.851416] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-01. [2025-10-20T19:30:45.910326] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:30:46.354151] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:30:46.651433] [INFO ] [REMOTE ] K: Fetching primary data for 2025-04. [2025-10-20T19:30:46.735415] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T19:30:47.451408] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-04. [2025-10-20T19:30:48.251356] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-02. [2025-10-20T19:30:48.385832] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T19:30:49.051411] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-03. [2025-10-20T19:30:49.095512] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:30:49.225352] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T19:30:49.851353] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-03. [2025-10-20T19:30:50.029542] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T19:30:50.651383] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-05. [2025-10-20T19:30:51.350891] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:30:51.451365] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-04. [2025-10-20T19:30:52.251448] [INFO ] [REMOTE ] K: Fetching primary data for 2025-05. [2025-10-20T19:30:52.301760] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:30:53.043385] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T19:30:53.051544] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-06. [2025-10-20T19:30:53.851369] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-05. [2025-10-20T19:30:53.984532] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T19:30:54.650187] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:30:54.658942] [INFO ] [REMOTE ] K: Fetching primary data for 2025-06. [2025-10-20T19:30:55.380190] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:30:55.451377] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-06. [2025-10-20T19:30:56.140084] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T19:30:56.251388] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-07. [2025-10-20T19:30:56.833607] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:30:57.051413] [INFO ] [REMOTE ] K: Fetching primary data for 2025-07. [2025-10-20T19:30:57.748051] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:30:57.851379] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-07. [2025-10-20T19:30:58.651441] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-08. [2025-10-20T19:30:58.810208] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:30:59.375912] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T19:30:59.451368] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-04. [2025-10-20T19:31:00.251348] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-08. [2025-10-20T19:31:00.311422] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:31:00.487894] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T19:31:01.051407] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-09. [2025-10-20T19:31:01.804819] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T19:31:01.851340] [INFO ] [REMOTE ] K: Fetching primary data for 2025-08. [2025-10-20T19:31:02.138209] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T19:31:02.651422] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-05. [2025-10-20T19:31:03.435767] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:31:03.451330] [INFO ] [REMOTE ] K: Fetching primary data for 2025-09. [2025-10-20T19:31:03.656899] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T19:31:04.251358] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-09. [2025-10-20T19:31:05.051404] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-06. [2025-10-20T19:31:05.369834] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T19:31:05.851349] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-10. [2025-10-20T19:31:05.962169] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:31:06.225955] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T19:31:06.651390] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-10. [2025-10-20T19:31:07.451381] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-07. [2025-10-20T19:31:07.591822] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:31:07.597825] [INFO ] [REMOTE ] JXN: Full fetch complete (65622 records). [2025-10-20T19:31:07.597971] [DEBUG] [VALIDATOR ] JXN: Validating primary data. [2025-10-20T19:31:08.004160] [DEBUG] [VALIDATOR ] JXN: All expected trading days are present. [2025-10-20T19:31:08.004255] [DEBUG] [VALIDATOR ] JXN: Primary data validation passed. [2025-10-20T19:31:08.005430] [INFO ] [MAIN ] JXN: Fetched primary data is valid. [2025-10-20T19:31:08.005496] [DEBUG] [LOCAL ] JXN: Saving primary data. [2025-10-20T19:31:08.007279] [DEBUG] [LOCAL ] JXN: Trimmed 65622 records to 44551 (keeping 200 days of data) [2025-10-20T19:31:08.242890] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T19:31:08.262855] [INFO ] [REMOTE ] K: Fetching primary data for 2025-10. [2025-10-20T19:31:08.272793] [INFO ] [REMOTE ] KAR: Full fetch complete (56954 records). [2025-10-20T19:31:08.273980] [DEBUG] [VALIDATOR ] KAR: Validating primary data. [2025-10-20T19:31:08.575113] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T19:31:08.642670] [DEBUG] [LOCAL ] JXN: Primary data saved (44551 records). [2025-10-20T19:31:08.645590] [INFO ] [MAIN ] JXN: Full history saved. [2025-10-20T19:31:08.645755] [INFO ] [MAIN ] Update Progress: 1170/2367 symbols completed. JXN took 37.8s. Estimated remaining: 2h 34m 33s. Rate: 7.7 symbols/min [2025-10-20T19:31:08.892154] [DEBUG] [VALIDATOR ] KAR: All expected trading days are present. [2025-10-20T19:31:08.897759] [DEBUG] [VALIDATOR ] KAR: Primary data validation passed. [2025-10-20T19:31:08.900030] [INFO ] [MAIN ] KAR: Fetched primary data is valid. [2025-10-20T19:31:08.901153] [DEBUG] [LOCAL ] KAR: Saving primary data. [2025-10-20T19:31:08.918947] [INFO ] [MAIN ] KB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:08.920489] [DEBUG] [LOCAL ] KAR: Trimmed 56954 records to 39864 (keeping 200 days of data) [2025-10-20T19:31:09.051405] [DEBUG] [LOCAL ] KB: Loaded 23613 existing records from primary storage. [2025-10-20T19:31:09.051481] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-08. [2025-10-20T19:31:09.058344] [INFO ] [MAIN ] KB: Loaded 23613 existing records. [2025-10-20T19:31:09.337295] [WARN ] [MAIN ] KB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:09.338293] [INFO ] [MAIN ] KB: Wiping existing primary data before full refresh. [2025-10-20T19:31:09.338685] [INFO ] [LOCAL ] KB: Starting deletion of primary data. [2025-10-20T19:31:09.341616] [DEBUG] [LOCAL ] KB: Deleted primary data file. [2025-10-20T19:31:09.341999] [INFO ] [MAIN ] KB: Starting full primary refresh. [2025-10-20T19:31:09.658703] [DEBUG] [LOCAL ] KAR: Primary data saved (39864 records). [2025-10-20T19:31:09.669135] [INFO ] [MAIN ] KAR: Full history saved. [2025-10-20T19:31:09.692809] [INFO ] [MAIN ] Update Progress: 1171/2367 symbols completed. KAR took 25.7s. Estimated remaining: 2h 34m 19s. Rate: 7.8 symbols/min [2025-10-20T19:31:09.803084] [INFO ] [REMOTE ] KB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:09.856440] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-01. [2025-10-20T19:31:09.916075] [INFO ] [MAIN ] KBDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:09.924791] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T19:31:09.946753] [DEBUG] [LOCAL ] KBDC: Loaded 22794 existing records from primary storage. [2025-10-20T19:31:09.947022] [INFO ] [MAIN ] KBDC: Loaded 22794 existing records. [2025-10-20T19:31:10.157327] [WARN ] [MAIN ] KBDC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:10.157582] [INFO ] [MAIN ] KBDC: Wiping existing primary data before full refresh. [2025-10-20T19:31:10.157684] [INFO ] [LOCAL ] KBDC: Starting deletion of primary data. [2025-10-20T19:31:10.158060] [DEBUG] [LOCAL ] KBDC: Deleted primary data file. [2025-10-20T19:31:10.158166] [INFO ] [MAIN ] KBDC: Starting full primary refresh. [2025-10-20T19:31:10.348574] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T19:31:10.387877] [INFO ] [REMOTE ] K: Full fetch complete (72302 records). [2025-10-20T19:31:10.435210] [DEBUG] [VALIDATOR ] K: Validating primary data. [2025-10-20T19:31:10.655236] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-09. [2025-10-20T19:31:10.708064] [INFO ] [REMOTE ] KBDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:10.986807] [DEBUG] [VALIDATOR ] K: All expected trading days are present. [2025-10-20T19:31:10.986972] [DEBUG] [VALIDATOR ] K: Primary data validation passed. [2025-10-20T19:31:10.988610] [INFO ] [MAIN ] K: Fetched primary data is valid. [2025-10-20T19:31:10.989240] [DEBUG] [LOCAL ] K: Saving primary data. [2025-10-20T19:31:10.989159] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T19:31:10.995065] [DEBUG] [LOCAL ] K: Trimmed 72302 records to 50372 (keeping 200 days of data) [2025-10-20T19:31:11.298428] [DEBUG] [LOCAL ] K: Primary data saved (50372 records). [2025-10-20T19:31:11.298570] [INFO ] [MAIN ] K: Full history saved. [2025-10-20T19:31:11.299298] [INFO ] [MAIN ] Update Progress: 1172/2367 symbols completed. K took 35.5s. Estimated remaining: 2h 34m 5s. Rate: 7.8 symbols/min [2025-10-20T19:31:11.464315] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-02. [2025-10-20T19:31:11.467040] [INFO ] [MAIN ] KBH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:11.513963] [DEBUG] [LOCAL ] KBH: Loaded 49532 existing records from primary storage. [2025-10-20T19:31:11.514263] [INFO ] [MAIN ] KBH: Loaded 49532 existing records. [2025-10-20T19:31:11.628214] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:31:11.701272] [WARN ] [MAIN ] KBH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:11.701478] [INFO ] [MAIN ] KBH: Wiping existing primary data before full refresh. [2025-10-20T19:31:11.701524] [INFO ] [LOCAL ] KBH: Starting deletion of primary data. [2025-10-20T19:31:11.701997] [DEBUG] [LOCAL ] KBH: Deleted primary data file. [2025-10-20T19:31:11.702053] [INFO ] [MAIN ] KBH: Starting full primary refresh. [2025-10-20T19:31:12.046105] [INFO ] [REMOTE ] KBH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:12.251337] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-01. [2025-10-20T19:31:12.512196] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T19:31:13.051395] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-10. [2025-10-20T19:31:13.851369] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-01. [2025-10-20T19:31:14.071862] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T19:31:14.120607] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T19:31:14.123237] [INFO ] [REMOTE ] KAI: Full fetch complete (19280 records). [2025-10-20T19:31:14.123308] [DEBUG] [VALIDATOR ] KAI: Validating primary data. [2025-10-20T19:31:14.480680] [DEBUG] [VALIDATOR ] KAI: All expected trading days are present. [2025-10-20T19:31:14.480766] [DEBUG] [VALIDATOR ] KAI: Primary data validation passed. [2025-10-20T19:31:14.481190] [INFO ] [MAIN ] KAI: Fetched primary data is valid. [2025-10-20T19:31:14.481234] [DEBUG] [LOCAL ] KAI: Saving primary data. [2025-10-20T19:31:14.481784] [DEBUG] [LOCAL ] KAI: Trimmed 19280 records to 13606 (keeping 200 days of data) [2025-10-20T19:31:14.558917] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T19:31:14.582864] [DEBUG] [LOCAL ] KAI: Primary data saved (13606 records). [2025-10-20T19:31:14.582999] [INFO ] [MAIN ] KAI: Full history saved. [2025-10-20T19:31:14.583075] [INFO ] [MAIN ] Update Progress: 1173/2367 symbols completed. KAI took 33.7s. Estimated remaining: 2h 33m 53s. Rate: 7.8 symbols/min [2025-10-20T19:31:14.656486] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-02. [2025-10-20T19:31:14.777475] [INFO ] [MAIN ] KBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:14.822257] [DEBUG] [LOCAL ] KBR: Loaded 48737 existing records from primary storage. [2025-10-20T19:31:14.822566] [INFO ] [MAIN ] KBR: Loaded 48737 existing records. [2025-10-20T19:31:15.000570] [WARN ] [MAIN ] KBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:15.000830] [INFO ] [MAIN ] KBR: Wiping existing primary data before full refresh. [2025-10-20T19:31:15.000877] [INFO ] [LOCAL ] KBR: Starting deletion of primary data. [2025-10-20T19:31:15.001064] [DEBUG] [LOCAL ] KBR: Deleted primary data file. [2025-10-20T19:31:15.001102] [INFO ] [MAIN ] KBR: Starting full primary refresh. [2025-10-20T19:31:15.364594] [INFO ] [REMOTE ] KBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:15.451411] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-03. [2025-10-20T19:31:15.657332] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T19:31:16.251339] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-03. [2025-10-20T19:31:16.423797] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T19:31:17.051429] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-02. [2025-10-20T19:31:17.321829] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T19:31:17.851465] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-04. [2025-10-20T19:31:18.506498] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:31:18.651354] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-03. [2025-10-20T19:31:18.932594] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T19:31:19.451422] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-05. [2025-10-20T19:31:20.251390] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-01. [2025-10-20T19:31:20.318959] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:31:20.504137] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T19:31:21.051439] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-04. [2025-10-20T19:31:21.851430] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-06. [2025-10-20T19:31:21.885261] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:31:22.663290] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-02. [2025-10-20T19:31:22.668729] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:31:22.836177] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:31:23.451425] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-04. [2025-10-20T19:31:24.251391] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-05. [2025-10-20T19:31:24.307100] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:31:24.596593] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:31:25.051392] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-07. [2025-10-20T19:31:25.712597] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T19:31:25.851415] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-06. [2025-10-20T19:31:26.110298] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T19:31:26.651449] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-05. [2025-10-20T19:31:27.408227] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:31:27.451359] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-07. [2025-10-20T19:31:27.638590] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T19:31:28.251408] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-08. [2025-10-20T19:31:29.051406] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-06. [2025-10-20T19:31:29.223965] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:31:29.292286] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T19:31:29.833187] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T19:31:29.851349] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-07. [2025-10-20T19:31:30.651435] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-03. [2025-10-20T19:31:31.180720] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T19:31:31.451374] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-08. [2025-10-20T19:31:32.129797] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T19:31:32.251370] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-09. [2025-10-20T19:31:33.051364] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-08. [2025-10-20T19:31:33.118285] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T19:31:33.509451] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T19:31:33.851347] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-04. [2025-10-20T19:31:34.123025] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T19:31:34.651449] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-10. [2025-10-20T19:31:35.316074] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T19:31:35.451362] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-09. [2025-10-20T19:31:35.961342] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:31:35.966347] [INFO ] [REMOTE ] KBDC: Full fetch complete (30743 records). [2025-10-20T19:31:35.966430] [DEBUG] [VALIDATOR ] KBDC: Validating primary data. [2025-10-20T19:31:36.256529] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-09. [2025-10-20T19:31:36.335105] [DEBUG] [VALIDATOR ] KBDC: All expected trading days are present. [2025-10-20T19:31:36.335212] [DEBUG] [VALIDATOR ] KBDC: Primary data validation passed. [2025-10-20T19:31:36.335924] [INFO ] [MAIN ] KBDC: Fetched primary data is valid. [2025-10-20T19:31:36.335984] [DEBUG] [LOCAL ] KBDC: Saving primary data. [2025-10-20T19:31:36.336694] [DEBUG] [LOCAL ] KBDC: Trimmed 30743 records to 22651 (keeping 200 days of data) [2025-10-20T19:31:36.472541] [DEBUG] [LOCAL ] KBDC: Primary data saved (22651 records). [2025-10-20T19:31:36.472702] [INFO ] [MAIN ] KBDC: Full history saved. [2025-10-20T19:31:36.472797] [INFO ] [MAIN ] Update Progress: 1174/2367 symbols completed. KBDC took 26.8s. Estimated remaining: 2h 33m 59s. Rate: 7.7 symbols/min [2025-10-20T19:31:36.647235] [INFO ] [MAIN ] KD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:36.695042] [DEBUG] [LOCAL ] KD: Loaded 53125 existing records from primary storage. [2025-10-20T19:31:36.695349] [INFO ] [MAIN ] KD: Loaded 53125 existing records. [2025-10-20T19:31:36.876931] [WARN ] [MAIN ] KD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:36.877147] [INFO ] [MAIN ] KD: Wiping existing primary data before full refresh. [2025-10-20T19:31:36.877206] [INFO ] [LOCAL ] KD: Starting deletion of primary data. [2025-10-20T19:31:36.877381] [DEBUG] [LOCAL ] KD: Deleted primary data file. [2025-10-20T19:31:36.877420] [INFO ] [MAIN ] KD: Starting full primary refresh. [2025-10-20T19:31:37.059090] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-05. [2025-10-20T19:31:37.225584] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:31:37.277633] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:31:37.346550] [INFO ] [REMOTE ] KD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:37.851401] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-10. [2025-10-20T19:31:38.651399] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-01. [2025-10-20T19:31:38.734785] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T19:31:39.451357] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-10. [2025-10-20T19:31:39.615923] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T19:31:39.623134] [INFO ] [REMOTE ] KBH: Full fetch complete (71099 records). [2025-10-20T19:31:39.623265] [DEBUG] [VALIDATOR ] KBH: Validating primary data. [2025-10-20T19:31:40.013937] [DEBUG] [VALIDATOR ] KBH: All expected trading days are present. [2025-10-20T19:31:40.014022] [DEBUG] [VALIDATOR ] KBH: Primary data validation passed. [2025-10-20T19:31:40.014978] [INFO ] [MAIN ] KBH: Fetched primary data is valid. [2025-10-20T19:31:40.015042] [DEBUG] [LOCAL ] KBH: Saving primary data. [2025-10-20T19:31:40.016385] [DEBUG] [LOCAL ] KBH: Trimmed 71099 records to 48759 (keeping 200 days of data) [2025-10-20T19:31:40.253696] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-06. [2025-10-20T19:31:40.315476] [DEBUG] [LOCAL ] KBH: Primary data saved (48759 records). [2025-10-20T19:31:40.315639] [INFO ] [MAIN ] KBH: Full history saved. [2025-10-20T19:31:40.316601] [INFO ] [MAIN ] Update Progress: 1175/2367 symbols completed. KBH took 29.0s. Estimated remaining: 2h 33m 48s. Rate: 7.8 symbols/min [2025-10-20T19:31:40.511029] [INFO ] [MAIN ] KEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:40.543571] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T19:31:40.547923] [DEBUG] [LOCAL ] KEN: Loaded 4731 existing records from primary storage. [2025-10-20T19:31:40.558900] [INFO ] [MAIN ] KEN: Loaded 4731 existing records. [2025-10-20T19:31:40.740857] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T19:31:40.780698] [WARN ] [MAIN ] KEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:40.780854] [INFO ] [REMOTE ] KB: Full fetch complete (33610 records). [2025-10-20T19:31:40.781183] [INFO ] [MAIN ] KEN: Wiping existing primary data before full refresh. [2025-10-20T19:31:40.781058] [DEBUG] [VALIDATOR ] KB: Validating primary data. [2025-10-20T19:31:40.781308] [INFO ] [LOCAL ] KEN: Starting deletion of primary data. [2025-10-20T19:31:40.812938] [DEBUG] [LOCAL ] KEN: Deleted primary data file. [2025-10-20T19:31:40.813029] [INFO ] [MAIN ] KEN: Starting full primary refresh. [2025-10-20T19:31:41.052704] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-02. [2025-10-20T19:31:41.533354] [INFO ] [REMOTE ] KEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:41.548036] [DEBUG] [VALIDATOR ] KB: All expected trading days are present. [2025-10-20T19:31:41.548414] [DEBUG] [VALIDATOR ] KB: Primary data validation passed. [2025-10-20T19:31:41.549116] [INFO ] [MAIN ] KB: Fetched primary data is valid. [2025-10-20T19:31:41.549177] [DEBUG] [LOCAL ] KB: Saving primary data. [2025-10-20T19:31:41.550065] [DEBUG] [LOCAL ] KB: Trimmed 33610 records to 23270 (keeping 200 days of data) [2025-10-20T19:31:41.687695] [DEBUG] [LOCAL ] KB: Primary data saved (23270 records). [2025-10-20T19:31:41.687909] [INFO ] [MAIN ] KB: Full history saved. [2025-10-20T19:31:41.688148] [INFO ] [MAIN ] Update Progress: 1176/2367 symbols completed. KB took 33.0s. Estimated remaining: 2h 33m 33s. Rate: 7.8 symbols/min [2025-10-20T19:31:41.856203] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-01. [2025-10-20T19:31:41.858910] [INFO ] [MAIN ] KEP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:41.888634] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T19:31:41.924599] [DEBUG] [LOCAL ] KEP: Loaded 32548 existing records from primary storage. [2025-10-20T19:31:41.924900] [INFO ] [MAIN ] KEP: Loaded 32548 existing records. [2025-10-20T19:31:42.098172] [WARN ] [MAIN ] KEP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:42.098412] [INFO ] [MAIN ] KEP: Wiping existing primary data before full refresh. [2025-10-20T19:31:42.098459] [INFO ] [LOCAL ] KEP: Starting deletion of primary data. [2025-10-20T19:31:42.098696] [DEBUG] [LOCAL ] KEP: Deleted primary data file. [2025-10-20T19:31:42.098754] [INFO ] [MAIN ] KEP: Starting full primary refresh. [2025-10-20T19:31:42.443069] [INFO ] [REMOTE ] KEP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:42.651353] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-01. [2025-10-20T19:31:42.830274] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T19:31:43.451362] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-02. [2025-10-20T19:31:44.070149] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-02 (3.02 seconds). [2025-10-20T19:31:44.251489] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-03. [2025-10-20T19:31:44.716708] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:31:45.051403] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-03. [2025-10-20T19:31:45.613987] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T19:31:45.666162] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-01 (3.01 seconds). [2025-10-20T19:31:45.851391] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-07. [2025-10-20T19:31:46.304522] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T19:31:46.651406] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-04. [2025-10-20T19:31:47.218734] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T19:31:47.451372] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-02. [2025-10-20T19:31:47.626943] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T19:31:48.251351] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-08. [2025-10-20T19:31:48.488696] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T19:31:49.051372] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-03. [2025-10-20T19:31:49.856536] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-05. [2025-10-20T19:31:49.862883] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T19:31:50.105101] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:31:50.505380] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T19:31:50.651374] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-04. [2025-10-20T19:31:51.451391] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-09. [2025-10-20T19:31:52.256513] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-06. [2025-10-20T19:31:52.276304] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:31:52.896716] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T19:31:52.938577] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:31:53.051368] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-07. [2025-10-20T19:31:53.699137] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T19:31:53.851369] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-10. [2025-10-20T19:31:54.651424] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-04. [2025-10-20T19:31:55.451438] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-08. [2025-10-20T19:31:55.729624] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T19:31:55.740831] [INFO ] [REMOTE ] KBR: Full fetch complete (69946 records). [2025-10-20T19:31:55.741015] [DEBUG] [VALIDATOR ] KBR: Validating primary data. [2025-10-20T19:31:56.060355] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:31:56.060497] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:31:56.184344] [DEBUG] [VALIDATOR ] KBR: All expected trading days are present. [2025-10-20T19:31:56.184431] [DEBUG] [VALIDATOR ] KBR: Primary data validation passed. [2025-10-20T19:31:56.185365] [INFO ] [MAIN ] KBR: Fetched primary data is valid. [2025-10-20T19:31:56.185428] [DEBUG] [LOCAL ] KBR: Saving primary data. [2025-10-20T19:31:56.186790] [DEBUG] [LOCAL ] KBR: Trimmed 69946 records to 48051 (keeping 200 days of data) [2025-10-20T19:31:56.275302] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-05. [2025-10-20T19:31:56.486315] [DEBUG] [LOCAL ] KBR: Primary data saved (48051 records). [2025-10-20T19:31:56.486457] [INFO ] [MAIN ] KBR: Full history saved. [2025-10-20T19:31:56.486535] [INFO ] [MAIN ] Update Progress: 1177/2367 symbols completed. KBR took 41.9s. Estimated remaining: 2h 33m 33s. Rate: 7.8 symbols/min [2025-10-20T19:31:56.662274] [INFO ] [MAIN ] KEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:56.699480] [DEBUG] [LOCAL ] KEX: Loaded 39285 existing records from primary storage. [2025-10-20T19:31:56.700000] [INFO ] [MAIN ] KEX: Loaded 39285 existing records. [2025-10-20T19:31:56.889660] [WARN ] [MAIN ] KEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:56.890993] [INFO ] [MAIN ] KEX: Wiping existing primary data before full refresh. [2025-10-20T19:31:56.891079] [INFO ] [LOCAL ] KEX: Starting deletion of primary data. [2025-10-20T19:31:56.891849] [DEBUG] [LOCAL ] KEX: Deleted primary data file. [2025-10-20T19:31:56.891911] [INFO ] [MAIN ] KEX: Starting full primary refresh. [2025-10-20T19:31:57.052676] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-09. [2025-10-20T19:31:57.264230] [INFO ] [REMOTE ] KEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:57.856487] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-01. [2025-10-20T19:31:57.883528] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:31:57.901561] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T19:31:58.651434] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-06. [2025-10-20T19:31:59.451474] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-10. [2025-10-20T19:31:59.484355] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:32:00.083991] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T19:32:00.085487] [INFO ] [REMOTE ] KEN: Full fetch complete (5938 records). [2025-10-20T19:32:00.085691] [DEBUG] [VALIDATOR ] KEN: Validating primary data. [2025-10-20T19:32:00.258583] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-05. [2025-10-20T19:32:00.477108] [DEBUG] [VALIDATOR ] KEN: All expected trading days are present. [2025-10-20T19:32:00.477225] [DEBUG] [VALIDATOR ] KEN: Primary data validation passed. [2025-10-20T19:32:00.477462] [INFO ] [MAIN ] KEN: Fetched primary data is valid. [2025-10-20T19:32:00.477570] [DEBUG] [LOCAL ] KEN: Saving primary data. [2025-10-20T19:32:00.478108] [DEBUG] [LOCAL ] KEN: Trimmed 5938 records to 4695 (keeping 200 days of data) [2025-10-20T19:32:00.536148] [DEBUG] [LOCAL ] KEN: Primary data saved (4695 records). [2025-10-20T19:32:00.536333] [INFO ] [MAIN ] KEN: Full history saved. [2025-10-20T19:32:00.536471] [INFO ] [MAIN ] Update Progress: 1178/2367 symbols completed. KEN took 20.2s. Estimated remaining: 2h 33m 21s. Rate: 7.8 symbols/min [2025-10-20T19:32:00.597922] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T19:32:00.749699] [INFO ] [MAIN ] KEY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:00.749886] [DEBUG] [LOCAL ] KEY: No existing data file found in primary storage. [2025-10-20T19:32:00.749923] [WARN ] [MAIN ] KEY: No existing data found, performing full refresh instead. [2025-10-20T19:32:00.750035] [INFO ] [MAIN ] KEY: Wiping existing primary data before full refresh. [2025-10-20T19:32:00.750076] [INFO ] [LOCAL ] KEY: Starting deletion of primary data. [2025-10-20T19:32:00.750116] [DEBUG] [LOCAL ] KEY: No primary data file found to delete. [2025-10-20T19:32:00.750147] [INFO ] [MAIN ] KEY: Starting full primary refresh. [2025-10-20T19:32:01.056482] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-07. [2025-10-20T19:32:01.114242] [INFO ] [REMOTE ] KEY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:01.574748] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T19:32:01.851370] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-01. [2025-10-20T19:32:02.651347] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-02. [2025-10-20T19:32:03.076690] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T19:32:03.451350] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-06. [2025-10-20T19:32:03.622673] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T19:32:03.959606] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T19:32:04.251384] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-03. [2025-10-20T19:32:05.003733] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T19:32:05.051380] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-07. [2025-10-20T19:32:05.694927] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:32:05.851387] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-04. [2025-10-20T19:32:06.392835] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T19:32:06.651393] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-02. [2025-10-20T19:32:07.339772] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T19:32:07.451441] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-05. [2025-10-20T19:32:08.251425] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-08. [2025-10-20T19:32:08.304885] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:32:08.872205] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T19:32:09.051372] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-06. [2025-10-20T19:32:09.851430] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-03. [2025-10-20T19:32:10.060458] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T19:32:10.175413] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T19:32:10.651415] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-07. [2025-10-20T19:32:11.451382] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-09. [2025-10-20T19:32:11.606075] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:32:12.082996] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T19:32:12.251365] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-08. [2025-10-20T19:32:13.041563] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:32:13.051963] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-08. [2025-10-20T19:32:13.665037] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T19:32:13.851423] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-09. [2025-10-20T19:32:14.426183] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T19:32:14.651395] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-10. [2025-10-20T19:32:15.279104] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T19:32:15.451415] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-04. [2025-10-20T19:32:16.251352] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-10. [2025-10-20T19:32:16.648557] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T19:32:16.657349] [INFO ] [REMOTE ] KD: Full fetch complete (76127 records). [2025-10-20T19:32:16.657545] [DEBUG] [VALIDATOR ] KD: Validating primary data. [2025-10-20T19:32:17.057027] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-09. [2025-10-20T19:32:17.082784] [DEBUG] [VALIDATOR ] KD: All expected trading days are present. [2025-10-20T19:32:17.082893] [DEBUG] [VALIDATOR ] KD: Primary data validation passed. [2025-10-20T19:32:17.084107] [INFO ] [MAIN ] KD: Fetched primary data is valid. [2025-10-20T19:32:17.084419] [DEBUG] [LOCAL ] KD: Saving primary data. [2025-10-20T19:32:17.086231] [DEBUG] [LOCAL ] KD: Trimmed 76127 records to 52354 (keeping 200 days of data) [2025-10-20T19:32:17.236479] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T19:32:17.434876] [DEBUG] [LOCAL ] KD: Primary data saved (52354 records). [2025-10-20T19:32:17.435011] [INFO ] [MAIN ] KD: Full history saved. [2025-10-20T19:32:17.435096] [INFO ] [MAIN ] Update Progress: 1179/2367 symbols completed. KD took 41.0s. Estimated remaining: 2h 33m 23s. Rate: 7.7 symbols/min [2025-10-20T19:32:17.608297] [INFO ] [MAIN ] KEYS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:17.608449] [DEBUG] [LOCAL ] KEYS: No existing data file found in primary storage. [2025-10-20T19:32:17.608502] [WARN ] [MAIN ] KEYS: No existing data found, performing full refresh instead. [2025-10-20T19:32:17.608607] [INFO ] [MAIN ] KEYS: Wiping existing primary data before full refresh. [2025-10-20T19:32:17.608702] [INFO ] [LOCAL ] KEYS: Starting deletion of primary data. [2025-10-20T19:32:17.608761] [DEBUG] [LOCAL ] KEYS: No primary data file found to delete. [2025-10-20T19:32:17.608795] [INFO ] [MAIN ] KEYS: Starting full primary refresh. [2025-10-20T19:32:17.856604] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-05. [2025-10-20T19:32:17.978112] [INFO ] [REMOTE ] KEYS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:18.066021] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T19:32:18.073604] [INFO ] [REMOTE ] KEX: Full fetch complete (56087 records). [2025-10-20T19:32:18.073840] [DEBUG] [VALIDATOR ] KEX: Validating primary data. [2025-10-20T19:32:18.199246] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T19:32:18.494330] [DEBUG] [VALIDATOR ] KEX: All expected trading days are present. [2025-10-20T19:32:18.494411] [DEBUG] [VALIDATOR ] KEX: Primary data validation passed. [2025-10-20T19:32:18.495229] [INFO ] [MAIN ] KEX: Fetched primary data is valid. [2025-10-20T19:32:18.495292] [DEBUG] [LOCAL ] KEX: Saving primary data. [2025-10-20T19:32:18.496377] [DEBUG] [LOCAL ] KEX: Trimmed 56087 records to 38824 (keeping 200 days of data) [2025-10-20T19:32:18.663571] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-01. [2025-10-20T19:32:18.735247] [DEBUG] [LOCAL ] KEX: Primary data saved (38824 records). [2025-10-20T19:32:18.735371] [INFO ] [MAIN ] KEX: Full history saved. [2025-10-20T19:32:18.735444] [INFO ] [MAIN ] Update Progress: 1180/2367 symbols completed. KEX took 22.2s. Estimated remaining: 2h 33m 9s. Rate: 7.8 symbols/min [2025-10-20T19:32:18.913040] [INFO ] [MAIN ] KF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:18.918157] [DEBUG] [LOCAL ] KF: Loaded 2385 existing records from primary storage. [2025-10-20T19:32:18.918501] [INFO ] [MAIN ] KF: Loaded 2385 existing records. [2025-10-20T19:32:19.111749] [WARN ] [MAIN ] KF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:19.111962] [INFO ] [MAIN ] KF: Wiping existing primary data before full refresh. [2025-10-20T19:32:19.112017] [INFO ] [LOCAL ] KF: Starting deletion of primary data. [2025-10-20T19:32:19.112191] [DEBUG] [LOCAL ] KF: Deleted primary data file. [2025-10-20T19:32:19.112247] [INFO ] [MAIN ] KF: Starting full primary refresh. [2025-10-20T19:32:19.462026] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-10. [2025-10-20T19:32:19.504146] [INFO ] [REMOTE ] KF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:19.742006] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T19:32:20.232119] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T19:32:20.251382] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-01. [2025-10-20T19:32:20.610720] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T19:32:20.619158] [INFO ] [REMOTE ] KEP: Full fetch complete (42358 records). [2025-10-20T19:32:20.619316] [DEBUG] [VALIDATOR ] KEP: Validating primary data. [2025-10-20T19:32:20.767277] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T19:32:21.008001] [DEBUG] [VALIDATOR ] KEP: All expected trading days are present. [2025-10-20T19:32:21.008110] [DEBUG] [VALIDATOR ] KEP: Primary data validation passed. [2025-10-20T19:32:21.008872] [INFO ] [MAIN ] KEP: Fetched primary data is valid. [2025-10-20T19:32:21.008934] [DEBUG] [LOCAL ] KEP: Saving primary data. [2025-10-20T19:32:21.009942] [DEBUG] [LOCAL ] KEP: Trimmed 42358 records to 32390 (keeping 200 days of data) [2025-10-20T19:32:21.056936] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-02. [2025-10-20T19:32:21.199265] [DEBUG] [LOCAL ] KEP: Primary data saved (32390 records). [2025-10-20T19:32:21.199405] [INFO ] [MAIN ] KEP: Full history saved. [2025-10-20T19:32:21.199484] [INFO ] [MAIN ] Update Progress: 1181/2367 symbols completed. KEP took 39.5s. Estimated remaining: 2h 32m 55s. Rate: 7.8 symbols/min [2025-10-20T19:32:21.367088] [INFO ] [MAIN ] KFRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:21.387816] [DEBUG] [LOCAL ] KFRC: Loaded 20885 existing records from primary storage. [2025-10-20T19:32:21.388056] [INFO ] [MAIN ] KFRC: Loaded 20885 existing records. [2025-10-20T19:32:21.571335] [WARN ] [MAIN ] KFRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:21.571540] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T19:32:21.573228] [INFO ] [MAIN ] KFRC: Wiping existing primary data before full refresh. [2025-10-20T19:32:21.573301] [INFO ] [LOCAL ] KFRC: Starting deletion of primary data. [2025-10-20T19:32:21.573616] [DEBUG] [LOCAL ] KFRC: Deleted primary data file. [2025-10-20T19:32:21.573698] [INFO ] [MAIN ] KFRC: Starting full primary refresh. [2025-10-20T19:32:21.856531] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-03. [2025-10-20T19:32:21.926476] [INFO ] [REMOTE ] KFRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:22.451492] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T19:32:22.651365] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-02. [2025-10-20T19:32:23.451427] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-04. [2025-10-20T19:32:24.049965] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T19:32:24.251370] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-05. [2025-10-20T19:32:24.302194] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:32:24.801662] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T19:32:25.051358] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-06. [2025-10-20T19:32:25.607327] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T19:32:25.851411] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-06. [2025-10-20T19:32:26.651375] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-07. [2025-10-20T19:32:27.098397] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T19:32:27.451396] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-08. [2025-10-20T19:32:27.672077] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T19:32:27.896936] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:32:28.251388] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-01. [2025-10-20T19:32:29.032622] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T19:32:29.051365] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-02. [2025-10-20T19:32:29.851384] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-03. [2025-10-20T19:32:29.960136] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T19:32:30.651395] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-07. [2025-10-20T19:32:31.451458] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-03. [2025-10-20T19:32:31.580421] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:32:32.251341] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-09. [2025-10-20T19:32:32.326549] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T19:32:32.533992] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-10-20T19:32:32.579874] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T19:32:33.051430] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-04. [2025-10-20T19:32:33.851381] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-08. [2025-10-20T19:32:34.071756] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T19:32:34.651387] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-10. [2025-10-20T19:32:35.229339] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T19:32:35.230257] [INFO ] [REMOTE ] KF: Full fetch complete (3519 records). [2025-10-20T19:32:35.230323] [DEBUG] [VALIDATOR ] KF: Validating primary data. [2025-10-20T19:32:35.459294] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-04. [2025-10-20T19:32:35.593180] [DEBUG] [VALIDATOR ] KF: All expected trading days are present. [2025-10-20T19:32:35.593330] [DEBUG] [VALIDATOR ] KF: Primary data validation passed. [2025-10-20T19:32:35.593482] [INFO ] [MAIN ] KF: Fetched primary data is valid. [2025-10-20T19:32:35.593564] [DEBUG] [LOCAL ] KF: Saving primary data. [2025-10-20T19:32:35.594163] [DEBUG] [LOCAL ] KF: Trimmed 3519 records to 2366 (keeping 200 days of data) [2025-10-20T19:32:35.607754] [DEBUG] [LOCAL ] KF: Primary data saved (2366 records). [2025-10-20T19:32:35.607814] [INFO ] [MAIN ] KF: Full history saved. [2025-10-20T19:32:35.607877] [INFO ] [MAIN ] Update Progress: 1182/2367 symbols completed. KF took 16.9s. Estimated remaining: 2h 32m 54s. Rate: 7.7 symbols/min [2025-10-20T19:32:35.814840] [INFO ] [MAIN ] KFS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:35.817364] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T19:32:35.836381] [DEBUG] [LOCAL ] KFS: Loaded 8837 existing records from primary storage. [2025-10-20T19:32:35.836576] [INFO ] [MAIN ] KFS: Loaded 8837 existing records. [2025-10-20T19:32:36.011384] [WARN ] [MAIN ] KFS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:36.011580] [INFO ] [MAIN ] KFS: Wiping existing primary data before full refresh. [2025-10-20T19:32:36.011677] [INFO ] [LOCAL ] KFS: Starting deletion of primary data. [2025-10-20T19:32:36.011863] [DEBUG] [LOCAL ] KFS: Deleted primary data file. [2025-10-20T19:32:36.011917] [INFO ] [MAIN ] KFS: Starting full primary refresh. [2025-10-20T19:32:36.256564] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-05. [2025-10-20T19:32:36.376478] [INFO ] [REMOTE ] KFS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:37.051415] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-09. [2025-10-20T19:32:37.361415] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T19:32:37.371377] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:32:37.851420] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-06. [2025-10-20T19:32:38.651402] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-01. [2025-10-20T19:32:38.740914] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T19:32:38.929491] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T19:32:39.303609] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T19:32:39.451375] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-07. [2025-10-20T19:32:40.251377] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-02. [2025-10-20T19:32:40.605780] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T19:32:40.817098] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T19:32:41.051379] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-03. [2025-10-20T19:32:41.640449] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T19:32:41.851384] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-10. [2025-10-20T19:32:42.651386] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-05. [2025-10-20T19:32:43.451447] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-04. [2025-10-20T19:32:43.727999] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T19:32:43.737931] [INFO ] [REMOTE ] KEY: Full fetch complete (77427 records). [2025-10-20T19:32:43.738109] [DEBUG] [VALIDATOR ] KEY: Validating primary data. [2025-10-20T19:32:43.973039] [WARN ] [VALIDATOR ] KEY: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:32:43.974141] [FATAL] [MAIN ] KEY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:32:43.974225] [WARN ] [MAIN ] KEY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:32:43.974287] [INFO ] [LOCAL ] KEY: Starting deletion of primary data. [2025-10-20T19:32:43.974365] [DEBUG] [LOCAL ] KEY: No primary data file found to delete. [2025-10-20T19:32:43.974440] [INFO ] [LOCAL ] KEY: Starting deletion of derivative data. [2025-10-20T19:32:43.974520] [DEBUG] [LOCAL ] KEY: No derivative data file found to delete. [2025-10-20T19:32:43.974598] [INFO ] [MAIN ] Update Progress: 1183/2367 symbols completed. KEY took 43.4s. Estimated remaining: 2h 32m 47s. Rate: 7.7 symbols/min [2025-10-20T19:32:44.198841] [INFO ] [MAIN ] KFY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:44.239750] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:32:44.241590] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T19:32:44.254014] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-05. [2025-10-20T19:32:44.292851] [DEBUG] [LOCAL ] KFY: Loaded 30339 existing records from primary storage. [2025-10-20T19:32:44.293302] [INFO ] [MAIN ] KFY: Loaded 30339 existing records. [2025-10-20T19:32:44.477687] [WARN ] [MAIN ] KFY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:44.477917] [INFO ] [MAIN ] KFY: Wiping existing primary data before full refresh. [2025-10-20T19:32:44.477964] [INFO ] [LOCAL ] KFY: Starting deletion of primary data. [2025-10-20T19:32:44.478219] [DEBUG] [LOCAL ] KFY: Deleted primary data file. [2025-10-20T19:32:44.478257] [INFO ] [MAIN ] KFY: Starting full primary refresh. [2025-10-20T19:32:44.834817] [INFO ] [REMOTE ] KFY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:45.017544] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T19:32:45.051416] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-01. [2025-10-20T19:32:45.851352] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-06. [2025-10-20T19:32:46.257152] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T19:32:46.651394] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-02. [2025-10-20T19:32:47.264551] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T19:32:47.451389] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-07. [2025-10-20T19:32:47.937939] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:32:48.251370] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-08. [2025-10-20T19:32:48.950817] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:32:49.051345] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-08. [2025-10-20T19:32:49.363135] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T19:32:49.851370] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-06. [2025-10-20T19:32:50.586046] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:32:50.619253] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:32:50.651342] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-09. [2025-10-20T19:32:51.451381] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-07. [2025-10-20T19:32:52.252472] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-03. [2025-10-20T19:32:52.269222] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T19:32:52.276992] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T19:32:53.051367] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-08. [2025-10-20T19:32:53.711782] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:32:53.802100] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T19:32:53.851365] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-04. [2025-10-20T19:32:54.651371] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-09. [2025-10-20T19:32:55.012741] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T19:32:55.431437] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:32:55.451390] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-10. [2025-10-20T19:32:56.251410] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-10. [2025-10-20T19:32:56.412736] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T19:32:56.414520] [INFO ] [REMOTE ] KFS: Full fetch complete (11131 records). [2025-10-20T19:32:56.414587] [DEBUG] [VALIDATOR ] KFS: Validating primary data. [2025-10-20T19:32:56.782135] [DEBUG] [VALIDATOR ] KFS: All expected trading days are present. [2025-10-20T19:32:56.782211] [DEBUG] [VALIDATOR ] KFS: Primary data validation passed. [2025-10-20T19:32:56.782470] [INFO ] [MAIN ] KFS: Fetched primary data is valid. [2025-10-20T19:32:56.782507] [DEBUG] [LOCAL ] KFS: Saving primary data. [2025-10-20T19:32:56.782974] [DEBUG] [LOCAL ] KFS: Trimmed 11131 records to 8699 (keeping 200 days of data) [2025-10-20T19:32:56.833920] [DEBUG] [LOCAL ] KFS: Primary data saved (8699 records). [2025-10-20T19:32:56.834061] [INFO ] [MAIN ] KFS: Full history saved. [2025-10-20T19:32:56.834139] [INFO ] [MAIN ] Update Progress: 1184/2367 symbols completed. KFS took 21.2s. Estimated remaining: 2h 32m 45s. Rate: 7.7 symbols/min [2025-10-20T19:32:57.013158] [INFO ] [MAIN ] KGC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:57.065521] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-05. [2025-10-20T19:32:57.068609] [DEBUG] [LOCAL ] KGC: Loaded 54480 existing records from primary storage. [2025-10-20T19:32:57.069069] [INFO ] [MAIN ] KGC: Loaded 54480 existing records. [2025-10-20T19:32:57.257266] [WARN ] [MAIN ] KGC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:57.257507] [INFO ] [MAIN ] KGC: Wiping existing primary data before full refresh. [2025-10-20T19:32:57.257561] [INFO ] [LOCAL ] KGC: Starting deletion of primary data. [2025-10-20T19:32:57.257770] [DEBUG] [LOCAL ] KGC: Deleted primary data file. [2025-10-20T19:32:57.257827] [INFO ] [MAIN ] KGC: Starting full primary refresh. [2025-10-20T19:32:57.635523] [INFO ] [REMOTE ] KGC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:57.851375] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-01. [2025-10-20T19:32:57.926810] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:32:57.933241] [INFO ] [REMOTE ] KEYS: Full fetch complete (63139 records). [2025-10-20T19:32:57.933364] [DEBUG] [VALIDATOR ] KEYS: Validating primary data. [2025-10-20T19:32:58.331871] [DEBUG] [VALIDATOR ] KEYS: All expected trading days are present. [2025-10-20T19:32:58.331973] [DEBUG] [VALIDATOR ] KEYS: Primary data validation passed. [2025-10-20T19:32:58.333010] [INFO ] [MAIN ] KEYS: Fetched primary data is valid. [2025-10-20T19:32:58.333089] [DEBUG] [LOCAL ] KEYS: Saving primary data. [2025-10-20T19:32:58.334385] [DEBUG] [LOCAL ] KEYS: Trimmed 63139 records to 43295 (keeping 200 days of data) [2025-10-20T19:32:58.453294] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:32:58.612104] [DEBUG] [LOCAL ] KEYS: Primary data saved (43295 records). [2025-10-20T19:32:58.612232] [INFO ] [MAIN ] KEYS: Full history saved. [2025-10-20T19:32:58.612298] [INFO ] [MAIN ] Update Progress: 1185/2367 symbols completed. KEYS took 41.2s. Estimated remaining: 2h 32m 31s. Rate: 7.7 symbols/min [2025-10-20T19:32:58.656589] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-09. [2025-10-20T19:32:58.781853] [INFO ] [MAIN ] KGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:58.822859] [DEBUG] [LOCAL ] KGS: Loaded 46744 existing records from primary storage. [2025-10-20T19:32:58.823146] [INFO ] [MAIN ] KGS: Loaded 46744 existing records. [2025-10-20T19:32:58.999803] [WARN ] [MAIN ] KGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:59.000058] [INFO ] [MAIN ] KGS: Wiping existing primary data before full refresh. [2025-10-20T19:32:59.000107] [INFO ] [LOCAL ] KGS: Starting deletion of primary data. [2025-10-20T19:32:59.000377] [DEBUG] [LOCAL ] KGS: Deleted primary data file. [2025-10-20T19:32:59.000443] [INFO ] [MAIN ] KGS: Starting full primary refresh. [2025-10-20T19:32:59.381130] [INFO ] [REMOTE ] KGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:59.451342] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-01. [2025-10-20T19:32:59.664803] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T19:32:59.821100] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T19:33:00.251399] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-10. [2025-10-20T19:33:01.051362] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-06. [2025-10-20T19:33:01.279018] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T19:33:01.330516] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T19:33:01.334408] [INFO ] [REMOTE ] KFRC: Full fetch complete (26845 records). [2025-10-20T19:33:01.334483] [DEBUG] [VALIDATOR ] KFRC: Validating primary data. [2025-10-20T19:33:01.722501] [DEBUG] [VALIDATOR ] KFRC: All expected trading days are present. [2025-10-20T19:33:01.722619] [DEBUG] [VALIDATOR ] KFRC: Primary data validation passed. [2025-10-20T19:33:01.723385] [INFO ] [MAIN ] KFRC: Fetched primary data is valid. [2025-10-20T19:33:01.723445] [DEBUG] [LOCAL ] KFRC: Saving primary data. [2025-10-20T19:33:01.724205] [DEBUG] [LOCAL ] KFRC: Trimmed 26845 records to 20727 (keeping 200 days of data) [2025-10-20T19:33:01.845985] [DEBUG] [LOCAL ] KFRC: Primary data saved (20727 records). [2025-10-20T19:33:01.846119] [INFO ] [MAIN ] KFRC: Full history saved. [2025-10-20T19:33:01.846198] [INFO ] [MAIN ] Update Progress: 1186/2367 symbols completed. KFRC took 40.6s. Estimated remaining: 2h 32m 19s. Rate: 7.8 symbols/min [2025-10-20T19:33:01.857399] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-02. [2025-10-20T19:33:02.017854] [INFO ] [MAIN ] KIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:02.067376] [DEBUG] [LOCAL ] KIM: Loaded 53783 existing records from primary storage. [2025-10-20T19:33:02.067714] [INFO ] [MAIN ] KIM: Loaded 53783 existing records. [2025-10-20T19:33:02.253469] [WARN ] [MAIN ] KIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:02.253849] [INFO ] [MAIN ] KIM: Wiping existing primary data before full refresh. [2025-10-20T19:33:02.253913] [INFO ] [LOCAL ] KIM: Starting deletion of primary data. [2025-10-20T19:33:02.254092] [DEBUG] [LOCAL ] KIM: Deleted primary data file. [2025-10-20T19:33:02.254131] [INFO ] [MAIN ] KIM: Starting full primary refresh. [2025-10-20T19:33:02.533855] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T19:33:02.645711] [INFO ] [REMOTE ] KIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:02.651472] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-01. [2025-10-20T19:33:03.221890] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T19:33:03.451386] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-07. [2025-10-20T19:33:04.251432] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-03. [2025-10-20T19:33:04.456943] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T19:33:04.757181] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:33:05.051476] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-08. [2025-10-20T19:33:05.851410] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-02. [2025-10-20T19:33:05.909936] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T19:33:06.159963] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T19:33:06.651383] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-02. [2025-10-20T19:33:07.451354] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-04. [2025-10-20T19:33:07.550342] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T19:33:08.251421] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-09. [2025-10-20T19:33:08.524752] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T19:33:09.051393] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-03. [2025-10-20T19:33:09.095520] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:33:09.609603] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:33:09.851457] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-05. [2025-10-20T19:33:10.651409] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-03. [2025-10-20T19:33:10.926903] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T19:33:11.448161] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T19:33:11.451441] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-10. [2025-10-20T19:33:12.251439] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-04. [2025-10-20T19:33:12.546829] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T19:33:13.051419] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-06. [2025-10-20T19:33:13.084572] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T19:33:13.089073] [INFO ] [REMOTE ] KFY: Full fetch complete (43311 records). [2025-10-20T19:33:13.089153] [DEBUG] [VALIDATOR ] KFY: Validating primary data. [2025-10-20T19:33:13.466183] [DEBUG] [VALIDATOR ] KFY: All expected trading days are present. [2025-10-20T19:33:13.466268] [DEBUG] [VALIDATOR ] KFY: Primary data validation passed. [2025-10-20T19:33:13.467006] [INFO ] [MAIN ] KFY: Fetched primary data is valid. [2025-10-20T19:33:13.467069] [DEBUG] [LOCAL ] KFY: Saving primary data. [2025-10-20T19:33:13.468103] [DEBUG] [LOCAL ] KFY: Trimmed 43311 records to 29744 (keeping 200 days of data) [2025-10-20T19:33:13.648317] [DEBUG] [LOCAL ] KFY: Primary data saved (29744 records). [2025-10-20T19:33:13.648461] [INFO ] [MAIN ] KFY: Full history saved. [2025-10-20T19:33:13.648541] [INFO ] [MAIN ] Update Progress: 1187/2367 symbols completed. KFY took 29.7s. Estimated remaining: 2h 32m 15s. Rate: 7.8 symbols/min [2025-10-20T19:33:13.840358] [INFO ] [MAIN ] KIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:13.854489] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-04. [2025-10-20T19:33:13.865493] [DEBUG] [LOCAL ] KIO: Loaded 22090 existing records from primary storage. [2025-10-20T19:33:13.865738] [INFO ] [MAIN ] KIO: Loaded 22090 existing records. [2025-10-20T19:33:14.042979] [WARN ] [MAIN ] KIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:14.043179] [INFO ] [MAIN ] KIO: Wiping existing primary data before full refresh. [2025-10-20T19:33:14.043232] [INFO ] [LOCAL ] KIO: Starting deletion of primary data. [2025-10-20T19:33:14.043539] [DEBUG] [LOCAL ] KIO: Deleted primary data file. [2025-10-20T19:33:14.043632] [INFO ] [MAIN ] KIO: Starting full primary refresh. [2025-10-20T19:33:14.153357] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T19:33:14.455110] [INFO ] [REMOTE ] KIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:14.519780] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:33:14.651374] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-05. [2025-10-20T19:33:15.451428] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-01. [2025-10-20T19:33:16.024148] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T19:33:16.251364] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-05. [2025-10-20T19:33:16.442538] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:33:16.549045] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:33:17.051375] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-07. [2025-10-20T19:33:17.851363] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-06. [2025-10-20T19:33:18.392886] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-10-20T19:33:18.432464] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T19:33:18.651364] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-08. [2025-10-20T19:33:19.124318] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T19:33:19.451354] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-02. [2025-10-20T19:33:20.251433] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-07. [2025-10-20T19:33:20.296109] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T19:33:20.662044] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T19:33:21.051421] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-06. [2025-10-20T19:33:21.851388] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-03. [2025-10-20T19:33:21.852633] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:33:22.651358] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-09. [2025-10-20T19:33:22.932169] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T19:33:23.045788] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T19:33:23.451393] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-04. [2025-10-20T19:33:24.152795] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T19:33:24.251335] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-08. [2025-10-20T19:33:24.491956] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T19:33:25.051411] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-10. [2025-10-20T19:33:25.851370] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-07. [2025-10-20T19:33:26.057529] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:33:26.651448] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-09. [2025-10-20T19:33:26.753141] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T19:33:26.762708] [INFO ] [REMOTE ] KGS: Full fetch complete (66516 records). [2025-10-20T19:33:26.762865] [DEBUG] [VALIDATOR ] KGS: Validating primary data. [2025-10-20T19:33:27.175920] [DEBUG] [VALIDATOR ] KGS: All expected trading days are present. [2025-10-20T19:33:27.176001] [DEBUG] [VALIDATOR ] KGS: Primary data validation passed. [2025-10-20T19:33:27.176889] [INFO ] [MAIN ] KGS: Fetched primary data is valid. [2025-10-20T19:33:27.176951] [DEBUG] [LOCAL ] KGS: Saving primary data. [2025-10-20T19:33:27.178221] [DEBUG] [LOCAL ] KGS: Trimmed 66516 records to 46120 (keeping 200 days of data) [2025-10-20T19:33:27.448751] [DEBUG] [LOCAL ] KGS: Primary data saved (46120 records). [2025-10-20T19:33:27.448898] [INFO ] [MAIN ] KGS: Full history saved. [2025-10-20T19:33:27.448974] [INFO ] [MAIN ] Update Progress: 1188/2367 symbols completed. KGS took 28.8s. Estimated remaining: 2h 32m 13s. Rate: 7.7 symbols/min [2025-10-20T19:33:27.460193] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-05. [2025-10-20T19:33:27.623203] [INFO ] [MAIN ] KKR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:27.677875] [DEBUG] [LOCAL ] KKR: Loaded 54283 existing records from primary storage. [2025-10-20T19:33:27.678213] [INFO ] [MAIN ] KKR: Loaded 54283 existing records. [2025-10-20T19:33:27.861317] [WARN ] [MAIN ] KKR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:27.861573] [INFO ] [MAIN ] KKR: Wiping existing primary data before full refresh. [2025-10-20T19:33:27.861668] [INFO ] [LOCAL ] KKR: Starting deletion of primary data. [2025-10-20T19:33:27.861914] [DEBUG] [LOCAL ] KKR: Deleted primary data file. [2025-10-20T19:33:27.861977] [INFO ] [MAIN ] KKR: Starting full primary refresh. [2025-10-20T19:33:28.264353] [INFO ] [REMOTE ] KKR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:28.264836] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-01. [2025-10-20T19:33:28.267836] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-07 (2.42 seconds). [2025-10-20T19:33:28.314441] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T19:33:28.565240] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:33:29.051370] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-06. [2025-10-20T19:33:29.851355] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-10. [2025-10-20T19:33:30.072280] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T19:33:30.116845] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T19:33:30.651398] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-08. [2025-10-20T19:33:31.454267] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-07. [2025-10-20T19:33:31.457156] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T19:33:31.465851] [INFO ] [REMOTE ] KIM: Full fetch complete (76978 records). [2025-10-20T19:33:31.465992] [DEBUG] [VALIDATOR ] KIM: Validating primary data. [2025-10-20T19:33:31.865672] [DEBUG] [VALIDATOR ] KIM: All expected trading days are present. [2025-10-20T19:33:31.865760] [DEBUG] [VALIDATOR ] KIM: Primary data validation passed. [2025-10-20T19:33:31.866741] [INFO ] [MAIN ] KIM: Fetched primary data is valid. [2025-10-20T19:33:31.866803] [DEBUG] [LOCAL ] KIM: Saving primary data. [2025-10-20T19:33:31.868223] [DEBUG] [LOCAL ] KIM: Trimmed 76978 records to 53012 (keeping 200 days of data) [2025-10-20T19:33:32.196129] [DEBUG] [LOCAL ] KIM: Primary data saved (53012 records). [2025-10-20T19:33:32.196243] [INFO ] [MAIN ] KIM: Full history saved. [2025-10-20T19:33:32.196313] [INFO ] [MAIN ] Update Progress: 1189/2367 symbols completed. KIM took 30.4s. Estimated remaining: 2h 32m 3s. Rate: 7.7 symbols/min [2025-10-20T19:33:32.259827] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-02. [2025-10-20T19:33:32.379941] [INFO ] [MAIN ] KKRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:32.388151] [DEBUG] [LOCAL ] KKRS: Loaded 5669 existing records from primary storage. [2025-10-20T19:33:32.388387] [INFO ] [MAIN ] KKRS: Loaded 5669 existing records. [2025-10-20T19:33:32.584057] [WARN ] [MAIN ] KKRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:32.584978] [INFO ] [MAIN ] KKRS: Wiping existing primary data before full refresh. [2025-10-20T19:33:32.585063] [INFO ] [LOCAL ] KKRS: Starting deletion of primary data. [2025-10-20T19:33:32.585267] [DEBUG] [LOCAL ] KKRS: Deleted primary data file. [2025-10-20T19:33:32.585347] [INFO ] [MAIN ] KKRS: Starting full primary refresh. [2025-10-20T19:33:32.716136] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T19:33:32.981084] [INFO ] [REMOTE ] KKRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:33.012754] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-08 (2.36 seconds). [2025-10-20T19:33:33.051344] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-09. [2025-10-20T19:33:33.851427] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-08. [2025-10-20T19:33:34.049127] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:33:34.651360] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-01. [2025-10-20T19:33:34.928546] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T19:33:35.218893] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T19:33:35.275922] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T19:33:35.451367] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-02. [2025-10-20T19:33:36.031179] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T19:33:36.251366] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-09. [2025-10-20T19:33:37.051449] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-03. [2025-10-20T19:33:37.316383] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T19:33:37.598226] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T19:33:37.851372] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-03. [2025-10-20T19:33:38.651406] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-10. [2025-10-20T19:33:39.451385] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-10. [2025-10-20T19:33:39.834812] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T19:33:39.963963] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:33:39.968819] [INFO ] [REMOTE ] KIO: Full fetch complete (32242 records). [2025-10-20T19:33:39.968993] [DEBUG] [VALIDATOR ] KIO: Validating primary data. [2025-10-20T19:33:40.256949] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-04. [2025-10-20T19:33:40.378137] [DEBUG] [VALIDATOR ] KIO: All expected trading days are present. [2025-10-20T19:33:40.378237] [DEBUG] [VALIDATOR ] KIO: Primary data validation passed. [2025-10-20T19:33:40.379288] [INFO ] [MAIN ] KIO: Fetched primary data is valid. [2025-10-20T19:33:40.379374] [DEBUG] [LOCAL ] KIO: Saving primary data. [2025-10-20T19:33:40.380518] [DEBUG] [LOCAL ] KIO: Trimmed 32242 records to 21766 (keeping 200 days of data) [2025-10-20T19:33:40.512860] [DEBUG] [LOCAL ] KIO: Primary data saved (21766 records). [2025-10-20T19:33:40.512998] [INFO ] [MAIN ] KIO: Full history saved. [2025-10-20T19:33:40.513078] [INFO ] [MAIN ] Update Progress: 1190/2367 symbols completed. KIO took 26.9s. Estimated remaining: 2h 31m 55s. Rate: 7.7 symbols/min [2025-10-20T19:33:40.695545] [INFO ] [MAIN ] KKRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:40.695753] [DEBUG] [LOCAL ] KKRT: No existing data file found in primary storage. [2025-10-20T19:33:40.695800] [WARN ] [MAIN ] KKRT: No existing data found, performing full refresh instead. [2025-10-20T19:33:40.695937] [INFO ] [MAIN ] KKRT: Wiping existing primary data before full refresh. [2025-10-20T19:33:40.695979] [INFO ] [LOCAL ] KKRT: Starting deletion of primary data. [2025-10-20T19:33:40.696023] [DEBUG] [LOCAL ] KKRT: No primary data file found to delete. [2025-10-20T19:33:40.696055] [INFO ] [MAIN ] KKRT: Starting full primary refresh. [2025-10-20T19:33:41.072499] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-04. [2025-10-20T19:33:41.077249] [INFO ] [REMOTE ] KKRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:41.108174] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T19:33:41.724022] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-10 (2.27 seconds). [2025-10-20T19:33:41.732457] [INFO ] [REMOTE ] KGC: Full fetch complete (77873 records). [2025-10-20T19:33:41.732615] [DEBUG] [VALIDATOR ] KGC: Validating primary data. [2025-10-20T19:33:41.856524] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-01. [2025-10-20T19:33:42.131741] [DEBUG] [REMOTE ] KKRT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:33:42.135611] [DEBUG] [VALIDATOR ] KGC: All expected trading days are present. [2025-10-20T19:33:42.135891] [DEBUG] [VALIDATOR ] KGC: Primary data validation passed. [2025-10-20T19:33:42.137126] [INFO ] [MAIN ] KGC: Fetched primary data is valid. [2025-10-20T19:33:42.136006] [WARN ] [REMOTE ] KKRT: No data returned for month 2025-01 [2025-10-20T19:33:42.137378] [DEBUG] [LOCAL ] KGC: Saving primary data. [2025-10-20T19:33:42.137316] [FATAL] [REMOTE ] KKRT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:33:42.137755] [FATAL] [MAIN ] KKRT: Failed to fetch primary data. [2025-10-20T19:33:42.137827] [WARN ] [MAIN ] KKRT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:33:42.137894] [INFO ] [LOCAL ] KKRT: Starting deletion of primary data. [2025-10-20T19:33:42.137971] [DEBUG] [LOCAL ] KKRT: No primary data file found to delete. [2025-10-20T19:33:42.138026] [INFO ] [LOCAL ] KKRT: Starting deletion of derivative data. [2025-10-20T19:33:42.138522] [DEBUG] [LOCAL ] KKRT: No derivative data file found to delete. [2025-10-20T19:33:42.138724] [INFO ] [MAIN ] Update Progress: 1191/2367 symbols completed. KKRT took 1.6s. Estimated remaining: 2h 31m 42s. Rate: 7.8 symbols/min [2025-10-20T19:33:42.215513] [DEBUG] [LOCAL ] KGC: Trimmed 77873 records to 53700 (keeping 200 days of data) [2025-10-20T19:33:42.417996] [INFO ] [MAIN ] KLAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:42.422058] [DEBUG] [LOCAL ] KLAR: No existing data file found in primary storage. [2025-10-20T19:33:42.424810] [WARN ] [MAIN ] KLAR: No existing data found, performing full refresh instead. [2025-10-20T19:33:42.427733] [INFO ] [MAIN ] KLAR: Wiping existing primary data before full refresh. [2025-10-20T19:33:42.428986] [INFO ] [LOCAL ] KLAR: Starting deletion of primary data. [2025-10-20T19:33:42.429787] [DEBUG] [LOCAL ] KLAR: No primary data file found to delete. [2025-10-20T19:33:42.436438] [INFO ] [MAIN ] KLAR: Starting full primary refresh. [2025-10-20T19:33:42.653039] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-05. [2025-10-20T19:33:42.923149] [INFO ] [REMOTE ] KLAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:43.016508] [DEBUG] [LOCAL ] KGC: Primary data saved (53700 records). [2025-10-20T19:33:43.016765] [INFO ] [MAIN ] KGC: Full history saved. [2025-10-20T19:33:43.016896] [INFO ] [MAIN ] Update Progress: 1192/2367 symbols completed. KGC took 46.2s. Estimated remaining: 2h 31m 27s. Rate: 7.8 symbols/min [2025-10-20T19:33:43.205677] [INFO ] [MAIN ] KLC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:43.220313] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T19:33:43.263564] [DEBUG] [LOCAL ] KLC: Loaded 37016 existing records from primary storage. [2025-10-20T19:33:43.263906] [INFO ] [MAIN ] KLC: Loaded 37016 existing records. [2025-10-20T19:33:43.440955] [WARN ] [MAIN ] KLC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:43.441138] [INFO ] [MAIN ] KLC: Wiping existing primary data before full refresh. [2025-10-20T19:33:43.441183] [INFO ] [LOCAL ] KLC: Starting deletion of primary data. [2025-10-20T19:33:43.441496] [DEBUG] [LOCAL ] KLC: Deleted primary data file. [2025-10-20T19:33:43.441551] [INFO ] [MAIN ] KLC: Starting full primary refresh. [2025-10-20T19:33:43.456607] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-01. [2025-10-20T19:33:43.530217] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T19:33:43.813496] [INFO ] [REMOTE ] KLC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:43.913409] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:33:43.913683] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-01 [2025-10-20T19:33:43.913768] [FATAL] [REMOTE ] KLAR: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:33:43.913814] [FATAL] [MAIN ] KLAR: Failed to fetch primary data. [2025-10-20T19:33:43.913888] [WARN ] [MAIN ] KLAR: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:33:43.913958] [INFO ] [LOCAL ] KLAR: Starting deletion of primary data. [2025-10-20T19:33:43.914042] [DEBUG] [LOCAL ] KLAR: No primary data file found to delete. [2025-10-20T19:33:43.914096] [INFO ] [LOCAL ] KLAR: Starting deletion of derivative data. [2025-10-20T19:33:43.914138] [DEBUG] [LOCAL ] KLAR: No derivative data file found to delete. [2025-10-20T19:33:43.914181] [INFO ] [MAIN ] Update Progress: 1193/2367 symbols completed. KLAR took 1.8s. Estimated remaining: 2h 31m 13s. Rate: 7.8 symbols/min [2025-10-20T19:33:44.081168] [INFO ] [MAIN ] KMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:44.133159] [DEBUG] [LOCAL ] KMI: Loaded 54473 existing records from primary storage. [2025-10-20T19:33:44.133460] [INFO ] [MAIN ] KMI: Loaded 54473 existing records. [2025-10-20T19:33:44.256555] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-05. [2025-10-20T19:33:44.314098] [WARN ] [MAIN ] KMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:44.314264] [INFO ] [MAIN ] KMI: Wiping existing primary data before full refresh. [2025-10-20T19:33:44.314309] [INFO ] [LOCAL ] KMI: Starting deletion of primary data. [2025-10-20T19:33:44.314808] [DEBUG] [LOCAL ] KMI: Deleted primary data file. [2025-10-20T19:33:44.314862] [INFO ] [MAIN ] KMI: Starting full primary refresh. [2025-10-20T19:33:44.672849] [INFO ] [REMOTE ] KMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:45.051417] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-01. [2025-10-20T19:33:45.851430] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-06. [2025-10-20T19:33:46.370564] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T19:33:46.505074] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-10-20T19:33:46.547931] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T19:33:46.651342] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-07. [2025-10-20T19:33:47.248588] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T19:33:47.451401] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-08. [2025-10-20T19:33:48.030616] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:33:48.251388] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-06. [2025-10-20T19:33:49.051391] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-09. [2025-10-20T19:33:49.624117] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T19:33:49.851345] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-10. [2025-10-20T19:33:50.142120] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:33:50.555741] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T19:33:50.556774] [INFO ] [REMOTE ] KKRS: Full fetch complete (7562 records). [2025-10-20T19:33:50.556839] [DEBUG] [VALIDATOR ] KKRS: Validating primary data. [2025-10-20T19:33:50.656683] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-02. [2025-10-20T19:33:50.911916] [DEBUG] [VALIDATOR ] KKRS: All expected trading days are present. [2025-10-20T19:33:50.911990] [DEBUG] [VALIDATOR ] KKRS: Primary data validation passed. [2025-10-20T19:33:50.912221] [INFO ] [MAIN ] KKRS: Fetched primary data is valid. [2025-10-20T19:33:50.912264] [DEBUG] [LOCAL ] KKRS: Saving primary data. [2025-10-20T19:33:50.912662] [DEBUG] [LOCAL ] KKRS: Trimmed 7562 records to 5590 (keeping 200 days of data) [2025-10-20T19:33:50.945511] [DEBUG] [LOCAL ] KKRS: Primary data saved (5590 records). [2025-10-20T19:33:50.945596] [INFO ] [MAIN ] KKRS: Full history saved. [2025-10-20T19:33:50.945686] [INFO ] [MAIN ] Update Progress: 1194/2367 symbols completed. KKRS took 18.7s. Estimated remaining: 2h 31m 4s. Rate: 7.8 symbols/min [2025-10-20T19:33:51.114757] [INFO ] [MAIN ] KMPB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:51.121258] [DEBUG] [LOCAL ] KMPB: Loaded 3680 existing records from primary storage. [2025-10-20T19:33:51.121447] [INFO ] [MAIN ] KMPB: Loaded 3680 existing records. [2025-10-20T19:33:51.298461] [WARN ] [MAIN ] KMPB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:51.298744] [INFO ] [MAIN ] KMPB: Wiping existing primary data before full refresh. [2025-10-20T19:33:51.298806] [INFO ] [LOCAL ] KMPB: Starting deletion of primary data. [2025-10-20T19:33:51.298971] [DEBUG] [LOCAL ] KMPB: Deleted primary data file. [2025-10-20T19:33:51.299029] [INFO ] [MAIN ] KMPB: Starting full primary refresh. [2025-10-20T19:33:51.460701] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-01. [2025-10-20T19:33:51.660917] [INFO ] [REMOTE ] KMPB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:51.753453] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T19:33:52.251596] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-03. [2025-10-20T19:33:53.051383] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-01. [2025-10-20T19:33:53.807332] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T19:33:53.809971] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:33:53.845914] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-01 (2.39 seconds). [2025-10-20T19:33:53.855985] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-04. [2025-10-20T19:33:54.651376] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-07. [2025-10-20T19:33:55.377840] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:33:55.451373] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-02. [2025-10-20T19:33:56.251371] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-05. [2025-10-20T19:33:56.663828] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T19:33:57.051395] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-02. [2025-10-20T19:33:57.413965] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-10-20T19:33:57.496194] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T19:33:57.704841] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:33:57.851367] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-03. [2025-10-20T19:33:58.651377] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-06. [2025-10-20T19:33:59.451365] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-08. [2025-10-20T19:33:59.934317] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T19:33:59.981414] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T19:34:00.251410] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-03. [2025-10-20T19:34:00.918371] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T19:34:01.051390] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-04. [2025-10-20T19:34:01.586553] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T19:34:01.635062] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T19:34:01.851366] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-04. [2025-10-20T19:34:02.651419] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-07. [2025-10-20T19:34:03.451415] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-09. [2025-10-20T19:34:03.963986] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-04 (2.11 seconds). [2025-10-20T19:34:04.078269] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T19:34:04.251403] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-08. [2025-10-20T19:34:05.044634] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:34:05.055107] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-05. [2025-10-20T19:34:05.713544] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T19:34:05.851369] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-09. [2025-10-20T19:34:06.651372] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-10. [2025-10-20T19:34:06.694828] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T19:34:07.242989] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T19:34:07.451439] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-06. [2025-10-20T19:34:08.251375] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-05. [2025-10-20T19:34:08.540070] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T19:34:08.548076] [INFO ] [REMOTE ] KKR: Full fetch complete (77648 records). [2025-10-20T19:34:08.548208] [DEBUG] [VALIDATOR ] KKR: Validating primary data. [2025-10-20T19:34:08.842105] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T19:34:08.966883] [DEBUG] [VALIDATOR ] KKR: All expected trading days are present. [2025-10-20T19:34:08.966960] [DEBUG] [VALIDATOR ] KKR: Primary data validation passed. [2025-10-20T19:34:08.967921] [INFO ] [MAIN ] KKR: Fetched primary data is valid. [2025-10-20T19:34:08.967981] [DEBUG] [LOCAL ] KKR: Saving primary data. [2025-10-20T19:34:08.969323] [DEBUG] [LOCAL ] KKR: Trimmed 77648 records to 53503 (keeping 200 days of data) [2025-10-20T19:34:09.058200] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-06. [2025-10-20T19:34:09.343447] [DEBUG] [LOCAL ] KKR: Primary data saved (53503 records). [2025-10-20T19:34:09.344237] [INFO ] [MAIN ] KKR: Full history saved. [2025-10-20T19:34:09.345456] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:34:09.346130] [INFO ] [MAIN ] Update Progress: 1195/2367 symbols completed. KKR took 41.9s. Estimated remaining: 2h 31m 7s. Rate: 7.8 symbols/min [2025-10-20T19:34:09.528388] [INFO ] [MAIN ] KMPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:09.567132] [DEBUG] [LOCAL ] KMPR: Loaded 40730 existing records from primary storage. [2025-10-20T19:34:09.567398] [INFO ] [MAIN ] KMPR: Loaded 40730 existing records. [2025-10-20T19:34:09.677349] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T19:34:09.750834] [WARN ] [MAIN ] KMPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:09.751034] [INFO ] [MAIN ] KMPR: Wiping existing primary data before full refresh. [2025-10-20T19:34:09.751140] [INFO ] [LOCAL ] KMPR: Starting deletion of primary data. [2025-10-20T19:34:09.751320] [DEBUG] [LOCAL ] KMPR: Deleted primary data file. [2025-10-20T19:34:09.751359] [INFO ] [MAIN ] KMPR: Starting full primary refresh. [2025-10-20T19:34:09.854436] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-07. [2025-10-20T19:34:10.147146] [INFO ] [REMOTE ] KMPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:10.651378] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-01. [2025-10-20T19:34:11.451355] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-07. [2025-10-20T19:34:11.927475] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T19:34:11.998515] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-10-20T19:34:12.077835] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T19:34:12.251413] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-02. [2025-10-20T19:34:13.051406] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-08. [2025-10-20T19:34:13.233681] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T19:34:13.628176] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:34:13.851376] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-08. [2025-10-20T19:34:14.651365] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-09. [2025-10-20T19:34:15.237540] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:34:15.451366] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-10. [2025-10-20T19:34:15.573144] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:34:16.251435] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-03. [2025-10-20T19:34:16.835506] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T19:34:16.841209] [INFO ] [REMOTE ] KLC: Full fetch complete (50521 records). [2025-10-20T19:34:16.841305] [DEBUG] [VALIDATOR ] KLC: Validating primary data. [2025-10-20T19:34:17.060404] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-09. [2025-10-20T19:34:17.225302] [DEBUG] [VALIDATOR ] KLC: All expected trading days are present. [2025-10-20T19:34:17.225480] [DEBUG] [VALIDATOR ] KLC: Primary data validation passed. [2025-10-20T19:34:17.226462] [INFO ] [MAIN ] KLC: Fetched primary data is valid. [2025-10-20T19:34:17.226523] [DEBUG] [LOCAL ] KLC: Saving primary data. [2025-10-20T19:34:17.227504] [DEBUG] [LOCAL ] KLC: Trimmed 50521 records to 36293 (keeping 200 days of data) [2025-10-20T19:34:17.425932] [DEBUG] [LOCAL ] KLC: Primary data saved (36293 records). [2025-10-20T19:34:17.426062] [INFO ] [MAIN ] KLC: Full history saved. [2025-10-20T19:34:17.426140] [INFO ] [MAIN ] Update Progress: 1196/2367 symbols completed. KLC took 34.4s. Estimated remaining: 2h 30m 60s. Rate: 7.8 symbols/min [2025-10-20T19:34:17.596429] [INFO ] [MAIN ] KMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:17.615241] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:34:17.658308] [DEBUG] [LOCAL ] KMT: Loaded 43532 existing records from primary storage. [2025-10-20T19:34:17.658639] [INFO ] [MAIN ] KMT: Loaded 43532 existing records. [2025-10-20T19:34:17.834033] [WARN ] [MAIN ] KMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:17.834281] [INFO ] [MAIN ] KMT: Wiping existing primary data before full refresh. [2025-10-20T19:34:17.834357] [INFO ] [LOCAL ] KMT: Starting deletion of primary data. [2025-10-20T19:34:17.834569] [DEBUG] [LOCAL ] KMT: Deleted primary data file. [2025-10-20T19:34:17.834629] [INFO ] [MAIN ] KMT: Starting full primary refresh. [2025-10-20T19:34:17.856606] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-04. [2025-10-20T19:34:18.196111] [INFO ] [REMOTE ] KMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:18.651364] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-10. [2025-10-20T19:34:18.949105] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T19:34:19.399032] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T19:34:19.400183] [INFO ] [REMOTE ] KMPB: Full fetch complete (4969 records). [2025-10-20T19:34:19.400255] [DEBUG] [VALIDATOR ] KMPB: Validating primary data. [2025-10-20T19:34:19.469939] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-10. [2025-10-20T19:34:19.587054] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T19:34:19.804600] [DEBUG] [VALIDATOR ] KMPB: All expected trading days are present. [2025-10-20T19:34:19.804701] [DEBUG] [VALIDATOR ] KMPB: Primary data validation passed. [2025-10-20T19:34:19.804858] [INFO ] [MAIN ] KMPB: Fetched primary data is valid. [2025-10-20T19:34:19.804896] [DEBUG] [LOCAL ] KMPB: Saving primary data. [2025-10-20T19:34:19.805269] [DEBUG] [LOCAL ] KMPB: Trimmed 4969 records to 3558 (keeping 200 days of data) [2025-10-20T19:34:19.826448] [DEBUG] [LOCAL ] KMPB: Primary data saved (3558 records). [2025-10-20T19:34:19.826572] [INFO ] [MAIN ] KMPB: Full history saved. [2025-10-20T19:34:19.826671] [INFO ] [MAIN ] Update Progress: 1197/2367 symbols completed. KMPB took 28.9s. Estimated remaining: 2h 30m 47s. Rate: 7.8 symbols/min [2025-10-20T19:34:20.251434] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-05. [2025-10-20T19:34:20.389056] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T19:34:20.396964] [INFO ] [REMOTE ] KMI: Full fetch complete (77873 records). [2025-10-20T19:34:20.397090] [DEBUG] [VALIDATOR ] KMI: Validating primary data. [2025-10-20T19:34:20.779978] [DEBUG] [VALIDATOR ] KMI: All expected trading days are present. [2025-10-20T19:34:20.780074] [DEBUG] [VALIDATOR ] KMI: Primary data validation passed. [2025-10-20T19:34:20.781053] [INFO ] [MAIN ] KMI: Fetched primary data is valid. [2025-10-20T19:34:20.781114] [DEBUG] [LOCAL ] KMI: Saving primary data. [2025-10-20T19:34:20.782535] [DEBUG] [LOCAL ] KMI: Trimmed 77873 records to 53693 (keeping 200 days of data) [2025-10-20T19:34:21.057518] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-01. [2025-10-20T19:34:21.100371] [DEBUG] [LOCAL ] KMI: Primary data saved (53693 records). [2025-10-20T19:34:21.100476] [INFO ] [MAIN ] KMI: Full history saved. [2025-10-20T19:34:21.101491] [INFO ] [MAIN ] Update Progress: 1198/2367 symbols completed. KMI took 37.2s. Estimated remaining: 2h 30m 33s. Rate: 7.8 symbols/min [2025-10-20T19:34:21.681633] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T19:34:21.851367] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-06. [2025-10-20T19:34:22.700148] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:34:22.708496] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-02. [2025-10-20T19:34:22.786004] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T19:34:23.451473] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-07. [2025-10-20T19:34:24.194814] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T19:34:24.251377] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-03. [2025-10-20T19:34:24.853001] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T19:34:25.051359] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-08. [2025-10-20T19:34:25.683729] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T19:34:25.851413] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-04. [2025-10-20T19:34:26.564030] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:34:26.651387] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-09. [2025-10-20T19:34:27.302445] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T19:34:27.451441] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-05. [2025-10-20T19:34:28.022024] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T19:34:28.251458] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-10. [2025-10-20T19:34:28.960092] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:34:29.051366] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-06. [2025-10-20T19:34:29.560839] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:34:29.567802] [INFO ] [REMOTE ] KMPR: Full fetch complete (53343 records). [2025-10-20T19:34:29.567904] [DEBUG] [VALIDATOR ] KMPR: Validating primary data. [2025-10-20T19:34:29.953440] [DEBUG] [VALIDATOR ] KMPR: All expected trading days are present. [2025-10-20T19:34:29.953522] [DEBUG] [VALIDATOR ] KMPR: Primary data validation passed. [2025-10-20T19:34:29.954318] [INFO ] [MAIN ] KMPR: Fetched primary data is valid. [2025-10-20T19:34:29.954378] [DEBUG] [LOCAL ] KMPR: Saving primary data. [2025-10-20T19:34:29.955418] [DEBUG] [LOCAL ] KMPR: Trimmed 53343 records to 40452 (keeping 200 days of data) [2025-10-20T19:34:30.191789] [DEBUG] [LOCAL ] KMPR: Primary data saved (40452 records). [2025-10-20T19:34:30.191912] [INFO ] [MAIN ] KMPR: Full history saved. [2025-10-20T19:34:30.192391] [INFO ] [MAIN ] Update Progress: 1199/2367 symbols completed. KMPR took 20.8s. Estimated remaining: 2h 30m 26s. Rate: 7.8 symbols/min [2025-10-20T19:34:30.498779] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T19:34:30.507303] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-07. [2025-10-20T19:34:32.056753] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:34:32.065970] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-08. [2025-10-20T19:34:33.525343] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T19:34:33.535145] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-09. [2025-10-20T19:34:34.806681] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T19:34:34.816163] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-10. [2025-10-20T19:34:36.165250] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T19:34:36.171117] [INFO ] [REMOTE ] KMT: Full fetch complete (61953 records). [2025-10-20T19:34:36.171213] [DEBUG] [VALIDATOR ] KMT: Validating primary data. [2025-10-20T19:34:36.572386] [DEBUG] [VALIDATOR ] KMT: All expected trading days are present. [2025-10-20T19:34:36.572471] [DEBUG] [VALIDATOR ] KMT: Primary data validation passed. [2025-10-20T19:34:36.573353] [INFO ] [MAIN ] KMT: Fetched primary data is valid. [2025-10-20T19:34:36.573414] [DEBUG] [LOCAL ] KMT: Saving primary data. [2025-10-20T19:34:36.574625] [DEBUG] [LOCAL ] KMT: Trimmed 61953 records to 42948 (keeping 200 days of data) [2025-10-20T19:34:36.836297] [DEBUG] [LOCAL ] KMT: Primary data saved (42948 records). [2025-10-20T19:34:36.836427] [INFO ] [MAIN ] KMT: Full history saved. [2025-10-20T19:34:36.836502] [INFO ] [MAIN ] Update Progress: 1200/2367 symbols completed. KMT took 19.4s. Estimated remaining: 2h 30m 17s. Rate: 7.8 symbols/min [2025-10-20T19:34:37.381623] [INFO ] [MAIN ] KN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:37.547334] [INFO ] [MAIN ] KMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:37.549088] [INFO ] [MAIN ] KNOP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:37.552273] [INFO ] [MAIN ] KNF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:37.580869] [DEBUG] [LOCAL ] KNOP: Loaded 10627 existing records from primary storage. [2025-10-20T19:34:37.590333] [INFO ] [MAIN ] KNOP: Loaded 10627 existing records. [2025-10-20T19:34:37.686820] [DEBUG] [LOCAL ] KNF: Loaded 37116 existing records from primary storage. [2025-10-20T19:34:37.707870] [DEBUG] [LOCAL ] KN: Loaded 39111 existing records from primary storage. [2025-10-20T19:34:37.713492] [INFO ] [MAIN ] KN: Loaded 39111 existing records. [2025-10-20T19:34:37.740170] [INFO ] [MAIN ] KNF: Loaded 37116 existing records. [2025-10-20T19:34:38.061058] [DEBUG] [LOCAL ] KMX: Loaded 54057 existing records from primary storage. [2025-10-20T19:34:38.105707] [WARN ] [MAIN ] KNOP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:38.137779] [INFO ] [MAIN ] KMX: Loaded 54057 existing records. [2025-10-20T19:34:38.152902] [INFO ] [MAIN ] KNOP: Wiping existing primary data before full refresh. [2025-10-20T19:34:38.196594] [WARN ] [MAIN ] KN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:38.212175] [INFO ] [LOCAL ] KNOP: Starting deletion of primary data. [2025-10-20T19:34:38.264518] [INFO ] [MAIN ] KN: Wiping existing primary data before full refresh. [2025-10-20T19:34:38.289553] [INFO ] [LOCAL ] KN: Starting deletion of primary data. [2025-10-20T19:34:38.300300] [DEBUG] [LOCAL ] KNOP: Deleted primary data file. [2025-10-20T19:34:38.331100] [DEBUG] [LOCAL ] KN: Deleted primary data file. [2025-10-20T19:34:38.324744] [WARN ] [MAIN ] KNF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:38.336496] [INFO ] [MAIN ] KN: Starting full primary refresh. [2025-10-20T19:34:38.357386] [INFO ] [MAIN ] KNF: Wiping existing primary data before full refresh. [2025-10-20T19:34:38.357542] [INFO ] [LOCAL ] KNF: Starting deletion of primary data. [2025-10-20T19:34:38.331027] [INFO ] [MAIN ] KNOP: Starting full primary refresh. [2025-10-20T19:34:38.418695] [DEBUG] [LOCAL ] KNF: Deleted primary data file. [2025-10-20T19:34:38.468584] [INFO ] [MAIN ] KNF: Starting full primary refresh. [2025-10-20T19:34:38.669525] [WARN ] [MAIN ] KMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:38.787002] [INFO ] [MAIN ] KMX: Wiping existing primary data before full refresh. [2025-10-20T19:34:38.855734] [INFO ] [LOCAL ] KMX: Starting deletion of primary data. [2025-10-20T19:34:39.046493] [DEBUG] [LOCAL ] KMX: Deleted primary data file. [2025-10-20T19:34:39.093066] [INFO ] [MAIN ] KMX: Starting full primary refresh. [2025-10-20T19:34:39.484137] [INFO ] [REMOTE ] KNOP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:39.530369] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-01. [2025-10-20T19:34:39.582335] [INFO ] [REMOTE ] KN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:39.593244] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-01. [2025-10-20T19:34:39.681540] [INFO ] [REMOTE ] KNF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:39.712062] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-01. [2025-10-20T19:34:39.911479] [INFO ] [REMOTE ] KMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:39.911804] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-01. [2025-10-20T19:34:40.670295] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:34:40.672954] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-02. [2025-10-20T19:34:41.093822] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T19:34:41.101612] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-02. [2025-10-20T19:34:41.229778] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T19:34:41.236214] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-02. [2025-10-20T19:34:41.365458] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T19:34:41.368986] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-03. [2025-10-20T19:34:41.738070] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T19:34:41.748407] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-02. [2025-10-20T19:34:42.228807] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T19:34:42.231829] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-04. [2025-10-20T19:34:42.369978] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:34:42.376842] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-03. [2025-10-20T19:34:42.591948] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T19:34:42.600084] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-03. [2025-10-20T19:34:42.764704] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-10-20T19:34:42.768075] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-05. [2025-10-20T19:34:43.334789] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T19:34:43.363061] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T19:34:43.451348] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-03. [2025-10-20T19:34:43.500751] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:34:44.057346] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:34:44.251357] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-04. [2025-10-20T19:34:45.051453] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-04. [2025-10-20T19:34:45.229533] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T19:34:45.851378] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-04. [2025-10-20T19:34:46.096865] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:34:46.309459] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T19:34:46.651371] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-05. [2025-10-20T19:34:47.451374] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-05. [2025-10-20T19:34:47.955106] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-10-20T19:34:48.006991] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T19:34:48.251370] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-06. [2025-10-20T19:34:48.711484] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T19:34:49.051379] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-06. [2025-10-20T19:34:49.591284] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T19:34:49.851417] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-06. [2025-10-20T19:34:50.046782] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T19:34:50.482242] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T19:34:50.651367] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-07. [2025-10-20T19:34:51.451370] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-07. [2025-10-20T19:34:52.059993] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T19:34:52.251368] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-05. [2025-10-20T19:34:52.841156] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T19:34:53.051351] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-08. [2025-10-20T19:34:53.851380] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-08. [2025-10-20T19:34:54.115358] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T19:34:54.486702] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T19:34:54.651380] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-07. [2025-10-20T19:34:55.251610] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T19:34:55.451434] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-09. [2025-10-20T19:34:55.508777] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T19:34:56.251416] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-09. [2025-10-20T19:34:56.849570] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T19:34:57.051391] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-10. [2025-10-20T19:34:57.636275] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T19:34:57.851350] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-10. [2025-10-20T19:34:58.633848] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T19:34:58.639819] [INFO ] [REMOTE ] KNF: Full fetch complete (51421 records). [2025-10-20T19:34:58.639913] [DEBUG] [VALIDATOR ] KNF: Validating primary data. [2025-10-20T19:34:58.661081] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-06. [2025-10-20T19:34:59.020628] [DEBUG] [VALIDATOR ] KNF: All expected trading days are present. [2025-10-20T19:34:59.020748] [DEBUG] [VALIDATOR ] KNF: Primary data validation passed. [2025-10-20T19:34:59.021522] [INFO ] [MAIN ] KNF: Fetched primary data is valid. [2025-10-20T19:34:59.021563] [DEBUG] [LOCAL ] KNF: Saving primary data. [2025-10-20T19:34:59.022580] [DEBUG] [LOCAL ] KNF: Trimmed 51421 records to 36667 (keeping 200 days of data) [2025-10-20T19:34:59.243063] [DEBUG] [LOCAL ] KNF: Primary data saved (36667 records). [2025-10-20T19:34:59.243312] [INFO ] [MAIN ] KNF: Full history saved. [2025-10-20T19:34:59.244101] [INFO ] [MAIN ] Update Progress: 1201/2367 symbols completed. KNF took 22.4s. Estimated remaining: 2h 30m 24s. Rate: 7.8 symbols/min [2025-10-20T19:34:59.430062] [INFO ] [MAIN ] KNSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:59.431261] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T19:34:59.437509] [INFO ] [REMOTE ] KN: Full fetch complete (56557 records). [2025-10-20T19:34:59.437723] [DEBUG] [VALIDATOR ] KN: Validating primary data. [2025-10-20T19:34:59.459039] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-08. [2025-10-20T19:34:59.593068] [DEBUG] [LOCAL ] KNSL: Loaded 18750 existing records from primary storage. [2025-10-20T19:34:59.612295] [INFO ] [MAIN ] KNSL: Loaded 18750 existing records. [2025-10-20T19:34:59.954479] [WARN ] [MAIN ] KNSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:59.986251] [INFO ] [MAIN ] KNSL: Wiping existing primary data before full refresh. [2025-10-20T19:34:59.986635] [INFO ] [LOCAL ] KNSL: Starting deletion of primary data. [2025-10-20T19:35:00.002200] [DEBUG] [LOCAL ] KNSL: Deleted primary data file. [2025-10-20T19:35:00.007710] [INFO ] [MAIN ] KNSL: Starting full primary refresh. [2025-10-20T19:35:00.057077] [DEBUG] [VALIDATOR ] KN: All expected trading days are present. [2025-10-20T19:35:00.064042] [DEBUG] [VALIDATOR ] KN: Primary data validation passed. [2025-10-20T19:35:00.090933] [INFO ] [MAIN ] KN: Fetched primary data is valid. [2025-10-20T19:35:00.096697] [DEBUG] [LOCAL ] KN: Saving primary data. [2025-10-20T19:35:00.140303] [DEBUG] [LOCAL ] KN: Trimmed 56557 records to 38488 (keeping 200 days of data) [2025-10-20T19:35:00.360397] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T19:35:00.430040] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-09. [2025-10-20T19:35:00.589037] [INFO ] [REMOTE ] KNSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:00.621015] [DEBUG] [LOCAL ] KN: Primary data saved (38488 records). [2025-10-20T19:35:00.621140] [INFO ] [MAIN ] KN: Full history saved. [2025-10-20T19:35:00.621247] [INFO ] [MAIN ] Update Progress: 1202/2367 symbols completed. KN took 23.8s. Estimated remaining: 2h 30m 10s. Rate: 7.8 symbols/min [2025-10-20T19:35:00.824679] [INFO ] [MAIN ] KNTK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:00.826344] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T19:35:00.881167] [DEBUG] [LOCAL ] KNTK: Loaded 47788 existing records from primary storage. [2025-10-20T19:35:00.881555] [INFO ] [MAIN ] KNTK: Loaded 47788 existing records. [2025-10-20T19:35:01.066234] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-01. [2025-10-20T19:35:01.072452] [WARN ] [MAIN ] KNTK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:01.073613] [INFO ] [MAIN ] KNTK: Wiping existing primary data before full refresh. [2025-10-20T19:35:01.073712] [INFO ] [LOCAL ] KNTK: Starting deletion of primary data. [2025-10-20T19:35:01.074456] [DEBUG] [LOCAL ] KNTK: Deleted primary data file. [2025-10-20T19:35:01.074515] [INFO ] [MAIN ] KNTK: Starting full primary refresh. [2025-10-20T19:35:01.423341] [INFO ] [REMOTE ] KNTK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:01.617414] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T19:35:01.851443] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-01. [2025-10-20T19:35:02.166970] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:35:02.651418] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-02. [2025-10-20T19:35:03.456668] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-10. [2025-10-20T19:35:03.475912] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:35:04.178102] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T19:35:04.251329] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-03. [2025-10-20T19:35:05.051435] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-02. [2025-10-20T19:35:05.851405] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-07. [2025-10-20T19:35:07.177727] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-10 (3.72 seconds). [2025-10-20T19:35:07.180024] [INFO ] [REMOTE ] KNOP: Full fetch complete (14665 records). [2025-10-20T19:35:07.180106] [DEBUG] [VALIDATOR ] KNOP: Validating primary data. [2025-10-20T19:35:07.546440] [DEBUG] [VALIDATOR ] KNOP: All expected trading days are present. [2025-10-20T19:35:07.546515] [DEBUG] [VALIDATOR ] KNOP: Primary data validation passed. [2025-10-20T19:35:07.546860] [INFO ] [MAIN ] KNOP: Fetched primary data is valid. [2025-10-20T19:35:07.546918] [DEBUG] [LOCAL ] KNOP: Saving primary data. [2025-10-20T19:35:07.547392] [DEBUG] [LOCAL ] KNOP: Trimmed 14665 records to 10181 (keeping 200 days of data) [2025-10-20T19:35:07.606315] [DEBUG] [LOCAL ] KNOP: Primary data saved (10181 records). [2025-10-20T19:35:07.606440] [INFO ] [MAIN ] KNOP: Full history saved. [2025-10-20T19:35:07.606518] [INFO ] [MAIN ] Update Progress: 1203/2367 symbols completed. KNOP took 30.7s. Estimated remaining: 2h 30m 2s. Rate: 7.8 symbols/min [2025-10-20T19:35:07.716846] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-03 (3.47 seconds). [2025-10-20T19:35:07.741954] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-04. [2025-10-20T19:35:07.791525] [INFO ] [MAIN ] KNX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:07.840317] [DEBUG] [LOCAL ] KNX: Loaded 53395 existing records from primary storage. [2025-10-20T19:35:07.840716] [INFO ] [MAIN ] KNX: Loaded 53395 existing records. [2025-10-20T19:35:08.015588] [WARN ] [MAIN ] KNX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:08.015877] [INFO ] [MAIN ] KNX: Wiping existing primary data before full refresh. [2025-10-20T19:35:08.015928] [INFO ] [LOCAL ] KNX: Starting deletion of primary data. [2025-10-20T19:35:08.016277] [DEBUG] [LOCAL ] KNX: Deleted primary data file. [2025-10-20T19:35:08.016335] [INFO ] [MAIN ] KNX: Starting full primary refresh. [2025-10-20T19:35:08.379603] [INFO ] [REMOTE ] KNX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:08.379915] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-01. [2025-10-20T19:35:08.726200] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-02 (3.67 seconds). [2025-10-20T19:35:08.734339] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-03. [2025-10-20T19:35:08.897341] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T19:35:09.051438] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-05. [2025-10-20T19:35:09.175877] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-07 (3.32 seconds). [2025-10-20T19:35:09.713707] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T19:35:09.843003] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T19:35:09.851351] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-08. [2025-10-20T19:35:10.321822] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T19:35:10.651365] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-02. [2025-10-20T19:35:11.451350] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-06. [2025-10-20T19:35:11.494074] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T19:35:12.141535] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T19:35:12.181716] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:35:12.251346] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-04. [2025-10-20T19:35:13.051460] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-09. [2025-10-20T19:35:13.724504] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:35:13.851332] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-05. [2025-10-20T19:35:14.651430] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-07. [2025-10-20T19:35:15.024445] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:35:15.120245] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T19:35:15.451334] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-06. [2025-10-20T19:35:15.769579] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T19:35:16.251439] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-03. [2025-10-20T19:35:17.051433] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-08. [2025-10-20T19:35:17.148150] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:35:17.851401] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-07. [2025-10-20T19:35:18.081345] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T19:35:18.102265] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:35:18.651468] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-04. [2025-10-20T19:35:19.451415] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-09. [2025-10-20T19:35:20.251340] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-10. [2025-10-20T19:35:20.744482] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-07 (2.89 seconds). [2025-10-20T19:35:21.024765] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T19:35:21.051332] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-10. [2025-10-20T19:35:21.339192] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-10-20T19:35:21.346916] [INFO ] [REMOTE ] KMX: Full fetch complete (76383 records). [2025-10-20T19:35:21.347037] [DEBUG] [VALIDATOR ] KMX: Validating primary data. [2025-10-20T19:35:21.735334] [DEBUG] [VALIDATOR ] KMX: All expected trading days are present. [2025-10-20T19:35:21.735479] [DEBUG] [VALIDATOR ] KMX: Primary data validation passed. [2025-10-20T19:35:21.736477] [INFO ] [MAIN ] KMX: Fetched primary data is valid. [2025-10-20T19:35:21.736537] [DEBUG] [LOCAL ] KMX: Saving primary data. [2025-10-20T19:35:21.737961] [DEBUG] [LOCAL ] KMX: Trimmed 76383 records to 53284 (keeping 200 days of data) [2025-10-20T19:35:21.851377] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-08. [2025-10-20T19:35:22.046720] [DEBUG] [LOCAL ] KMX: Primary data saved (53284 records). [2025-10-20T19:35:22.046864] [INFO ] [MAIN ] KMX: Full history saved. [2025-10-20T19:35:22.046970] [INFO ] [MAIN ] Update Progress: 1204/2367 symbols completed. KMX took 45.2s. Estimated remaining: 2h 30m 0s. Rate: 7.8 symbols/min [2025-10-20T19:35:22.214076] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-04 (3.56 seconds). [2025-10-20T19:35:22.278286] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:35:22.267833] [INFO ] [MAIN ] KO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:22.282865] [DEBUG] [LOCAL ] KO: No existing data file found in primary storage. [2025-10-20T19:35:22.283215] [WARN ] [MAIN ] KO: No existing data found, performing full refresh instead. [2025-10-20T19:35:22.282770] [INFO ] [REMOTE ] KNSL: Full fetch complete (27226 records). [2025-10-20T19:35:22.283394] [INFO ] [MAIN ] KO: Wiping existing primary data before full refresh. [2025-10-20T19:35:22.283484] [DEBUG] [VALIDATOR ] KNSL: Validating primary data. [2025-10-20T19:35:22.283565] [INFO ] [LOCAL ] KO: Starting deletion of primary data. [2025-10-20T19:35:22.283769] [DEBUG] [LOCAL ] KO: No primary data file found to delete. [2025-10-20T19:35:22.283881] [INFO ] [MAIN ] KO: Starting full primary refresh. [2025-10-20T19:35:22.656457] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-05. [2025-10-20T19:35:23.016271] [INFO ] [REMOTE ] KO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:23.026174] [DEBUG] [VALIDATOR ] KNSL: All expected trading days are present. [2025-10-20T19:35:23.026224] [DEBUG] [VALIDATOR ] KNSL: Primary data validation passed. [2025-10-20T19:35:23.026757] [INFO ] [MAIN ] KNSL: Fetched primary data is valid. [2025-10-20T19:35:23.026816] [DEBUG] [LOCAL ] KNSL: Saving primary data. [2025-10-20T19:35:23.027416] [DEBUG] [LOCAL ] KNSL: Trimmed 27226 records to 18516 (keeping 200 days of data) [2025-10-20T19:35:23.140047] [DEBUG] [LOCAL ] KNSL: Primary data saved (18516 records). [2025-10-20T19:35:23.140182] [INFO ] [MAIN ] KNSL: Full history saved. [2025-10-20T19:35:23.140262] [INFO ] [MAIN ] Update Progress: 1205/2367 symbols completed. KNSL took 23.9s. Estimated remaining: 2h 29m 46s. Rate: 7.8 symbols/min [2025-10-20T19:35:23.318007] [INFO ] [MAIN ] KODK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:23.358597] [DEBUG] [LOCAL ] KODK: Loaded 44414 existing records from primary storage. [2025-10-20T19:35:23.358949] [INFO ] [MAIN ] KODK: Loaded 44414 existing records. [2025-10-20T19:35:23.456330] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-01. [2025-10-20T19:35:23.541512] [WARN ] [MAIN ] KODK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:23.541877] [INFO ] [MAIN ] KODK: Wiping existing primary data before full refresh. [2025-10-20T19:35:23.541950] [INFO ] [LOCAL ] KODK: Starting deletion of primary data. [2025-10-20T19:35:23.550896] [DEBUG] [LOCAL ] KODK: Deleted primary data file. [2025-10-20T19:35:23.566330] [INFO ] [MAIN ] KODK: Starting full primary refresh. [2025-10-20T19:35:23.601180] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T19:35:23.967161] [INFO ] [REMOTE ] KODK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:24.048216] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:35:24.251407] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-06. [2025-10-20T19:35:25.051433] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-09. [2025-10-20T19:35:25.851381] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-01. [2025-10-20T19:35:25.954627] [DEBUG] [REMOTE ] KODK: Error fetching primary data for 2025-01: 'API call returned an empty response.' [2025-10-20T19:35:25.954966] [WARN ] [REMOTE ] KODK: No data returned for month 2025-01 [2025-10-20T19:35:25.955054] [FATAL] [REMOTE ] KODK: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:35:25.955100] [FATAL] [MAIN ] KODK: Failed to fetch primary data. [2025-10-20T19:35:25.955161] [WARN ] [MAIN ] KODK: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:35:25.955232] [INFO ] [LOCAL ] KODK: Starting deletion of primary data. [2025-10-20T19:35:25.955293] [DEBUG] [LOCAL ] KODK: No primary data file found to delete. [2025-10-20T19:35:25.955329] [INFO ] [LOCAL ] KODK: Starting deletion of derivative data. [2025-10-20T19:35:25.955484] [DEBUG] [LOCAL ] KODK: Deleted derivative data file. [2025-10-20T19:35:25.955882] [INFO ] [MAIN ] Update Progress: 1206/2367 symbols completed. KODK took 2.8s. Estimated remaining: 2h 29m 34s. Rate: 7.8 symbols/min [2025-10-20T19:35:26.133838] [INFO ] [MAIN ] KOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:26.166494] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T19:35:26.199273] [DEBUG] [LOCAL ] KOF: Loaded 26961 existing records from primary storage. [2025-10-20T19:35:26.199724] [INFO ] [MAIN ] KOF: Loaded 26961 existing records. [2025-10-20T19:35:26.395552] [WARN ] [MAIN ] KOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:26.420224] [INFO ] [MAIN ] KOF: Wiping existing primary data before full refresh. [2025-10-20T19:35:26.423270] [INFO ] [LOCAL ] KOF: Starting deletion of primary data. [2025-10-20T19:35:26.448361] [DEBUG] [LOCAL ] KOF: Deleted primary data file. [2025-10-20T19:35:26.448920] [INFO ] [MAIN ] KOF: Starting full primary refresh. [2025-10-20T19:35:26.450924] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-01 (2.99 seconds). [2025-10-20T19:35:26.497009] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T19:35:26.651345] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-07. [2025-10-20T19:35:26.858556] [INFO ] [REMOTE ] KOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:27.451382] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-10. [2025-10-20T19:35:28.251393] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-01. [2025-10-20T19:35:28.451222] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T19:35:29.009508] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T19:35:29.016841] [INFO ] [REMOTE ] KNTK: Full fetch complete (65669 records). [2025-10-20T19:35:29.016962] [DEBUG] [VALIDATOR ] KNTK: Validating primary data. [2025-10-20T19:35:29.059046] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-02. [2025-10-20T19:35:29.414044] [DEBUG] [VALIDATOR ] KNTK: All expected trading days are present. [2025-10-20T19:35:29.414182] [DEBUG] [VALIDATOR ] KNTK: Primary data validation passed. [2025-10-20T19:35:29.415552] [INFO ] [MAIN ] KNTK: Fetched primary data is valid. [2025-10-20T19:35:29.415619] [DEBUG] [LOCAL ] KNTK: Saving primary data. [2025-10-20T19:35:29.417119] [DEBUG] [LOCAL ] KNTK: Trimmed 65669 records to 47220 (keeping 200 days of data) [2025-10-20T19:35:29.730733] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T19:35:29.749906] [DEBUG] [LOCAL ] KNTK: Primary data saved (47220 records). [2025-10-20T19:35:29.750047] [INFO ] [MAIN ] KNTK: Full history saved. [2025-10-20T19:35:29.750816] [INFO ] [MAIN ] Update Progress: 1207/2367 symbols completed. KNTK took 29.1s. Estimated remaining: 2h 29m 22s. Rate: 7.8 symbols/min [2025-10-20T19:35:29.856522] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-08. [2025-10-20T19:35:29.940529] [INFO ] [MAIN ] KOP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:29.960760] [DEBUG] [LOCAL ] KOP: Loaded 17674 existing records from primary storage. [2025-10-20T19:35:29.961044] [INFO ] [MAIN ] KOP: Loaded 17674 existing records. [2025-10-20T19:35:30.160423] [WARN ] [MAIN ] KOP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:30.160846] [INFO ] [MAIN ] KOP: Wiping existing primary data before full refresh. [2025-10-20T19:35:30.160914] [INFO ] [LOCAL ] KOP: Starting deletion of primary data. [2025-10-20T19:35:30.161179] [DEBUG] [LOCAL ] KOP: Deleted primary data file. [2025-10-20T19:35:30.161245] [INFO ] [MAIN ] KOP: Starting full primary refresh. [2025-10-20T19:35:30.578720] [INFO ] [REMOTE ] KOP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:30.651351] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-01. [2025-10-20T19:35:31.332388] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-02 (2.27 seconds). [2025-10-20T19:35:31.451385] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-02. [2025-10-20T19:35:32.246883] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:35:32.251329] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-02. [2025-10-20T19:35:32.439815] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-08 (2.58 seconds). [2025-10-20T19:35:32.849969] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T19:35:32.921451] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T19:35:33.051382] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-03. [2025-10-20T19:35:33.851417] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-09. [2025-10-20T19:35:34.651407] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-03. [2025-10-20T19:35:35.451382] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-03. [2025-10-20T19:35:35.655164] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:35:35.687883] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T19:35:35.737994] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-03 (2.69 seconds). [2025-10-20T19:35:36.251425] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-10. [2025-10-20T19:35:36.442693] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T19:35:37.053120] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-04. [2025-10-20T19:35:37.055606] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T19:35:37.063899] [INFO ] [REMOTE ] KNX: Full fetch complete (76289 records). [2025-10-20T19:35:37.064048] [DEBUG] [VALIDATOR ] KNX: Validating primary data. [2025-10-20T19:35:37.452853] [DEBUG] [VALIDATOR ] KNX: All expected trading days are present. [2025-10-20T19:35:37.452932] [DEBUG] [VALIDATOR ] KNX: Primary data validation passed. [2025-10-20T19:35:37.453990] [INFO ] [MAIN ] KNX: Fetched primary data is valid. [2025-10-20T19:35:37.454054] [DEBUG] [LOCAL ] KNX: Saving primary data. [2025-10-20T19:35:37.455448] [DEBUG] [LOCAL ] KNX: Trimmed 76289 records to 52639 (keeping 200 days of data) [2025-10-20T19:35:37.800033] [DEBUG] [LOCAL ] KNX: Primary data saved (52639 records). [2025-10-20T19:35:37.800201] [INFO ] [MAIN ] KNX: Full history saved. [2025-10-20T19:35:37.800287] [INFO ] [MAIN ] Update Progress: 1208/2367 symbols completed. KNX took 30.2s. Estimated remaining: 2h 29m 15s. Rate: 7.8 symbols/min [2025-10-20T19:35:37.864116] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-04. [2025-10-20T19:35:38.033083] [INFO ] [MAIN ] KORE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:38.039712] [DEBUG] [LOCAL ] KORE: Loaded 2191 existing records from primary storage. [2025-10-20T19:35:38.039997] [INFO ] [MAIN ] KORE: Loaded 2191 existing records. [2025-10-20T19:35:38.231694] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T19:35:38.345638] [WARN ] [MAIN ] KORE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:38.345976] [INFO ] [MAIN ] KORE: Wiping existing primary data before full refresh. [2025-10-20T19:35:38.346067] [INFO ] [LOCAL ] KORE: Starting deletion of primary data. [2025-10-20T19:35:38.346303] [DEBUG] [LOCAL ] KORE: Deleted primary data file. [2025-10-20T19:35:38.346383] [INFO ] [MAIN ] KORE: Starting full primary refresh. [2025-10-20T19:35:38.658424] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-05. [2025-10-20T19:35:38.794986] [INFO ] [REMOTE ] KORE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:39.451383] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-01. [2025-10-20T19:35:39.784929] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T19:35:40.251418] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-04. [2025-10-20T19:35:40.385611] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T19:35:40.491322] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-04 (2.63 seconds). [2025-10-20T19:35:41.051372] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-02. [2025-10-20T19:35:41.357147] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-10-20T19:35:41.384319] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T19:35:41.851416] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-05. [2025-10-20T19:35:42.651341] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-06. [2025-10-20T19:35:43.004771] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T19:35:43.197759] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T19:35:43.451412] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-03. [2025-10-20T19:35:43.861597] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-10-20T19:35:44.251351] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-04. [2025-10-20T19:35:44.700039] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T19:35:45.051383] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-05. [2025-10-20T19:35:45.851374] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-06. [2025-10-20T19:35:46.651359] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-07. [2025-10-20T19:35:47.042963] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T19:35:47.288407] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-05 (2.24 seconds). [2025-10-20T19:35:47.451357] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-07. [2025-10-20T19:35:47.552933] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:35:48.251369] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-08. [2025-10-20T19:35:48.656984] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T19:35:49.051355] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-06. [2025-10-20T19:35:49.282845] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T19:35:49.851363] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-09. [2025-10-20T19:35:50.651432] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-05. [2025-10-20T19:35:50.827475] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T19:35:51.074567] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T19:35:51.456537] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-06. [2025-10-20T19:35:51.469475] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-06 (2.42 seconds). [2025-10-20T19:35:51.887815] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T19:35:52.251351] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-10. [2025-10-20T19:35:53.051358] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-07. [2025-10-20T19:35:53.230957] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:35:53.234029] [INFO ] [REMOTE ] KOP: Full fetch complete (25442 records). [2025-10-20T19:35:53.234121] [DEBUG] [VALIDATOR ] KOP: Validating primary data. [2025-10-20T19:35:53.620036] [DEBUG] [VALIDATOR ] KOP: All expected trading days are present. [2025-10-20T19:35:53.620440] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T19:35:53.620525] [DEBUG] [VALIDATOR ] KOP: Primary data validation passed. [2025-10-20T19:35:53.622709] [INFO ] [MAIN ] KOP: Fetched primary data is valid. [2025-10-20T19:35:53.623057] [DEBUG] [LOCAL ] KOP: Saving primary data. [2025-10-20T19:35:53.623750] [DEBUG] [LOCAL ] KOP: Trimmed 25442 records to 17454 (keeping 200 days of data) [2025-10-20T19:35:53.732192] [DEBUG] [LOCAL ] KOP: Primary data saved (17454 records). [2025-10-20T19:35:53.732326] [INFO ] [MAIN ] KOP: Full history saved. [2025-10-20T19:35:53.732418] [INFO ] [MAIN ] Update Progress: 1209/2367 symbols completed. KOP took 24.0s. Estimated remaining: 2h 29m 15s. Rate: 7.8 symbols/min [2025-10-20T19:35:53.856602] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-08. [2025-10-20T19:35:53.902409] [INFO ] [MAIN ] KOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:53.954552] [DEBUG] [LOCAL ] KOS: Loaded 53875 existing records from primary storage. [2025-10-20T19:35:53.954899] [INFO ] [MAIN ] KOS: Loaded 53875 existing records. [2025-10-20T19:35:54.128092] [WARN ] [MAIN ] KOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:54.128353] [INFO ] [MAIN ] KOS: Wiping existing primary data before full refresh. [2025-10-20T19:35:54.128400] [INFO ] [LOCAL ] KOS: Starting deletion of primary data. [2025-10-20T19:35:54.128693] [DEBUG] [LOCAL ] KOS: Deleted primary data file. [2025-10-20T19:35:54.128750] [INFO ] [MAIN ] KOS: Starting full primary refresh. [2025-10-20T19:35:54.491897] [INFO ] [REMOTE ] KOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:54.651354] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-01. [2025-10-20T19:35:55.005715] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T19:35:55.451396] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-07. [2025-10-20T19:35:56.251355] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-08. [2025-10-20T19:35:56.539817] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-10-20T19:35:56.695550] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-01 (2.04 seconds). [2025-10-20T19:35:57.051433] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-02. [2025-10-20T19:35:57.851369] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-09. [2025-10-20T19:35:57.922314] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-07 (2.47 seconds). [2025-10-20T19:35:58.657278] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-08. [2025-10-20T19:35:58.682397] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T19:35:59.078202] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T19:35:59.451447] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-03. [2025-10-20T19:36:00.251423] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-10. [2025-10-20T19:36:01.051348] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-09. [2025-10-20T19:36:01.620608] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-03 (2.17 seconds). [2025-10-20T19:36:01.655414] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T19:36:01.659340] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-08 (3.00 seconds). [2025-10-20T19:36:01.851631] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-04. [2025-10-20T19:36:02.118600] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T19:36:02.122974] [INFO ] [REMOTE ] KOF: Full fetch complete (38180 records). [2025-10-20T19:36:02.123047] [DEBUG] [VALIDATOR ] KOF: Validating primary data. [2025-10-20T19:36:02.497876] [DEBUG] [VALIDATOR ] KOF: All expected trading days are present. [2025-10-20T19:36:02.497965] [DEBUG] [VALIDATOR ] KOF: Primary data validation passed. [2025-10-20T19:36:02.498741] [INFO ] [MAIN ] KOF: Fetched primary data is valid. [2025-10-20T19:36:02.498803] [DEBUG] [LOCAL ] KOF: Saving primary data. [2025-10-20T19:36:02.499789] [DEBUG] [LOCAL ] KOF: Trimmed 38180 records to 26742 (keeping 200 days of data) [2025-10-20T19:36:02.653013] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-09. [2025-10-20T19:36:02.664348] [DEBUG] [LOCAL ] KOF: Primary data saved (26742 records). [2025-10-20T19:36:02.665371] [INFO ] [MAIN ] KOF: Full history saved. [2025-10-20T19:36:02.665463] [INFO ] [MAIN ] Update Progress: 1210/2367 symbols completed. KOF took 36.7s. Estimated remaining: 2h 29m 8s. Rate: 7.8 symbols/min [2025-10-20T19:36:02.840254] [INFO ] [MAIN ] KR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:02.889274] [DEBUG] [LOCAL ] KR: Loaded 54460 existing records from primary storage. [2025-10-20T19:36:02.889580] [INFO ] [MAIN ] KR: Loaded 54460 existing records. [2025-10-20T19:36:03.069051] [WARN ] [MAIN ] KR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:03.069257] [INFO ] [MAIN ] KR: Wiping existing primary data before full refresh. [2025-10-20T19:36:03.069313] [INFO ] [LOCAL ] KR: Starting deletion of primary data. [2025-10-20T19:36:03.069508] [DEBUG] [LOCAL ] KR: Deleted primary data file. [2025-10-20T19:36:03.069545] [INFO ] [MAIN ] KR: Starting full primary refresh. [2025-10-20T19:36:03.439881] [INFO ] [REMOTE ] KR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:03.451379] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-10. [2025-10-20T19:36:03.530917] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:36:03.945838] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T19:36:03.947217] [INFO ] [REMOTE ] KORE: Full fetch complete (4423 records). [2025-10-20T19:36:03.947301] [DEBUG] [VALIDATOR ] KORE: Validating primary data. [2025-10-20T19:36:04.256624] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-01. [2025-10-20T19:36:04.314365] [DEBUG] [VALIDATOR ] KORE: All expected trading days are present. [2025-10-20T19:36:04.314683] [DEBUG] [VALIDATOR ] KORE: Primary data validation passed. [2025-10-20T19:36:04.314917] [INFO ] [MAIN ] KORE: Fetched primary data is valid. [2025-10-20T19:36:04.314964] [DEBUG] [LOCAL ] KORE: Saving primary data. [2025-10-20T19:36:04.315453] [DEBUG] [LOCAL ] KORE: Trimmed 4423 records to 2176 (keeping 200 days of data) [2025-10-20T19:36:04.329119] [DEBUG] [LOCAL ] KORE: Primary data saved (2176 records). [2025-10-20T19:36:04.329255] [INFO ] [MAIN ] KORE: Full history saved. [2025-10-20T19:36:04.329337] [INFO ] [MAIN ] Update Progress: 1211/2367 symbols completed. KORE took 26.5s. Estimated remaining: 2h 28m 55s. Rate: 7.8 symbols/min [2025-10-20T19:36:04.496838] [INFO ] [MAIN ] KRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:04.540787] [DEBUG] [LOCAL ] KRC: Loaded 48982 existing records from primary storage. [2025-10-20T19:36:04.541119] [INFO ] [MAIN ] KRC: Loaded 48982 existing records. [2025-10-20T19:36:04.720105] [WARN ] [MAIN ] KRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:04.721064] [INFO ] [MAIN ] KRC: Wiping existing primary data before full refresh. [2025-10-20T19:36:04.721240] [INFO ] [LOCAL ] KRC: Starting deletion of primary data. [2025-10-20T19:36:04.721474] [DEBUG] [LOCAL ] KRC: Deleted primary data file. [2025-10-20T19:36:04.721530] [INFO ] [MAIN ] KRC: Starting full primary refresh. [2025-10-20T19:36:05.074145] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-05. [2025-10-20T19:36:05.087578] [INFO ] [REMOTE ] KRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:05.224357] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-09 (2.57 seconds). [2025-10-20T19:36:05.851349] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-10. [2025-10-20T19:36:06.165831] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T19:36:06.651393] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-02. [2025-10-20T19:36:06.763157] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:36:06.794014] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:36:06.801808] [INFO ] [REMOTE ] KO: Full fetch complete (77218 records). [2025-10-20T19:36:06.801927] [DEBUG] [VALIDATOR ] KO: Validating primary data. [2025-10-20T19:36:06.995988] [WARN ] [VALIDATOR ] KO: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T19:36:06.996791] [FATAL] [MAIN ] KO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:36:06.996896] [WARN ] [MAIN ] KO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:36:06.996987] [INFO ] [LOCAL ] KO: Starting deletion of primary data. [2025-10-20T19:36:06.997068] [DEBUG] [LOCAL ] KO: No primary data file found to delete. [2025-10-20T19:36:06.997106] [INFO ] [LOCAL ] KO: Starting deletion of derivative data. [2025-10-20T19:36:06.997145] [DEBUG] [LOCAL ] KO: No derivative data file found to delete. [2025-10-20T19:36:06.997202] [INFO ] [MAIN ] Update Progress: 1212/2367 symbols completed. KO took 45.0s. Estimated remaining: 2h 28m 42s. Rate: 7.8 symbols/min [2025-10-20T19:36:07.163432] [INFO ] [MAIN ] KREF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:07.198225] [DEBUG] [LOCAL ] KREF: Loaded 38479 existing records from primary storage. [2025-10-20T19:36:07.198527] [INFO ] [MAIN ] KREF: Loaded 38479 existing records. [2025-10-20T19:36:07.373055] [WARN ] [MAIN ] KREF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:07.373266] [INFO ] [MAIN ] KREF: Wiping existing primary data before full refresh. [2025-10-20T19:36:07.373323] [INFO ] [LOCAL ] KREF: Starting deletion of primary data. [2025-10-20T19:36:07.373521] [DEBUG] [LOCAL ] KREF: Deleted primary data file. [2025-10-20T19:36:07.373585] [INFO ] [MAIN ] KREF: Starting full primary refresh. [2025-10-20T19:36:07.456686] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-01. [2025-10-20T19:36:07.727301] [INFO ] [REMOTE ] KREF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:08.251395] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-06. [2025-10-20T19:36:08.438986] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:36:09.051404] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-03. [2025-10-20T19:36:09.094580] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:36:09.861768] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-01. [2025-10-20T19:36:09.864628] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T19:36:10.651345] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-07. [2025-10-20T19:36:10.848914] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T19:36:11.144763] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T19:36:11.451414] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-02. [2025-10-20T19:36:12.251351] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-04. [2025-10-20T19:36:12.570973] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T19:36:12.981408] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T19:36:13.051364] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-03. [2025-10-20T19:36:13.851365] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-08. [2025-10-20T19:36:14.287005] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T19:36:14.651410] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-05. [2025-10-20T19:36:14.807885] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T19:36:15.451411] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-04. [2025-10-20T19:36:16.024996] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-08 (2.17 seconds). [2025-10-20T19:36:16.251388] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-09. [2025-10-20T19:36:17.051438] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-02. [2025-10-20T19:36:17.567054] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-05 (2.92 seconds). [2025-10-20T19:36:17.611833] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T19:36:17.851363] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-06. [2025-10-20T19:36:18.320604] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:36:18.431638] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-10-20T19:36:18.651353] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-10. [2025-10-20T19:36:19.451358] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-03. [2025-10-20T19:36:19.744542] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:36:20.232675] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T19:36:20.240512] [INFO ] [REMOTE ] KOS: Full fetch complete (77025 records). [2025-10-20T19:36:20.240715] [DEBUG] [VALIDATOR ] KOS: Validating primary data. [2025-10-20T19:36:20.256564] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-05. [2025-10-20T19:36:20.662061] [DEBUG] [VALIDATOR ] KOS: All expected trading days are present. [2025-10-20T19:36:20.662188] [DEBUG] [VALIDATOR ] KOS: Primary data validation passed. [2025-10-20T19:36:20.663427] [INFO ] [MAIN ] KOS: Fetched primary data is valid. [2025-10-20T19:36:20.663578] [DEBUG] [LOCAL ] KOS: Saving primary data. [2025-10-20T19:36:20.665297] [DEBUG] [LOCAL ] KOS: Trimmed 77025 records to 53095 (keeping 200 days of data) [2025-10-20T19:36:20.916200] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:36:21.022125] [DEBUG] [LOCAL ] KOS: Primary data saved (53095 records). [2025-10-20T19:36:21.022277] [INFO ] [MAIN ] KOS: Full history saved. [2025-10-20T19:36:21.023102] [INFO ] [MAIN ] Update Progress: 1213/2367 symbols completed. KOS took 27.3s. Estimated remaining: 2h 28m 41s. Rate: 7.8 symbols/min [2025-10-20T19:36:21.054980] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-07. [2025-10-20T19:36:21.195790] [INFO ] [MAIN ] KRG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:21.242704] [DEBUG] [LOCAL ] KRG: Loaded 50023 existing records from primary storage. [2025-10-20T19:36:21.243062] [INFO ] [MAIN ] KRG: Loaded 50023 existing records. [2025-10-20T19:36:21.447095] [WARN ] [MAIN ] KRG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:21.447368] [INFO ] [MAIN ] KRG: Wiping existing primary data before full refresh. [2025-10-20T19:36:21.447415] [INFO ] [LOCAL ] KRG: Starting deletion of primary data. [2025-10-20T19:36:21.447632] [DEBUG] [LOCAL ] KRG: Deleted primary data file. [2025-10-20T19:36:21.447726] [INFO ] [MAIN ] KRG: Starting full primary refresh. [2025-10-20T19:36:21.803983] [INFO ] [REMOTE ] KRG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:21.839731] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T19:36:21.851336] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-01. [2025-10-20T19:36:22.651363] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-06. [2025-10-20T19:36:23.125771] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T19:36:23.451412] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-04. [2025-10-20T19:36:23.539067] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:36:24.235303] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:36:24.251349] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-08. [2025-10-20T19:36:24.971216] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:36:25.051391] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-02. [2025-10-20T19:36:25.851360] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-05. [2025-10-20T19:36:26.156406] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T19:36:26.557808] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T19:36:26.651413] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-03. [2025-10-20T19:36:27.343590] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T19:36:27.451379] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-07. [2025-10-20T19:36:28.251352] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-09. [2025-10-20T19:36:28.384120] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:36:29.051472] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-06. [2025-10-20T19:36:29.137679] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T19:36:29.851344] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-08. [2025-10-20T19:36:30.185555] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T19:36:30.517526] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:36:30.651390] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-07. [2025-10-20T19:36:31.451457] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-04. [2025-10-20T19:36:31.515482] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:36:32.191744] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:36:32.251375] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-08. [2025-10-20T19:36:33.051398] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-10. [2025-10-20T19:36:33.273245] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T19:36:33.510965] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T19:36:33.851355] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-09. [2025-10-20T19:36:33.923187] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T19:36:33.931181] [INFO ] [REMOTE ] KR: Full fetch complete (77843 records). [2025-10-20T19:36:33.931353] [DEBUG] [VALIDATOR ] KR: Validating primary data. [2025-10-20T19:36:34.350361] [DEBUG] [VALIDATOR ] KR: All expected trading days are present. [2025-10-20T19:36:34.350481] [DEBUG] [VALIDATOR ] KR: Primary data validation passed. [2025-10-20T19:36:34.351920] [INFO ] [MAIN ] KR: Fetched primary data is valid. [2025-10-20T19:36:34.351988] [DEBUG] [LOCAL ] KR: Saving primary data. [2025-10-20T19:36:34.354121] [DEBUG] [LOCAL ] KR: Trimmed 77843 records to 53681 (keeping 200 days of data) [2025-10-20T19:36:34.661584] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-09. [2025-10-20T19:36:34.717816] [DEBUG] [LOCAL ] KR: Primary data saved (53681 records). [2025-10-20T19:36:34.718021] [INFO ] [MAIN ] KR: Full history saved. [2025-10-20T19:36:34.718375] [INFO ] [MAIN ] Update Progress: 1214/2367 symbols completed. KR took 32.1s. Estimated remaining: 2h 28m 38s. Rate: 7.8 symbols/min [2025-10-20T19:36:34.898970] [INFO ] [MAIN ] KRMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:34.899196] [DEBUG] [LOCAL ] KRMN: No existing data file found in primary storage. [2025-10-20T19:36:34.899233] [WARN ] [MAIN ] KRMN: No existing data found, performing full refresh instead. [2025-10-20T19:36:34.899359] [INFO ] [MAIN ] KRMN: Wiping existing primary data before full refresh. [2025-10-20T19:36:34.899399] [INFO ] [LOCAL ] KRMN: Starting deletion of primary data. [2025-10-20T19:36:34.899439] [DEBUG] [LOCAL ] KRMN: No primary data file found to delete. [2025-10-20T19:36:34.899472] [INFO ] [MAIN ] KRMN: Starting full primary refresh. [2025-10-20T19:36:35.280291] [INFO ] [REMOTE ] KRMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:35.324017] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:36:35.451388] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-10. [2025-10-20T19:36:36.251348] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-05. [2025-10-20T19:36:36.368773] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:36:36.932906] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T19:36:36.939188] [INFO ] [REMOTE ] KREF: Full fetch complete (52830 records). [2025-10-20T19:36:36.939330] [DEBUG] [VALIDATOR ] KREF: Validating primary data. [2025-10-20T19:36:37.056481] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-01. [2025-10-20T19:36:37.336201] [DEBUG] [REMOTE ] KRMN: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:36:37.349115] [DEBUG] [VALIDATOR ] KREF: All expected trading days are present. [2025-10-20T19:36:37.349189] [DEBUG] [VALIDATOR ] KREF: Primary data validation passed. [2025-10-20T19:36:37.350130] [INFO ] [MAIN ] KREF: Fetched primary data is valid. [2025-10-20T19:36:37.350207] [WARN ] [REMOTE ] KRMN: No data returned for month 2025-01 [2025-10-20T19:36:37.350387] [DEBUG] [LOCAL ] KREF: Saving primary data. [2025-10-20T19:36:37.350340] [FATAL] [REMOTE ] KRMN: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:36:37.350610] [FATAL] [MAIN ] KRMN: Failed to fetch primary data. [2025-10-20T19:36:37.350705] [WARN ] [MAIN ] KRMN: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:36:37.350782] [INFO ] [LOCAL ] KRMN: Starting deletion of primary data. [2025-10-20T19:36:37.351027] [DEBUG] [LOCAL ] KRMN: No primary data file found to delete. [2025-10-20T19:36:37.351089] [INFO ] [LOCAL ] KRMN: Starting deletion of derivative data. [2025-10-20T19:36:37.351216] [DEBUG] [LOCAL ] KRMN: No derivative data file found to delete. [2025-10-20T19:36:37.351368] [INFO ] [MAIN ] Update Progress: 1215/2367 symbols completed. KRMN took 2.6s. Estimated remaining: 2h 28m 26s. Rate: 7.8 symbols/min [2025-10-20T19:36:37.427464] [DEBUG] [LOCAL ] KREF: Trimmed 52830 records to 38036 (keeping 200 days of data) [2025-10-20T19:36:37.629658] [INFO ] [MAIN ] KRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:37.782308] [DEBUG] [LOCAL ] KREF: Primary data saved (38036 records). [2025-10-20T19:36:37.782876] [DEBUG] [LOCAL ] KRO: Loaded 26887 existing records from primary storage. [2025-10-20T19:36:37.783147] [INFO ] [MAIN ] KREF: Full history saved. [2025-10-20T19:36:37.783487] [INFO ] [MAIN ] Update Progress: 1216/2367 symbols completed. KREF took 30.8s. Estimated remaining: 2h 28m 11s. Rate: 7.8 symbols/min [2025-10-20T19:36:37.783694] [INFO ] [MAIN ] KRO: Loaded 26887 existing records. [2025-10-20T19:36:37.852827] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-10. [2025-10-20T19:36:38.120234] [WARN ] [MAIN ] KRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:38.132522] [INFO ] [MAIN ] KRO: Wiping existing primary data before full refresh. [2025-10-20T19:36:38.167550] [INFO ] [LOCAL ] KRO: Starting deletion of primary data. [2025-10-20T19:36:38.152963] [INFO ] [MAIN ] KRP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:38.188504] [DEBUG] [LOCAL ] KRO: Deleted primary data file. [2025-10-20T19:36:38.192471] [INFO ] [MAIN ] KRO: Starting full primary refresh. [2025-10-20T19:36:38.224507] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T19:36:38.459740] [DEBUG] [LOCAL ] KRP: Loaded 36833 existing records from primary storage. [2025-10-20T19:36:38.498862] [INFO ] [MAIN ] KRP: Loaded 36833 existing records. [2025-10-20T19:36:38.663939] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-06. [2025-10-20T19:36:38.733013] [INFO ] [REMOTE ] KRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:38.810008] [WARN ] [MAIN ] KRP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:38.810249] [INFO ] [MAIN ] KRP: Wiping existing primary data before full refresh. [2025-10-20T19:36:38.810297] [INFO ] [LOCAL ] KRP: Starting deletion of primary data. [2025-10-20T19:36:38.810476] [DEBUG] [LOCAL ] KRP: Deleted primary data file. [2025-10-20T19:36:38.810512] [INFO ] [MAIN ] KRP: Starting full primary refresh. [2025-10-20T19:36:39.175477] [INFO ] [REMOTE ] KRP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:39.451370] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-01. [2025-10-20T19:36:39.520585] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:36:39.529621] [INFO ] [REMOTE ] KRC: Full fetch complete (70047 records). [2025-10-20T19:36:39.529843] [DEBUG] [VALIDATOR ] KRC: Validating primary data. [2025-10-20T19:36:39.963571] [DEBUG] [VALIDATOR ] KRC: All expected trading days are present. [2025-10-20T19:36:39.963695] [DEBUG] [VALIDATOR ] KRC: Primary data validation passed. [2025-10-20T19:36:39.964726] [INFO ] [MAIN ] KRC: Fetched primary data is valid. [2025-10-20T19:36:39.964788] [DEBUG] [LOCAL ] KRC: Saving primary data. [2025-10-20T19:36:39.966286] [DEBUG] [LOCAL ] KRC: Trimmed 70047 records to 48217 (keeping 200 days of data) [2025-10-20T19:36:40.253229] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-01. [2025-10-20T19:36:40.268771] [DEBUG] [LOCAL ] KRC: Primary data saved (48217 records). [2025-10-20T19:36:40.268917] [INFO ] [MAIN ] KRC: Full history saved. [2025-10-20T19:36:40.269053] [INFO ] [MAIN ] Update Progress: 1217/2367 symbols completed. KRC took 35.9s. Estimated remaining: 2h 27m 59s. Rate: 7.8 symbols/min [2025-10-20T19:36:40.436805] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T19:36:40.481047] [INFO ] [MAIN ] KSS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:40.532562] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T19:36:40.556744] [DEBUG] [LOCAL ] KSS: Loaded 54372 existing records from primary storage. [2025-10-20T19:36:40.557097] [INFO ] [MAIN ] KSS: Loaded 54372 existing records. [2025-10-20T19:36:40.731382] [WARN ] [MAIN ] KSS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:40.731597] [INFO ] [MAIN ] KSS: Wiping existing primary data before full refresh. [2025-10-20T19:36:40.731669] [INFO ] [LOCAL ] KSS: Starting deletion of primary data. [2025-10-20T19:36:40.731903] [DEBUG] [LOCAL ] KSS: Deleted primary data file. [2025-10-20T19:36:40.731960] [INFO ] [MAIN ] KSS: Starting full primary refresh. [2025-10-20T19:36:41.062972] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-02. [2025-10-20T19:36:41.082713] [INFO ] [REMOTE ] KSS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:41.844510] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T19:36:41.852529] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-07. [2025-10-20T19:36:41.894552] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T19:36:42.651382] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-02. [2025-10-20T19:36:43.451411] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-03. [2025-10-20T19:36:43.495189] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T19:36:44.118378] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:36:44.251362] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-03. [2025-10-20T19:36:44.568724] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:36:45.051384] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-04. [2025-10-20T19:36:45.851408] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-08. [2025-10-20T19:36:46.097339] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T19:36:46.255188] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:36:46.651348] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-01. [2025-10-20T19:36:47.451373] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-05. [2025-10-20T19:36:47.486048] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T19:36:48.251393] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-09. [2025-10-20T19:36:48.611606] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T19:36:48.651504] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T19:36:49.051381] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-06. [2025-10-20T19:36:49.599600] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:36:49.851377] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-02. [2025-10-20T19:36:50.071940] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T19:36:50.651375] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-10. [2025-10-20T19:36:51.451394] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-07. [2025-10-20T19:36:51.749819] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T19:36:52.245217] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T19:36:52.252691] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-03. [2025-10-20T19:36:52.255084] [INFO ] [REMOTE ] KRG: Full fetch complete (71323 records). [2025-10-20T19:36:52.255365] [DEBUG] [VALIDATOR ] KRG: Validating primary data. [2025-10-20T19:36:52.674039] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T19:36:52.674393] [DEBUG] [VALIDATOR ] KRG: All expected trading days are present. [2025-10-20T19:36:52.680091] [DEBUG] [VALIDATOR ] KRG: Primary data validation passed. [2025-10-20T19:36:52.681701] [INFO ] [MAIN ] KRG: Fetched primary data is valid. [2025-10-20T19:36:52.681789] [DEBUG] [LOCAL ] KRG: Saving primary data. [2025-10-20T19:36:52.683420] [DEBUG] [LOCAL ] KRG: Trimmed 71323 records to 49298 (keeping 200 days of data) [2025-10-20T19:36:52.977464] [DEBUG] [LOCAL ] KRG: Primary data saved (49298 records). [2025-10-20T19:36:52.977606] [INFO ] [MAIN ] KRG: Full history saved. [2025-10-20T19:36:52.978396] [INFO ] [MAIN ] Update Progress: 1218/2367 symbols completed. KRG took 32.0s. Estimated remaining: 2h 27m 56s. Rate: 7.8 symbols/min [2025-10-20T19:36:53.056716] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-04. [2025-10-20T19:36:53.152293] [INFO ] [MAIN ] KT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:53.197344] [DEBUG] [LOCAL ] KT: Loaded 46205 existing records from primary storage. [2025-10-20T19:36:53.197673] [INFO ] [MAIN ] KT: Loaded 46205 existing records. [2025-10-20T19:36:53.378544] [WARN ] [MAIN ] KT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:53.378811] [INFO ] [MAIN ] KT: Wiping existing primary data before full refresh. [2025-10-20T19:36:53.378861] [INFO ] [LOCAL ] KT: Starting deletion of primary data. [2025-10-20T19:36:53.379054] [DEBUG] [LOCAL ] KT: Deleted primary data file. [2025-10-20T19:36:53.379110] [INFO ] [MAIN ] KT: Starting full primary refresh. [2025-10-20T19:36:53.742564] [INFO ] [REMOTE ] KT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:53.851381] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-01. [2025-10-20T19:36:54.394567] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T19:36:54.651398] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-08. [2025-10-20T19:36:54.700816] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:36:55.432013] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:36:55.451338] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-05. [2025-10-20T19:36:55.966354] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:36:56.251432] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-09. [2025-10-20T19:36:56.967812] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T19:36:57.051403] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-02. [2025-10-20T19:36:57.071302] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T19:36:57.851402] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-10. [2025-10-20T19:36:58.626282] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:36:58.651334] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-03. [2025-10-20T19:36:59.276799] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T19:36:59.281007] [INFO ] [REMOTE ] KRO: Full fetch complete (35287 records). [2025-10-20T19:36:59.281093] [DEBUG] [VALIDATOR ] KRO: Validating primary data. [2025-10-20T19:36:59.454445] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-06. [2025-10-20T19:36:59.659630] [DEBUG] [VALIDATOR ] KRO: All expected trading days are present. [2025-10-20T19:36:59.659765] [DEBUG] [VALIDATOR ] KRO: Primary data validation passed. [2025-10-20T19:36:59.660516] [INFO ] [MAIN ] KRO: Fetched primary data is valid. [2025-10-20T19:36:59.660558] [DEBUG] [LOCAL ] KRO: Saving primary data. [2025-10-20T19:36:59.661366] [DEBUG] [LOCAL ] KRO: Trimmed 35287 records to 26644 (keeping 200 days of data) [2025-10-20T19:36:59.812659] [DEBUG] [LOCAL ] KRO: Primary data saved (26644 records). [2025-10-20T19:36:59.812807] [INFO ] [MAIN ] KRO: Full history saved. [2025-10-20T19:36:59.812887] [INFO ] [MAIN ] Update Progress: 1219/2367 symbols completed. KRO took 22.5s. Estimated remaining: 2h 27m 47s. Rate: 7.8 symbols/min [2025-10-20T19:36:59.988183] [INFO ] [MAIN ] KTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:00.028697] [DEBUG] [LOCAL ] KTB: Loaded 41064 existing records from primary storage. [2025-10-20T19:37:00.029015] [INFO ] [MAIN ] KTB: Loaded 41064 existing records. [2025-10-20T19:37:00.215611] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:37:00.235022] [WARN ] [MAIN ] KTB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:00.244355] [INFO ] [MAIN ] KTB: Wiping existing primary data before full refresh. [2025-10-20T19:37:00.244753] [INFO ] [LOCAL ] KTB: Starting deletion of primary data. [2025-10-20T19:37:00.245066] [DEBUG] [LOCAL ] KTB: Deleted primary data file. [2025-10-20T19:37:00.245123] [INFO ] [MAIN ] KTB: Starting full primary refresh. [2025-10-20T19:37:00.257677] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-04. [2025-10-20T19:37:00.599858] [INFO ] [REMOTE ] KTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:00.623128] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T19:37:01.051380] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-01. [2025-10-20T19:37:01.851425] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-07. [2025-10-20T19:37:02.417228] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:37:02.568388] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-04 (2.31 seconds). [2025-10-20T19:37:02.651373] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-05. [2025-10-20T19:37:03.451390] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-02. [2025-10-20T19:37:03.741998] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:37:04.251443] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-08. [2025-10-20T19:37:04.735533] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T19:37:04.828753] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T19:37:05.051411] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-03. [2025-10-20T19:37:05.236880] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:37:05.851373] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-06. [2025-10-20T19:37:06.362923] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T19:37:06.651411] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-04. [2025-10-20T19:37:07.451347] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-09. [2025-10-20T19:37:07.788198] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-10-20T19:37:08.224386] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T19:37:08.251359] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-05. [2025-10-20T19:37:08.522023] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T19:37:09.051409] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-07. [2025-10-20T19:37:09.785299] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:37:09.851340] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-06. [2025-10-20T19:37:10.651388] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-10. [2025-10-20T19:37:10.984639] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T19:37:11.022840] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T19:37:11.451426] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-04. [2025-10-20T19:37:12.189597] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T19:37:12.195402] [INFO ] [REMOTE ] KRP: Full fetch complete (55037 records). [2025-10-20T19:37:12.195599] [DEBUG] [VALIDATOR ] KRP: Validating primary data. [2025-10-20T19:37:12.262991] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-08. [2025-10-20T19:37:12.617432] [DEBUG] [VALIDATOR ] KRP: All expected trading days are present. [2025-10-20T19:37:12.617553] [DEBUG] [VALIDATOR ] KRP: Primary data validation passed. [2025-10-20T19:37:12.619153] [INFO ] [MAIN ] KRP: Fetched primary data is valid. [2025-10-20T19:37:12.619258] [DEBUG] [LOCAL ] KRP: Saving primary data. [2025-10-20T19:37:12.621010] [DEBUG] [LOCAL ] KRP: Trimmed 55037 records to 36203 (keeping 200 days of data) [2025-10-20T19:37:12.864404] [DEBUG] [LOCAL ] KRP: Primary data saved (36203 records). [2025-10-20T19:37:12.864615] [INFO ] [MAIN ] KRP: Full history saved. [2025-10-20T19:37:12.864754] [INFO ] [MAIN ] Update Progress: 1220/2367 symbols completed. KRP took 35.1s. Estimated remaining: 2h 27m 44s. Rate: 7.8 symbols/min [2025-10-20T19:37:13.050613] [INFO ] [MAIN ] KTF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:13.051359] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-07. [2025-10-20T19:37:13.068036] [DEBUG] [LOCAL ] KTF: Loaded 11900 existing records from primary storage. [2025-10-20T19:37:13.068547] [INFO ] [MAIN ] KTF: Loaded 11900 existing records. [2025-10-20T19:37:13.264362] [WARN ] [MAIN ] KTF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:13.281309] [INFO ] [MAIN ] KTF: Wiping existing primary data before full refresh. [2025-10-20T19:37:13.293176] [INFO ] [LOCAL ] KTF: Starting deletion of primary data. [2025-10-20T19:37:13.293482] [DEBUG] [LOCAL ] KTF: Deleted primary data file. [2025-10-20T19:37:13.293737] [INFO ] [MAIN ] KTF: Starting full primary refresh. [2025-10-20T19:37:13.300831] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:37:13.689951] [INFO ] [REMOTE ] KTF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:13.851475] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-01. [2025-10-20T19:37:14.485521] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-08 (2.22 seconds). [2025-10-20T19:37:14.525334] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:37:14.651408] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-08. [2025-10-20T19:37:14.711049] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T19:37:15.451467] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-09. [2025-10-20T19:37:16.042469] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T19:37:16.251430] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-02. [2025-10-20T19:37:17.022856] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T19:37:17.051420] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-03. [2025-10-20T19:37:17.280329] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T19:37:17.845042] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T19:37:17.851331] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-04. [2025-10-20T19:37:18.651381] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-05. [2025-10-20T19:37:18.893275] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T19:37:19.451443] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-05. [2025-10-20T19:37:20.114154] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T19:37:20.189150] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T19:37:20.251337] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-06. [2025-10-20T19:37:21.051347] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-10. [2025-10-20T19:37:21.725562] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:37:21.851563] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-07. [2025-10-20T19:37:22.651368] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-06. [2025-10-20T19:37:22.900998] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:37:22.909363] [INFO ] [REMOTE ] KSS: Full fetch complete (77753 records). [2025-10-20T19:37:22.909544] [DEBUG] [VALIDATOR ] KSS: Validating primary data. [2025-10-20T19:37:23.339826] [DEBUG] [VALIDATOR ] KSS: All expected trading days are present. [2025-10-20T19:37:23.341240] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T19:37:23.340426] [DEBUG] [VALIDATOR ] KSS: Primary data validation passed. [2025-10-20T19:37:23.351073] [INFO ] [MAIN ] KSS: Fetched primary data is valid. [2025-10-20T19:37:23.351590] [DEBUG] [LOCAL ] KSS: Saving primary data. [2025-10-20T19:37:23.353176] [DEBUG] [LOCAL ] KSS: Trimmed 77753 records to 53591 (keeping 200 days of data) [2025-10-20T19:37:23.456314] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-08. [2025-10-20T19:37:23.475868] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T19:37:23.691886] [DEBUG] [LOCAL ] KSS: Primary data saved (53591 records). [2025-10-20T19:37:23.692038] [INFO ] [MAIN ] KSS: Full history saved. [2025-10-20T19:37:23.692128] [INFO ] [MAIN ] Update Progress: 1221/2367 symbols completed. KSS took 43.4s. Estimated remaining: 2h 27m 39s. Rate: 7.8 symbols/min [2025-10-20T19:37:23.857544] [INFO ] [MAIN ] KUKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:23.857727] [DEBUG] [LOCAL ] KUKE: No existing data file found in primary storage. [2025-10-20T19:37:23.857773] [WARN ] [MAIN ] KUKE: No existing data found, performing full refresh instead. [2025-10-20T19:37:23.857878] [INFO ] [MAIN ] KUKE: Wiping existing primary data before full refresh. [2025-10-20T19:37:23.857919] [INFO ] [LOCAL ] KUKE: Starting deletion of primary data. [2025-10-20T19:37:23.857958] [DEBUG] [LOCAL ] KUKE: No primary data file found to delete. [2025-10-20T19:37:23.857990] [INFO ] [MAIN ] KUKE: Starting full primary refresh. [2025-10-20T19:37:24.201601] [INFO ] [REMOTE ] KUKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:24.251368] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-01. [2025-10-20T19:37:25.034123] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:37:25.051338] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-09. [2025-10-20T19:37:25.195871] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T19:37:25.851373] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-02. [2025-10-20T19:37:26.596210] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T19:37:26.651386] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-03. [2025-10-20T19:37:26.725858] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T19:37:27.451368] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-10. [2025-10-20T19:37:27.557443] [DEBUG] [REMOTE ] KT: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T19:37:27.557765] [WARN ] [REMOTE ] KT: No data returned for month 2025-10 [2025-10-20T19:37:27.557857] [FATAL] [REMOTE ] KT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:37:27.557971] [FATAL] [MAIN ] KT: Failed to fetch primary data. [2025-10-20T19:37:27.558048] [WARN ] [MAIN ] KT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:37:27.558134] [INFO ] [LOCAL ] KT: Starting deletion of primary data. [2025-10-20T19:37:27.558211] [DEBUG] [LOCAL ] KT: No primary data file found to delete. [2025-10-20T19:37:27.558261] [INFO ] [LOCAL ] KT: Starting deletion of derivative data. [2025-10-20T19:37:27.558440] [DEBUG] [LOCAL ] KT: Deleted derivative data file. [2025-10-20T19:37:27.558986] [INFO ] [MAIN ] Update Progress: 1222/2367 symbols completed. KT took 34.6s. Estimated remaining: 2h 27m 28s. Rate: 7.8 symbols/min [2025-10-20T19:37:27.670015] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T19:37:27.742665] [INFO ] [MAIN ] KVUE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:27.794790] [DEBUG] [LOCAL ] KVUE: Loaded 54447 existing records from primary storage. [2025-10-20T19:37:27.795141] [INFO ] [MAIN ] KVUE: Loaded 54447 existing records. [2025-10-20T19:37:27.977116] [WARN ] [MAIN ] KVUE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:27.977375] [INFO ] [MAIN ] KVUE: Wiping existing primary data before full refresh. [2025-10-20T19:37:27.977466] [INFO ] [LOCAL ] KVUE: Starting deletion of primary data. [2025-10-20T19:37:27.977713] [DEBUG] [LOCAL ] KVUE: Deleted primary data file. [2025-10-20T19:37:27.977771] [INFO ] [MAIN ] KVUE: Starting full primary refresh. [2025-10-20T19:37:28.254332] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-04. [2025-10-20T19:37:28.333385] [INFO ] [REMOTE ] KVUE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:28.898544] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T19:37:29.051371] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-07. [2025-10-20T19:37:29.831028] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T19:37:29.851363] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-08. [2025-10-20T19:37:30.633035] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T19:37:30.651324] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-09. [2025-10-20T19:37:31.451392] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-01. [2025-10-20T19:37:31.557938] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T19:37:32.251376] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-10. [2025-10-20T19:37:33.009072] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T19:37:33.010823] [INFO ] [REMOTE ] KTF: Full fetch complete (17266 records). [2025-10-20T19:37:33.010891] [DEBUG] [VALIDATOR ] KTF: Validating primary data. [2025-10-20T19:37:33.056626] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-05. [2025-10-20T19:37:33.380322] [DEBUG] [VALIDATOR ] KTF: All expected trading days are present. [2025-10-20T19:37:33.380628] [DEBUG] [VALIDATOR ] KTF: Primary data validation passed. [2025-10-20T19:37:33.381052] [INFO ] [MAIN ] KTF: Fetched primary data is valid. [2025-10-20T19:37:33.392926] [DEBUG] [LOCAL ] KTF: Saving primary data. [2025-10-20T19:37:33.414846] [DEBUG] [LOCAL ] KTF: Trimmed 17266 records to 11669 (keeping 200 days of data) [2025-10-20T19:37:33.417213] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T19:37:33.499138] [DEBUG] [LOCAL ] KTF: Primary data saved (11669 records). [2025-10-20T19:37:33.499273] [INFO ] [MAIN ] KTF: Full history saved. [2025-10-20T19:37:33.499352] [INFO ] [MAIN ] Update Progress: 1223/2367 symbols completed. KTF took 20.6s. Estimated remaining: 2h 27m 19s. Rate: 7.8 symbols/min [2025-10-20T19:37:33.577774] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T19:37:33.672536] [INFO ] [MAIN ] KVYO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:33.720634] [DEBUG] [LOCAL ] KVYO: Loaded 51549 existing records from primary storage. [2025-10-20T19:37:33.721013] [INFO ] [MAIN ] KVYO: Loaded 51549 existing records. [2025-10-20T19:37:33.858242] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-02. [2025-10-20T19:37:33.897825] [WARN ] [MAIN ] KVYO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:33.898196] [INFO ] [MAIN ] KVYO: Wiping existing primary data before full refresh. [2025-10-20T19:37:33.898249] [INFO ] [LOCAL ] KVYO: Starting deletion of primary data. [2025-10-20T19:37:33.898467] [DEBUG] [LOCAL ] KVYO: Deleted primary data file. [2025-10-20T19:37:33.898523] [INFO ] [MAIN ] KVYO: Starting full primary refresh. [2025-10-20T19:37:34.245302] [INFO ] [REMOTE ] KVYO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:34.651430] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-01. [2025-10-20T19:37:35.451455] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-06. [2025-10-20T19:37:35.656247] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T19:37:35.931800] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T19:37:36.179630] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T19:37:36.251345] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-07. [2025-10-20T19:37:36.705815] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T19:37:37.051415] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-08. [2025-10-20T19:37:37.679955] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T19:37:37.851408] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-02. [2025-10-20T19:37:38.651371] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-09. [2025-10-20T19:37:39.417074] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:37:39.451337] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-03. [2025-10-20T19:37:40.124319] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:37:40.251384] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-10. [2025-10-20T19:37:40.933619] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T19:37:41.051443] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-03. [2025-10-20T19:37:41.851360] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-04. [2025-10-20T19:37:41.873208] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T19:37:41.878812] [INFO ] [REMOTE ] KTB: Full fetch complete (58258 records). [2025-10-20T19:37:41.878944] [DEBUG] [VALIDATOR ] KTB: Validating primary data. [2025-10-20T19:37:42.263958] [DEBUG] [VALIDATOR ] KTB: All expected trading days are present. [2025-10-20T19:37:42.264040] [DEBUG] [VALIDATOR ] KTB: Primary data validation passed. [2025-10-20T19:37:42.264898] [INFO ] [MAIN ] KTB: Fetched primary data is valid. [2025-10-20T19:37:42.264961] [DEBUG] [LOCAL ] KTB: Saving primary data. [2025-10-20T19:37:42.266208] [DEBUG] [LOCAL ] KTB: Trimmed 58258 records to 40517 (keeping 200 days of data) [2025-10-20T19:37:42.508341] [DEBUG] [LOCAL ] KTB: Primary data saved (40517 records). [2025-10-20T19:37:42.508455] [INFO ] [MAIN ] KTB: Full history saved. [2025-10-20T19:37:42.508524] [INFO ] [MAIN ] Update Progress: 1224/2367 symbols completed. KTB took 42.7s. Estimated remaining: 2h 27m 12s. Rate: 7.8 symbols/min [2025-10-20T19:37:42.679605] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-09. [2025-10-20T19:37:42.683229] [INFO ] [MAIN ] KW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:42.719625] [DEBUG] [LOCAL ] KW: Loaded 41091 existing records from primary storage. [2025-10-20T19:37:42.719928] [INFO ] [MAIN ] KW: Loaded 41091 existing records. [2025-10-20T19:37:42.925370] [WARN ] [MAIN ] KW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:42.925783] [INFO ] [MAIN ] KW: Wiping existing primary data before full refresh. [2025-10-20T19:37:42.926009] [INFO ] [LOCAL ] KW: Starting deletion of primary data. [2025-10-20T19:37:42.926413] [DEBUG] [LOCAL ] KW: Deleted primary data file. [2025-10-20T19:37:42.926484] [INFO ] [MAIN ] KW: Starting full primary refresh. [2025-10-20T19:37:43.062760] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T19:37:43.331606] [INFO ] [REMOTE ] KW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:43.422861] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:37:43.451327] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-04. [2025-10-20T19:37:43.546958] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:37:44.251441] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-05. [2025-10-20T19:37:45.051357] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-01. [2025-10-20T19:37:45.318152] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T19:37:45.621698] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:37:45.851375] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-10. [2025-10-20T19:37:46.457695] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:37:46.651365] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-02. [2025-10-20T19:37:47.007308] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T19:37:47.009692] [INFO ] [REMOTE ] KUKE: Full fetch complete (9164 records). [2025-10-20T19:37:47.009766] [DEBUG] [VALIDATOR ] KUKE: Validating primary data. [2025-10-20T19:37:47.195509] [WARN ] [VALIDATOR ] KUKE: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:37:47.195826] [FATAL] [MAIN ] KUKE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:37:47.195912] [WARN ] [MAIN ] KUKE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:37:47.195985] [INFO ] [LOCAL ] KUKE: Starting deletion of primary data. [2025-10-20T19:37:47.196050] [DEBUG] [LOCAL ] KUKE: No primary data file found to delete. [2025-10-20T19:37:47.196126] [INFO ] [LOCAL ] KUKE: Starting deletion of derivative data. [2025-10-20T19:37:47.196179] [DEBUG] [LOCAL ] KUKE: No derivative data file found to delete. [2025-10-20T19:37:47.196238] [INFO ] [MAIN ] Update Progress: 1225/2367 symbols completed. KUKE took 23.5s. Estimated remaining: 2h 27m 2s. Rate: 7.8 symbols/min [2025-10-20T19:37:47.371206] [INFO ] [MAIN ] KWR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:47.388857] [DEBUG] [LOCAL ] KWR: Loaded 16242 existing records from primary storage. [2025-10-20T19:37:47.389085] [INFO ] [MAIN ] KWR: Loaded 16242 existing records. [2025-10-20T19:37:47.456570] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-06. [2025-10-20T19:37:47.576979] [WARN ] [MAIN ] KWR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:47.577289] [INFO ] [MAIN ] KWR: Wiping existing primary data before full refresh. [2025-10-20T19:37:47.577387] [INFO ] [LOCAL ] KWR: Starting deletion of primary data. [2025-10-20T19:37:47.577628] [DEBUG] [LOCAL ] KWR: Deleted primary data file. [2025-10-20T19:37:47.577727] [INFO ] [MAIN ] KWR: Starting full primary refresh. [2025-10-20T19:37:47.886053] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:37:47.988209] [INFO ] [REMOTE ] KWR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:48.251349] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-03. [2025-10-20T19:37:48.950198] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T19:37:49.051380] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-07. [2025-10-20T19:37:49.789362] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:37:49.851369] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-05. [2025-10-20T19:37:50.651372] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-01. [2025-10-20T19:37:50.735944] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T19:37:51.451379] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-08. [2025-10-20T19:37:51.678108] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T19:37:51.783623] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T19:37:52.251397] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-06. [2025-10-20T19:37:53.051351] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-02. [2025-10-20T19:37:53.136259] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T19:37:53.848623] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T19:37:53.851991] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-09. [2025-10-20T19:37:54.068167] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T19:37:54.651412] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-03. [2025-10-20T19:37:55.451418] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-07. [2025-10-20T19:37:55.548907] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T19:37:55.733591] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T19:37:56.251425] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-10. [2025-10-20T19:37:57.051364] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-04. [2025-10-20T19:37:57.303903] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T19:37:57.851422] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-04. [2025-10-20T19:37:58.048152] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T19:37:58.056009] [INFO ] [REMOTE ] KVYO: Full fetch complete (72146 records). [2025-10-20T19:37:58.056163] [DEBUG] [VALIDATOR ] KVYO: Validating primary data. [2025-10-20T19:37:58.452664] [DEBUG] [VALIDATOR ] KVYO: All expected trading days are present. [2025-10-20T19:37:58.452767] [DEBUG] [VALIDATOR ] KVYO: Primary data validation passed. [2025-10-20T19:37:58.453730] [INFO ] [MAIN ] KVYO: Fetched primary data is valid. [2025-10-20T19:37:58.453811] [DEBUG] [LOCAL ] KVYO: Saving primary data. [2025-10-20T19:37:58.479828] [DEBUG] [LOCAL ] KVYO: Trimmed 72146 records to 50869 (keeping 200 days of data) [2025-10-20T19:37:58.481465] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T19:37:58.658849] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-08. [2025-10-20T19:37:58.780122] [DEBUG] [LOCAL ] KVYO: Primary data saved (50869 records). [2025-10-20T19:37:58.780255] [INFO ] [MAIN ] KVYO: Full history saved. [2025-10-20T19:37:58.780331] [INFO ] [MAIN ] Update Progress: 1226/2367 symbols completed. KVYO took 25.3s. Estimated remaining: 2h 26m 58s. Rate: 7.8 symbols/min [2025-10-20T19:37:58.947985] [INFO ] [MAIN ] KYN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:58.959814] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:37:58.992929] [DEBUG] [LOCAL ] KYN: Loaded 30763 existing records from primary storage. [2025-10-20T19:37:58.993214] [INFO ] [MAIN ] KYN: Loaded 30763 existing records. [2025-10-20T19:37:59.163378] [WARN ] [MAIN ] KYN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:59.163597] [INFO ] [MAIN ] KYN: Wiping existing primary data before full refresh. [2025-10-20T19:37:59.163682] [INFO ] [LOCAL ] KYN: Starting deletion of primary data. [2025-10-20T19:37:59.163952] [DEBUG] [LOCAL ] KYN: Deleted primary data file. [2025-10-20T19:37:59.164023] [INFO ] [MAIN ] KYN: Starting full primary refresh. [2025-10-20T19:37:59.456477] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-05. [2025-10-20T19:37:59.513379] [INFO ] [REMOTE ] KYN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:00.251358] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-05. [2025-10-20T19:38:00.306386] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T19:38:00.390732] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T19:38:01.051384] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-09. [2025-10-20T19:38:01.851348] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-01. [2025-10-20T19:38:02.155413] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:38:02.651403] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-06. [2025-10-20T19:38:03.260348] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T19:38:03.272210] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:38:03.380955] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:38:03.451348] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-06. [2025-10-20T19:38:04.251381] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-02. [2025-10-20T19:38:04.695799] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T19:38:05.051377] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-07. [2025-10-20T19:38:05.385485] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:38:05.851422] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-07. [2025-10-20T19:38:06.243394] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T19:38:06.651406] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-08. [2025-10-20T19:38:06.921180] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T19:38:07.451438] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-10. [2025-10-20T19:38:08.177601] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T19:38:08.251358] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-09. [2025-10-20T19:38:08.316103] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:38:08.324161] [INFO ] [REMOTE ] KVUE: Full fetch complete (77842 records). [2025-10-20T19:38:08.324327] [DEBUG] [VALIDATOR ] KVUE: Validating primary data. [2025-10-20T19:38:08.710728] [DEBUG] [VALIDATOR ] KVUE: All expected trading days are present. [2025-10-20T19:38:08.710812] [DEBUG] [VALIDATOR ] KVUE: Primary data validation passed. [2025-10-20T19:38:08.711788] [INFO ] [MAIN ] KVUE: Fetched primary data is valid. [2025-10-20T19:38:08.711850] [DEBUG] [LOCAL ] KVUE: Saving primary data. [2025-10-20T19:38:08.713233] [DEBUG] [LOCAL ] KVUE: Trimmed 77842 records to 53667 (keeping 200 days of data) [2025-10-20T19:38:09.038316] [DEBUG] [LOCAL ] KVUE: Primary data saved (53667 records). [2025-10-20T19:38:09.038452] [INFO ] [MAIN ] KVUE: Full history saved. [2025-10-20T19:38:09.038527] [INFO ] [MAIN ] Update Progress: 1227/2367 symbols completed. KVUE took 41.5s. Estimated remaining: 2h 26m 52s. Rate: 7.8 symbols/min [2025-10-20T19:38:09.058928] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-03. [2025-10-20T19:38:09.207169] [INFO ] [MAIN ] L: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:09.245436] [DEBUG] [LOCAL ] L: Loaded 41844 existing records from primary storage. [2025-10-20T19:38:09.245741] [INFO ] [MAIN ] L: Loaded 41844 existing records. [2025-10-20T19:38:09.422502] [WARN ] [MAIN ] L: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:09.422715] [INFO ] [MAIN ] L: Wiping existing primary data before full refresh. [2025-10-20T19:38:09.422769] [INFO ] [LOCAL ] L: Starting deletion of primary data. [2025-10-20T19:38:09.422932] [DEBUG] [LOCAL ] L: Deleted primary data file. [2025-10-20T19:38:09.422969] [INFO ] [MAIN ] L: Starting full primary refresh. [2025-10-20T19:38:09.784167] [INFO ] [REMOTE ] L: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:09.804950] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T19:38:09.851333] [INFO ] [REMOTE ] L: Fetching primary data for 2025-01. [2025-10-20T19:38:10.416817] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:38:10.651374] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-08. [2025-10-20T19:38:11.396673] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T19:38:11.451431] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-04. [2025-10-20T19:38:11.622718] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T19:38:12.251485] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-09. [2025-10-20T19:38:12.856076] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:38:12.871602] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T19:38:13.051421] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-05. [2025-10-20T19:38:13.851370] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-10. [2025-10-20T19:38:14.339196] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T19:38:14.651489] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-10. [2025-10-20T19:38:14.854082] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:38:14.856315] [INFO ] [REMOTE ] KWR: Full fetch complete (23302 records). [2025-10-20T19:38:14.856427] [DEBUG] [VALIDATOR ] KWR: Validating primary data. [2025-10-20T19:38:15.234082] [DEBUG] [VALIDATOR ] KWR: All expected trading days are present. [2025-10-20T19:38:15.234167] [DEBUG] [VALIDATOR ] KWR: Primary data validation passed. [2025-10-20T19:38:15.234721] [INFO ] [MAIN ] KWR: Fetched primary data is valid. [2025-10-20T19:38:15.234784] [DEBUG] [LOCAL ] KWR: Saving primary data. [2025-10-20T19:38:15.235565] [DEBUG] [LOCAL ] KWR: Trimmed 23302 records to 15952 (keeping 200 days of data) [2025-10-20T19:38:15.336094] [DEBUG] [LOCAL ] KWR: Primary data saved (15952 records). [2025-10-20T19:38:15.336233] [INFO ] [MAIN ] KWR: Full history saved. [2025-10-20T19:38:15.336311] [INFO ] [MAIN ] Update Progress: 1228/2367 symbols completed. KWR took 28.1s. Estimated remaining: 2h 26m 43s. Rate: 7.8 symbols/min [2025-10-20T19:38:15.456459] [INFO ] [REMOTE ] L: Fetching primary data for 2025-02. [2025-10-20T19:38:15.510019] [INFO ] [MAIN ] LAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:15.557572] [DEBUG] [LOCAL ] LAC: Loaded 53290 existing records from primary storage. [2025-10-20T19:38:15.557936] [INFO ] [MAIN ] LAC: Loaded 53290 existing records. [2025-10-20T19:38:15.734118] [WARN ] [MAIN ] LAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:15.734311] [INFO ] [MAIN ] LAC: Wiping existing primary data before full refresh. [2025-10-20T19:38:15.734357] [INFO ] [LOCAL ] LAC: Starting deletion of primary data. [2025-10-20T19:38:15.734618] [DEBUG] [LOCAL ] LAC: Deleted primary data file. [2025-10-20T19:38:15.734693] [INFO ] [MAIN ] LAC: Starting full primary refresh. [2025-10-20T19:38:16.101477] [INFO ] [REMOTE ] LAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:16.139147] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:38:16.145751] [INFO ] [REMOTE ] KW: Full fetch complete (57310 records). [2025-10-20T19:38:16.145863] [DEBUG] [VALIDATOR ] KW: Validating primary data. [2025-10-20T19:38:16.256566] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-06. [2025-10-20T19:38:16.574206] [DEBUG] [VALIDATOR ] KW: All expected trading days are present. [2025-10-20T19:38:16.574283] [DEBUG] [VALIDATOR ] KW: Primary data validation passed. [2025-10-20T19:38:16.576003] [INFO ] [MAIN ] KW: Fetched primary data is valid. [2025-10-20T19:38:16.576065] [DEBUG] [LOCAL ] KW: Saving primary data. [2025-10-20T19:38:16.577199] [DEBUG] [LOCAL ] KW: Trimmed 57310 records to 40515 (keeping 200 days of data) [2025-10-20T19:38:16.807720] [DEBUG] [LOCAL ] KW: Primary data saved (40515 records). [2025-10-20T19:38:16.807928] [INFO ] [MAIN ] KW: Full history saved. [2025-10-20T19:38:16.808274] [INFO ] [MAIN ] Update Progress: 1229/2367 symbols completed. KW took 34.3s. Estimated remaining: 2h 26m 30s. Rate: 7.8 symbols/min [2025-10-20T19:38:16.981775] [INFO ] [MAIN ] LAD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:17.005631] [DEBUG] [LOCAL ] LAD: Loaded 23663 existing records from primary storage. [2025-10-20T19:38:17.005905] [INFO ] [MAIN ] LAD: Loaded 23663 existing records. [2025-10-20T19:38:17.060082] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-01. [2025-10-20T19:38:17.175498] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:38:17.227361] [WARN ] [MAIN ] LAD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:17.228775] [INFO ] [MAIN ] LAD: Wiping existing primary data before full refresh. [2025-10-20T19:38:17.228836] [INFO ] [LOCAL ] LAD: Starting deletion of primary data. [2025-10-20T19:38:17.229026] [DEBUG] [LOCAL ] LAD: Deleted primary data file. [2025-10-20T19:38:17.229063] [INFO ] [MAIN ] LAD: Starting full primary refresh. [2025-10-20T19:38:17.583144] [INFO ] [REMOTE ] LAD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:17.604151] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T19:38:17.851381] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-01. [2025-10-20T19:38:18.651379] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-07. [2025-10-20T19:38:19.153411] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T19:38:19.152612] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-01 (2.09 seconds). [2025-10-20T19:38:19.451442] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-02. [2025-10-20T19:38:19.800587] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T19:38:20.251430] [INFO ] [REMOTE ] L: Fetching primary data for 2025-03. [2025-10-20T19:38:21.051388] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-08. [2025-10-20T19:38:21.299483] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T19:38:21.856470] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-02. [2025-10-20T19:38:21.870170] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:38:22.408848] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T19:38:22.651378] [INFO ] [REMOTE ] L: Fetching primary data for 2025-04. [2025-10-20T19:38:23.010417] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T19:38:23.451393] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-03. [2025-10-20T19:38:24.229908] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T19:38:24.251365] [INFO ] [REMOTE ] L: Fetching primary data for 2025-05. [2025-10-20T19:38:24.844214] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T19:38:25.051391] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-03. [2025-10-20T19:38:25.753176] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T19:38:25.851385] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-09. [2025-10-20T19:38:26.651380] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-04. [2025-10-20T19:38:27.023476] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T19:38:27.149850] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T19:38:27.451392] [INFO ] [REMOTE ] L: Fetching primary data for 2025-06. [2025-10-20T19:38:27.560417] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T19:38:28.251417] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-10. [2025-10-20T19:38:28.994216] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:38:29.051338] [INFO ] [REMOTE ] L: Fetching primary data for 2025-07. [2025-10-20T19:38:29.566988] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T19:38:29.572054] [INFO ] [REMOTE ] KYN: Full fetch complete (44449 records). [2025-10-20T19:38:29.572165] [DEBUG] [VALIDATOR ] KYN: Validating primary data. [2025-10-20T19:38:29.856613] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-05. [2025-10-20T19:38:29.978015] [DEBUG] [VALIDATOR ] KYN: All expected trading days are present. [2025-10-20T19:38:29.978126] [DEBUG] [VALIDATOR ] KYN: Primary data validation passed. [2025-10-20T19:38:29.979572] [INFO ] [MAIN ] KYN: Fetched primary data is valid. [2025-10-20T19:38:29.979661] [DEBUG] [LOCAL ] KYN: Saving primary data. [2025-10-20T19:38:29.981024] [DEBUG] [LOCAL ] KYN: Trimmed 44449 records to 30326 (keeping 200 days of data) [2025-10-20T19:38:30.178550] [DEBUG] [LOCAL ] KYN: Primary data saved (30326 records). [2025-10-20T19:38:30.178801] [INFO ] [MAIN ] KYN: Full history saved. [2025-10-20T19:38:30.178915] [INFO ] [MAIN ] Update Progress: 1230/2367 symbols completed. KYN took 31.4s. Estimated remaining: 2h 26m 27s. Rate: 7.8 symbols/min [2025-10-20T19:38:30.357194] [INFO ] [MAIN ] LADR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:30.402391] [DEBUG] [LOCAL ] LADR: Loaded 39947 existing records from primary storage. [2025-10-20T19:38:30.402866] [INFO ] [MAIN ] LADR: Loaded 39947 existing records. [2025-10-20T19:38:30.615329] [WARN ] [MAIN ] LADR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:30.644493] [INFO ] [MAIN ] LADR: Wiping existing primary data before full refresh. [2025-10-20T19:38:30.644678] [INFO ] [LOCAL ] LADR: Starting deletion of primary data. [2025-10-20T19:38:30.645066] [DEBUG] [LOCAL ] LADR: Deleted primary data file. [2025-10-20T19:38:30.645353] [INFO ] [MAIN ] LADR: Starting full primary refresh. [2025-10-20T19:38:30.659993] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-04. [2025-10-20T19:38:30.653476] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:38:31.079711] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T19:38:31.086981] [INFO ] [REMOTE ] LADR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:31.451448] [INFO ] [REMOTE ] L: Fetching primary data for 2025-08. [2025-10-20T19:38:32.251426] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-06. [2025-10-20T19:38:32.660130] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T19:38:32.958537] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:38:33.051435] [INFO ] [REMOTE ] L: Fetching primary data for 2025-09. [2025-10-20T19:38:33.713080] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:38:33.851476] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-01. [2025-10-20T19:38:34.651372] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-07. [2025-10-20T19:38:35.451384] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-05. [2025-10-20T19:38:35.732282] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-09 (2.68 seconds). [2025-10-20T19:38:36.251365] [INFO ] [REMOTE ] L: Fetching primary data for 2025-10. [2025-10-20T19:38:36.494196] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-01 (2.64 seconds). [2025-10-20T19:38:36.709739] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T19:38:37.051357] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-02. [2025-10-20T19:38:37.482144] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T19:38:37.851094] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T19:38:37.856792] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-08. [2025-10-20T19:38:37.858429] [INFO ] [REMOTE ] L: Full fetch complete (59436 records). [2025-10-20T19:38:37.858521] [DEBUG] [VALIDATOR ] L: Validating primary data. [2025-10-20T19:38:38.244143] [DEBUG] [VALIDATOR ] L: All expected trading days are present. [2025-10-20T19:38:38.244276] [DEBUG] [VALIDATOR ] L: Primary data validation passed. [2025-10-20T19:38:38.245166] [INFO ] [MAIN ] L: Fetched primary data is valid. [2025-10-20T19:38:38.245312] [DEBUG] [LOCAL ] L: Saving primary data. [2025-10-20T19:38:38.246567] [DEBUG] [LOCAL ] L: Trimmed 59436 records to 41243 (keeping 200 days of data) [2025-10-20T19:38:38.482954] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T19:38:38.517604] [DEBUG] [LOCAL ] L: Primary data saved (41243 records). [2025-10-20T19:38:38.517736] [INFO ] [MAIN ] L: Full history saved. [2025-10-20T19:38:38.517813] [INFO ] [MAIN ] Update Progress: 1231/2367 symbols completed. L took 29.5s. Estimated remaining: 2h 26m 20s. Rate: 7.8 symbols/min [2025-10-20T19:38:38.657770] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-03. [2025-10-20T19:38:38.691010] [INFO ] [MAIN ] LANV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:38.695934] [DEBUG] [LOCAL ] LANV: Loaded 2807 existing records from primary storage. [2025-10-20T19:38:38.696090] [INFO ] [MAIN ] LANV: Loaded 2807 existing records. [2025-10-20T19:38:38.875747] [WARN ] [MAIN ] LANV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:38.875940] [INFO ] [MAIN ] LANV: Wiping existing primary data before full refresh. [2025-10-20T19:38:38.875990] [INFO ] [LOCAL ] LANV: Starting deletion of primary data. [2025-10-20T19:38:38.876118] [DEBUG] [LOCAL ] LANV: Deleted primary data file. [2025-10-20T19:38:38.876158] [INFO ] [MAIN ] LANV: Starting full primary refresh. [2025-10-20T19:38:39.209918] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:38:39.252749] [INFO ] [REMOTE ] LANV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:39.451372] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-01. [2025-10-20T19:38:39.848020] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-01 (0.40 seconds). [2025-10-20T19:38:40.107818] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T19:38:40.251415] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-02. [2025-10-20T19:38:40.731937] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T19:38:41.051375] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-06. [2025-10-20T19:38:41.851421] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-09. [2025-10-20T19:38:42.651379] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-04. [2025-10-20T19:38:42.909395] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:38:42.943077] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:38:43.451376] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-10. [2025-10-20T19:38:44.254765] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-03. [2025-10-20T19:38:44.270780] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:38:44.315135] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:38:44.318977] [INFO ] [REMOTE ] LAD: Full fetch complete (35396 records). [2025-10-20T19:38:44.319113] [DEBUG] [VALIDATOR ] LAD: Validating primary data. [2025-10-20T19:38:44.685551] [DEBUG] [VALIDATOR ] LAD: All expected trading days are present. [2025-10-20T19:38:44.685688] [DEBUG] [VALIDATOR ] LAD: Primary data validation passed. [2025-10-20T19:38:44.687162] [INFO ] [MAIN ] LAD: Fetched primary data is valid. [2025-10-20T19:38:44.687258] [DEBUG] [LOCAL ] LAD: Saving primary data. [2025-10-20T19:38:44.688138] [DEBUG] [LOCAL ] LAD: Trimmed 35396 records to 23301 (keeping 200 days of data) [2025-10-20T19:38:44.778421] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T19:38:44.832792] [DEBUG] [LOCAL ] LAD: Primary data saved (23301 records). [2025-10-20T19:38:44.832908] [INFO ] [MAIN ] LAD: Full history saved. [2025-10-20T19:38:44.832983] [INFO ] [MAIN ] Update Progress: 1232/2367 symbols completed. LAD took 28.0s. Estimated remaining: 2h 26m 11s. Rate: 7.8 symbols/min [2025-10-20T19:38:45.003569] [INFO ] [MAIN ] LAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:45.003783] [DEBUG] [LOCAL ] LAR: No existing data file found in primary storage. [2025-10-20T19:38:45.003833] [WARN ] [MAIN ] LAR: No existing data found, performing full refresh instead. [2025-10-20T19:38:45.003963] [INFO ] [MAIN ] LAR: Wiping existing primary data before full refresh. [2025-10-20T19:38:45.004031] [INFO ] [LOCAL ] LAR: Starting deletion of primary data. [2025-10-20T19:38:45.004087] [DEBUG] [LOCAL ] LAR: No primary data file found to delete. [2025-10-20T19:38:45.004120] [INFO ] [MAIN ] LAR: Starting full primary refresh. [2025-10-20T19:38:45.056571] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-05. [2025-10-20T19:38:45.358626] [INFO ] [REMOTE ] LAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:45.851364] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-04. [2025-10-20T19:38:46.298493] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T19:38:46.571363] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:38:46.651347] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-01. [2025-10-20T19:38:47.277482] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T19:38:47.451370] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-07. [2025-10-20T19:38:48.251377] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-05. [2025-10-20T19:38:49.051365] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-02. [2025-10-20T19:38:49.414579] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T19:38:49.851432] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-06. [2025-10-20T19:38:50.534985] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-07 (3.08 seconds). [2025-10-20T19:38:50.651360] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-08. [2025-10-20T19:38:51.400786] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-10-20T19:38:51.451336] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-03. [2025-10-20T19:38:51.525197] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:38:52.251454] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-06. [2025-10-20T19:38:52.668940] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T19:38:52.729758] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T19:38:53.032252] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:38:53.051335] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-04. [2025-10-20T19:38:53.851362] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-07. [2025-10-20T19:38:54.651440] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-07. [2025-10-20T19:38:54.728076] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:38:55.336966] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-10-20T19:38:55.376360] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T19:38:55.451399] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-05. [2025-10-20T19:38:56.251396] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-08. [2025-10-20T19:38:56.800038] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T19:38:57.051459] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-09. [2025-10-20T19:38:57.094172] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T19:38:57.851416] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-09. [2025-10-20T19:38:58.500242] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T19:38:58.651400] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-10. [2025-10-20T19:38:59.196231] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T19:38:59.197277] [INFO ] [REMOTE ] LANV: Full fetch complete (4246 records). [2025-10-20T19:38:59.197350] [DEBUG] [VALIDATOR ] LANV: Validating primary data. [2025-10-20T19:38:59.462938] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-08. [2025-10-20T19:38:59.516439] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-09 (2.46 seconds). [2025-10-20T19:38:59.606373] [DEBUG] [VALIDATOR ] LANV: All expected trading days are present. [2025-10-20T19:38:59.606448] [DEBUG] [VALIDATOR ] LANV: Primary data validation passed. [2025-10-20T19:38:59.606571] [INFO ] [MAIN ] LANV: Fetched primary data is valid. [2025-10-20T19:38:59.606608] [DEBUG] [LOCAL ] LANV: Saving primary data. [2025-10-20T19:38:59.606973] [DEBUG] [LOCAL ] LANV: Trimmed 4246 records to 2795 (keeping 200 days of data) [2025-10-20T19:38:59.621702] [DEBUG] [LOCAL ] LANV: Primary data saved (2795 records). [2025-10-20T19:38:59.621761] [INFO ] [MAIN ] LANV: Full history saved. [2025-10-20T19:38:59.621825] [INFO ] [MAIN ] Update Progress: 1233/2367 symbols completed. LANV took 21.1s. Estimated remaining: 2h 26m 10s. Rate: 7.8 symbols/min [2025-10-20T19:38:59.786164] [INFO ] [MAIN ] LAW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:59.801742] [DEBUG] [LOCAL ] LAW: Loaded 14898 existing records from primary storage. [2025-10-20T19:38:59.801966] [INFO ] [MAIN ] LAW: Loaded 14898 existing records. [2025-10-20T19:38:59.976151] [WARN ] [MAIN ] LAW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:59.976397] [INFO ] [MAIN ] LAW: Wiping existing primary data before full refresh. [2025-10-20T19:38:59.976445] [INFO ] [LOCAL ] LAW: Starting deletion of primary data. [2025-10-20T19:38:59.976580] [DEBUG] [LOCAL ] LAW: Deleted primary data file. [2025-10-20T19:38:59.976619] [INFO ] [MAIN ] LAW: Starting full primary refresh. [2025-10-20T19:39:00.256479] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-10. [2025-10-20T19:39:00.333695] [INFO ] [REMOTE ] LAW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:00.952893] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T19:39:01.051367] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-09. [2025-10-20T19:39:01.851386] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-06. [2025-10-20T19:39:02.573050] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:39:02.651421] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-01. [2025-10-20T19:39:03.175687] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-10 (2.92 seconds). [2025-10-20T19:39:03.184729] [INFO ] [REMOTE ] LAC: Full fetch complete (76140 records). [2025-10-20T19:39:03.184892] [DEBUG] [VALIDATOR ] LAC: Validating primary data. [2025-10-20T19:39:03.472273] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-10. [2025-10-20T19:39:03.604545] [DEBUG] [VALIDATOR ] LAC: All expected trading days are present. [2025-10-20T19:39:03.605198] [DEBUG] [VALIDATOR ] LAC: Primary data validation passed. [2025-10-20T19:39:03.606612] [INFO ] [MAIN ] LAC: Fetched primary data is valid. [2025-10-20T19:39:03.605533] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T19:39:03.606850] [DEBUG] [LOCAL ] LAC: Saving primary data. [2025-10-20T19:39:03.649558] [DEBUG] [LOCAL ] LAC: Trimmed 76140 records to 52554 (keeping 200 days of data) [2025-10-20T19:39:03.653148] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T19:39:03.972971] [DEBUG] [LOCAL ] LAC: Primary data saved (52554 records). [2025-10-20T19:39:03.973144] [INFO ] [MAIN ] LAC: Full history saved. [2025-10-20T19:39:03.973237] [INFO ] [MAIN ] Update Progress: 1234/2367 symbols completed. LAC took 48.6s. Estimated remaining: 2h 25m 59s. Rate: 7.8 symbols/min [2025-10-20T19:39:04.162177] [INFO ] [MAIN ] LAZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:04.203381] [DEBUG] [LOCAL ] LAZ: Loaded 45283 existing records from primary storage. [2025-10-20T19:39:04.203746] [INFO ] [MAIN ] LAZ: Loaded 45283 existing records. [2025-10-20T19:39:04.257599] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-07. [2025-10-20T19:39:04.391104] [WARN ] [MAIN ] LAZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:04.391323] [INFO ] [MAIN ] LAZ: Wiping existing primary data before full refresh. [2025-10-20T19:39:04.391386] [INFO ] [LOCAL ] LAZ: Starting deletion of primary data. [2025-10-20T19:39:04.391630] [DEBUG] [LOCAL ] LAZ: Deleted primary data file. [2025-10-20T19:39:04.391744] [INFO ] [MAIN ] LAZ: Starting full primary refresh. [2025-10-20T19:39:04.745530] [INFO ] [REMOTE ] LAZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:05.051389] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-02. [2025-10-20T19:39:05.214928] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:39:05.221290] [INFO ] [REMOTE ] LADR: Full fetch complete (55626 records). [2025-10-20T19:39:05.221414] [DEBUG] [VALIDATOR ] LADR: Validating primary data. [2025-10-20T19:39:05.601202] [DEBUG] [VALIDATOR ] LADR: All expected trading days are present. [2025-10-20T19:39:05.601302] [DEBUG] [VALIDATOR ] LADR: Primary data validation passed. [2025-10-20T19:39:05.602119] [INFO ] [MAIN ] LADR: Fetched primary data is valid. [2025-10-20T19:39:05.602179] [DEBUG] [LOCAL ] LADR: Saving primary data. [2025-10-20T19:39:05.603258] [DEBUG] [LOCAL ] LADR: Trimmed 55626 records to 39435 (keeping 200 days of data) [2025-10-20T19:39:05.838094] [DEBUG] [LOCAL ] LADR: Primary data saved (39435 records). [2025-10-20T19:39:05.838226] [INFO ] [MAIN ] LADR: Full history saved. [2025-10-20T19:39:05.838299] [INFO ] [MAIN ] Update Progress: 1235/2367 symbols completed. LADR took 35.7s. Estimated remaining: 2h 25m 46s. Rate: 7.8 symbols/min [2025-10-20T19:39:05.859754] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-01. [2025-10-20T19:39:06.017780] [INFO ] [MAIN ] LB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:06.026277] [DEBUG] [LOCAL ] LB: No existing data file found in primary storage. [2025-10-20T19:39:06.027945] [WARN ] [MAIN ] LB: No existing data found, performing full refresh instead. [2025-10-20T19:39:06.028295] [INFO ] [MAIN ] LB: Wiping existing primary data before full refresh. [2025-10-20T19:39:06.029410] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T19:39:06.029469] [INFO ] [LOCAL ] LB: Starting deletion of primary data. [2025-10-20T19:39:06.033739] [DEBUG] [LOCAL ] LB: No primary data file found to delete. [2025-10-20T19:39:06.033798] [INFO ] [MAIN ] LB: Starting full primary refresh. [2025-10-20T19:39:06.345128] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-10-20T19:39:06.432789] [INFO ] [REMOTE ] LB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:06.651363] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-03. [2025-10-20T19:39:07.430370] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T19:39:07.451345] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-08. [2025-10-20T19:39:07.546179] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:39:08.251383] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-04. [2025-10-20T19:39:09.052896] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-02. [2025-10-20T19:39:09.053718] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T19:39:09.232707] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T19:39:09.851406] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-01. [2025-10-20T19:39:10.320729] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:39:10.651378] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-05. [2025-10-20T19:39:11.408934] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:39:11.451365] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-03. [2025-10-20T19:39:11.586737] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T19:39:12.251428] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-06. [2025-10-20T19:39:13.051401] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-09. [2025-10-20T19:39:13.187877] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T19:39:13.219898] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T19:39:13.851355] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-07. [2025-10-20T19:39:14.651477] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-04. [2025-10-20T19:39:14.747602] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:39:14.781849] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:39:15.451425] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-08. [2025-10-20T19:39:16.256206] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-02. [2025-10-20T19:39:16.262276] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:39:16.427857] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T19:39:17.051419] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-09. [2025-10-20T19:39:17.756936] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T19:39:17.829009] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:39:17.851398] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-05. [2025-10-20T19:39:18.651402] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-10. [2025-10-20T19:39:19.465878] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-03. [2025-10-20T19:39:19.499844] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:39:19.564394] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T19:39:19.567493] [INFO ] [REMOTE ] LAW: Full fetch complete (20914 records). [2025-10-20T19:39:19.567624] [DEBUG] [VALIDATOR ] LAW: Validating primary data. [2025-10-20T19:39:19.933754] [DEBUG] [VALIDATOR ] LAW: All expected trading days are present. [2025-10-20T19:39:19.933840] [DEBUG] [VALIDATOR ] LAW: Primary data validation passed. [2025-10-20T19:39:19.934294] [INFO ] [MAIN ] LAW: Fetched primary data is valid. [2025-10-20T19:39:19.934332] [DEBUG] [LOCAL ] LAW: Saving primary data. [2025-10-20T19:39:19.934979] [DEBUG] [LOCAL ] LAW: Trimmed 20914 records to 14803 (keeping 200 days of data) [2025-10-20T19:39:20.021210] [DEBUG] [LOCAL ] LAW: Primary data saved (14803 records). [2025-10-20T19:39:20.021374] [INFO ] [MAIN ] LAW: Full history saved. [2025-10-20T19:39:20.021461] [INFO ] [MAIN ] Update Progress: 1236/2367 symbols completed. LAW took 20.4s. Estimated remaining: 2h 25m 44s. Rate: 7.8 symbols/min [2025-10-20T19:39:20.220466] [INFO ] [MAIN ] LBRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:20.251386] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-10. [2025-10-20T19:39:20.275879] [DEBUG] [LOCAL ] LBRT: Loaded 53738 existing records from primary storage. [2025-10-20T19:39:20.276223] [INFO ] [MAIN ] LBRT: Loaded 53738 existing records. [2025-10-20T19:39:20.491126] [WARN ] [MAIN ] LBRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:20.491338] [INFO ] [MAIN ] LBRT: Wiping existing primary data before full refresh. [2025-10-20T19:39:20.491384] [INFO ] [LOCAL ] LBRT: Starting deletion of primary data. [2025-10-20T19:39:20.491939] [DEBUG] [LOCAL ] LBRT: Deleted primary data file. [2025-10-20T19:39:20.491994] [INFO ] [MAIN ] LBRT: Starting full primary refresh. [2025-10-20T19:39:20.842736] [INFO ] [REMOTE ] LBRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:21.059009] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-06. [2025-10-20T19:39:21.081695] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:39:21.851359] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-04. [2025-10-20T19:39:22.325755] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-10-20T19:39:22.333872] [INFO ] [REMOTE ] LAR: Full fetch complete (61701 records). [2025-10-20T19:39:22.333998] [DEBUG] [VALIDATOR ] LAR: Validating primary data. [2025-10-20T19:39:22.555904] [WARN ] [VALIDATOR ] LAR: Validation failed - insufficient historical data. Have 185 unique days, need 200 days [2025-10-20T19:39:22.556587] [FATAL] [MAIN ] LAR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:39:22.556759] [WARN ] [MAIN ] LAR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:39:22.556841] [INFO ] [LOCAL ] LAR: Starting deletion of primary data. [2025-10-20T19:39:22.556965] [DEBUG] [LOCAL ] LAR: No primary data file found to delete. [2025-10-20T19:39:22.557036] [INFO ] [LOCAL ] LAR: Starting deletion of derivative data. [2025-10-20T19:39:22.557091] [DEBUG] [LOCAL ] LAR: No derivative data file found to delete. [2025-10-20T19:39:22.557210] [INFO ] [MAIN ] Update Progress: 1237/2367 symbols completed. LAR took 37.7s. Estimated remaining: 2h 25m 31s. Rate: 7.8 symbols/min [2025-10-20T19:39:22.581996] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T19:39:22.651344] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-07. [2025-10-20T19:39:23.320288] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:39:23.451364] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-05. [2025-10-20T19:39:23.787963] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:39:24.251394] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-08. [2025-10-20T19:39:24.606768] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T19:39:25.051422] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-01. [2025-10-20T19:39:25.485518] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T19:39:25.851390] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-06. [2025-10-20T19:39:26.651459] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-09. [2025-10-20T19:39:26.900168] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T19:39:27.232908] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:39:27.451630] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-02. [2025-10-20T19:39:28.251364] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-07. [2025-10-20T19:39:28.320004] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T19:39:29.051395] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-10. [2025-10-20T19:39:29.090399] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:39:29.851396] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-03. [2025-10-20T19:39:29.885436] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T19:39:30.565450] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T19:39:30.571495] [INFO ] [REMOTE ] LAZ: Full fetch complete (65448 records). [2025-10-20T19:39:30.571609] [DEBUG] [VALIDATOR ] LAZ: Validating primary data. [2025-10-20T19:39:30.661383] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-08. [2025-10-20T19:39:30.956810] [DEBUG] [VALIDATOR ] LAZ: All expected trading days are present. [2025-10-20T19:39:30.956907] [DEBUG] [VALIDATOR ] LAZ: Primary data validation passed. [2025-10-20T19:39:30.957832] [INFO ] [MAIN ] LAZ: Fetched primary data is valid. [2025-10-20T19:39:30.957897] [DEBUG] [LOCAL ] LAZ: Saving primary data. [2025-10-20T19:39:30.959224] [DEBUG] [LOCAL ] LAZ: Trimmed 65448 records to 44543 (keeping 200 days of data) [2025-10-20T19:39:31.227449] [DEBUG] [LOCAL ] LAZ: Primary data saved (44543 records). [2025-10-20T19:39:31.227558] [INFO ] [MAIN ] LAZ: Full history saved. [2025-10-20T19:39:31.227636] [INFO ] [MAIN ] Update Progress: 1238/2367 symbols completed. LAZ took 27.3s. Estimated remaining: 2h 25m 25s. Rate: 7.8 symbols/min [2025-10-20T19:39:31.546060] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:39:31.557066] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-04. [2025-10-20T19:39:32.207302] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T19:39:32.251345] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-09. [2025-10-20T19:39:33.283941] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T19:39:33.296194] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-05. [2025-10-20T19:39:33.357751] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T19:39:33.851430] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-10. [2025-10-20T19:39:35.030166] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T19:39:35.041185] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-06. [2025-10-20T19:39:35.320737] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T19:39:35.326332] [INFO ] [REMOTE ] LB: Full fetch complete (55925 records). [2025-10-20T19:39:35.326471] [DEBUG] [VALIDATOR ] LB: Validating primary data. [2025-10-20T19:39:35.700131] [DEBUG] [VALIDATOR ] LB: All expected trading days are present. [2025-10-20T19:39:35.700206] [DEBUG] [VALIDATOR ] LB: Primary data validation passed. [2025-10-20T19:39:35.701013] [INFO ] [MAIN ] LB: Fetched primary data is valid. [2025-10-20T19:39:35.701074] [DEBUG] [LOCAL ] LB: Saving primary data. [2025-10-20T19:39:35.702115] [DEBUG] [LOCAL ] LB: Trimmed 55925 records to 37212 (keeping 200 days of data) [2025-10-20T19:39:35.956785] [DEBUG] [LOCAL ] LB: Primary data saved (37212 records). [2025-10-20T19:39:35.956994] [INFO ] [MAIN ] LB: Full history saved. [2025-10-20T19:39:35.957109] [INFO ] [MAIN ] Update Progress: 1239/2367 symbols completed. LB took 30.1s. Estimated remaining: 2h 25m 14s. Rate: 7.8 symbols/min [2025-10-20T19:39:36.613980] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:39:36.629153] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-07. [2025-10-20T19:39:38.392273] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:39:38.403783] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-08. [2025-10-20T19:39:40.034400] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T19:39:40.046493] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-09. [2025-10-20T19:39:41.906329] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T19:39:41.917775] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-10. [2025-10-20T19:39:43.835533] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T19:39:43.844171] [INFO ] [REMOTE ] LBRT: Full fetch complete (76910 records). [2025-10-20T19:39:43.844321] [DEBUG] [VALIDATOR ] LBRT: Validating primary data. [2025-10-20T19:39:44.240147] [DEBUG] [VALIDATOR ] LBRT: All expected trading days are present. [2025-10-20T19:39:44.240254] [DEBUG] [VALIDATOR ] LBRT: Primary data validation passed. [2025-10-20T19:39:44.241256] [INFO ] [MAIN ] LBRT: Fetched primary data is valid. [2025-10-20T19:39:44.241323] [DEBUG] [LOCAL ] LBRT: Saving primary data. [2025-10-20T19:39:44.242764] [DEBUG] [LOCAL ] LBRT: Trimmed 76910 records to 53003 (keeping 200 days of data) [2025-10-20T19:39:44.568221] [DEBUG] [LOCAL ] LBRT: Primary data saved (53003 records). [2025-10-20T19:39:44.568380] [INFO ] [MAIN ] LBRT: Full history saved. [2025-10-20T19:39:44.569191] [INFO ] [MAIN ] Update Progress: 1240/2367 symbols completed. LBRT took 24.5s. Estimated remaining: 2h 25m 7s. Rate: 7.8 symbols/min [2025-10-20T19:39:45.226481] [INFO ] [MAIN ] LDI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:45.255891] [INFO ] [MAIN ] LCII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:45.303552] [INFO ] [MAIN ] LDOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:45.308807] [INFO ] [MAIN ] LC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:45.348353] [DEBUG] [LOCAL ] LCII: Loaded 27730 existing records from primary storage. [2025-10-20T19:39:45.358860] [INFO ] [MAIN ] LCII: Loaded 27730 existing records. [2025-10-20T19:39:45.487461] [DEBUG] [LOCAL ] LDI: Loaded 45582 existing records from primary storage. [2025-10-20T19:39:45.500948] [INFO ] [MAIN ] LDI: Loaded 45582 existing records. [2025-10-20T19:39:45.553234] [DEBUG] [LOCAL ] LDOS: Loaded 45546 existing records from primary storage. [2025-10-20T19:39:45.638424] [INFO ] [MAIN ] LDOS: Loaded 45546 existing records. [2025-10-20T19:39:45.656989] [DEBUG] [LOCAL ] LC: Loaded 48927 existing records from primary storage. [2025-10-20T19:39:45.763429] [WARN ] [MAIN ] LCII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:45.797195] [INFO ] [MAIN ] LCII: Wiping existing primary data before full refresh. [2025-10-20T19:39:45.730317] [INFO ] [MAIN ] LC: Loaded 48927 existing records. [2025-10-20T19:39:45.813235] [INFO ] [LOCAL ] LCII: Starting deletion of primary data. [2025-10-20T19:39:45.967453] [DEBUG] [LOCAL ] LCII: Deleted primary data file. [2025-10-20T19:39:46.005725] [WARN ] [MAIN ] LDI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:46.031705] [INFO ] [MAIN ] LCII: Starting full primary refresh. [2025-10-20T19:39:46.031664] [INFO ] [MAIN ] LDI: Wiping existing primary data before full refresh. [2025-10-20T19:39:46.140213] [WARN ] [MAIN ] LDOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:46.113005] [INFO ] [LOCAL ] LDI: Starting deletion of primary data. [2025-10-20T19:39:46.198090] [DEBUG] [LOCAL ] LDI: Deleted primary data file. [2025-10-20T19:39:46.192878] [INFO ] [MAIN ] LDOS: Wiping existing primary data before full refresh. [2025-10-20T19:39:46.256827] [INFO ] [LOCAL ] LDOS: Starting deletion of primary data. [2025-10-20T19:39:46.287284] [INFO ] [MAIN ] LDI: Starting full primary refresh. [2025-10-20T19:39:46.287176] [DEBUG] [LOCAL ] LDOS: Deleted primary data file. [2025-10-20T19:39:46.290753] [WARN ] [MAIN ] LC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:46.301461] [INFO ] [MAIN ] LC: Wiping existing primary data before full refresh. [2025-10-20T19:39:46.307421] [INFO ] [MAIN ] LDOS: Starting full primary refresh. [2025-10-20T19:39:46.312758] [INFO ] [LOCAL ] LC: Starting deletion of primary data. [2025-10-20T19:39:46.456588] [DEBUG] [LOCAL ] LC: Deleted primary data file. [2025-10-20T19:39:46.528133] [INFO ] [MAIN ] LC: Starting full primary refresh. [2025-10-20T19:39:47.147579] [INFO ] [REMOTE ] LCII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:47.281904] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-01. [2025-10-20T19:39:47.527636] [INFO ] [REMOTE ] LDI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:47.614389] [INFO ] [REMOTE ] LDOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:47.603869] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-01. [2025-10-20T19:39:47.625796] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-01. [2025-10-20T19:39:47.706860] [INFO ] [REMOTE ] LC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:47.707157] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-01. [2025-10-20T19:39:48.559343] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T19:39:48.564877] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-02. [2025-10-20T19:39:48.961563] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T19:39:48.968409] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-02. [2025-10-20T19:39:49.076460] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T19:39:49.086314] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-02. [2025-10-20T19:39:49.398554] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:39:49.409799] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-02. [2025-10-20T19:39:49.733443] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T19:39:49.738867] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-03. [2025-10-20T19:39:50.428538] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:39:50.435766] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-03. [2025-10-20T19:39:50.822863] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:39:50.833184] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-03. [2025-10-20T19:39:50.992712] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T19:39:51.002736] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-03. [2025-10-20T19:39:51.026546] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T19:39:51.033472] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-04. [2025-10-20T19:39:51.970995] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:39:51.979897] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-04. [2025-10-20T19:39:52.453036] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:39:52.463696] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-04. [2025-10-20T19:39:52.600786] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T19:39:52.609037] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-05. [2025-10-20T19:39:52.679869] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T19:39:52.689943] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-04. [2025-10-20T19:39:53.104162] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T19:39:53.113437] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-05. [2025-10-20T19:39:53.886030] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T19:39:53.893741] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-06. [2025-10-20T19:39:54.183467] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T19:39:54.194170] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-05. [2025-10-20T19:39:54.724952] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:39:54.733249] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-06. [2025-10-20T19:39:54.854255] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T19:39:54.881892] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T19:39:55.451345] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-05. [2025-10-20T19:39:55.892703] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:39:55.985597] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T19:39:56.251400] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-06. [2025-10-20T19:39:57.051358] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-07. [2025-10-20T19:39:57.150299] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:39:57.851392] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-06. [2025-10-20T19:39:57.933327] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T19:39:58.286780] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T19:39:58.651412] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-07. [2025-10-20T19:39:59.346627] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T19:39:59.451401] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-08. [2025-10-20T19:40:00.251707] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-07. [2025-10-20T19:40:00.254227] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:40:00.816909] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T19:40:01.051368] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-08. [2025-10-20T19:40:01.851387] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-07. [2025-10-20T19:40:02.258802] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T19:40:02.651377] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-09. [2025-10-20T19:40:03.178014] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T19:40:03.451363] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-08. [2025-10-20T19:40:03.622409] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:40:03.734164] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T19:40:04.251400] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-09. [2025-10-20T19:40:05.051369] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-08. [2025-10-20T19:40:05.283353] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T19:40:05.851452] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-09. [2025-10-20T19:40:06.514625] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-09 (2.26 seconds). [2025-10-20T19:40:06.564800] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:40:06.651383] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-10. [2025-10-20T19:40:07.451409] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-10. [2025-10-20T19:40:07.515536] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T19:40:08.251408] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-09. [2025-10-20T19:40:08.623042] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T19:40:08.627698] [INFO ] [REMOTE ] LCII: Full fetch complete (39104 records). [2025-10-20T19:40:08.627963] [DEBUG] [VALIDATOR ] LCII: Validating primary data. [2025-10-20T19:40:09.013263] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-10 (2.36 seconds). [2025-10-20T19:40:09.051310] [DEBUG] [VALIDATOR ] LCII: All expected trading days are present. [2025-10-20T19:40:09.051385] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-10. [2025-10-20T19:40:09.051525] [DEBUG] [VALIDATOR ] LCII: Primary data validation passed. [2025-10-20T19:40:09.052359] [INFO ] [MAIN ] LCII: Fetched primary data is valid. [2025-10-20T19:40:09.053358] [DEBUG] [LOCAL ] LCII: Saving primary data. [2025-10-20T19:40:09.052464] [INFO ] [REMOTE ] LDI: Full fetch complete (61986 records). [2025-10-20T19:40:09.053460] [DEBUG] [VALIDATOR ] LDI: Validating primary data. [2025-10-20T19:40:09.097917] [DEBUG] [LOCAL ] LCII: Trimmed 39104 records to 27217 (keeping 200 days of data) [2025-10-20T19:40:09.550719] [DEBUG] [LOCAL ] LCII: Primary data saved (27217 records). [2025-10-20T19:40:09.617110] [INFO ] [MAIN ] LCII: Full history saved. [2025-10-20T19:40:09.617485] [DEBUG] [VALIDATOR ] LDI: All expected trading days are present. [2025-10-20T19:40:09.617788] [INFO ] [MAIN ] Update Progress: 1241/2367 symbols completed. LCII took 25.0s. Estimated remaining: 2h 25m 15s. Rate: 7.8 symbols/min [2025-10-20T19:40:09.623232] [DEBUG] [VALIDATOR ] LDI: Primary data validation passed. [2025-10-20T19:40:09.632226] [INFO ] [MAIN ] LDI: Fetched primary data is valid. [2025-10-20T19:40:09.632320] [DEBUG] [LOCAL ] LDI: Saving primary data. [2025-10-20T19:40:09.705451] [DEBUG] [LOCAL ] LDI: Trimmed 61986 records to 44983 (keeping 200 days of data) [2025-10-20T19:40:09.926963] [INFO ] [MAIN ] LDP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:10.063498] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T19:40:10.085080] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-10. [2025-10-20T19:40:10.095488] [DEBUG] [LOCAL ] LDP: Loaded 12949 existing records from primary storage. [2025-10-20T19:40:10.099155] [INFO ] [MAIN ] LDP: Loaded 12949 existing records. [2025-10-20T19:40:10.252224] [DEBUG] [LOCAL ] LDI: Primary data saved (44983 records). [2025-10-20T19:40:10.269352] [INFO ] [MAIN ] LDI: Full history saved. [2025-10-20T19:40:10.269481] [INFO ] [MAIN ] Update Progress: 1242/2367 symbols completed. LDI took 25.7s. Estimated remaining: 2h 25m 1s. Rate: 7.8 symbols/min [2025-10-20T19:40:10.491383] [WARN ] [MAIN ] LDP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:10.525745] [INFO ] [MAIN ] LDP: Wiping existing primary data before full refresh. [2025-10-20T19:40:10.545808] [INFO ] [MAIN ] LEA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:10.546159] [DEBUG] [LOCAL ] LEA: No existing data file found in primary storage. [2025-10-20T19:40:10.546253] [WARN ] [MAIN ] LEA: No existing data found, performing full refresh instead. [2025-10-20T19:40:10.546018] [INFO ] [LOCAL ] LDP: Starting deletion of primary data. [2025-10-20T19:40:10.546740] [INFO ] [MAIN ] LEA: Wiping existing primary data before full refresh. [2025-10-20T19:40:10.546808] [INFO ] [LOCAL ] LEA: Starting deletion of primary data. [2025-10-20T19:40:10.546854] [DEBUG] [LOCAL ] LEA: No primary data file found to delete. [2025-10-20T19:40:10.546885] [INFO ] [MAIN ] LEA: Starting full primary refresh. [2025-10-20T19:40:10.578125] [DEBUG] [LOCAL ] LDP: Deleted primary data file. [2025-10-20T19:40:10.627232] [INFO ] [MAIN ] LDP: Starting full primary refresh. [2025-10-20T19:40:11.250784] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T19:40:11.302956] [INFO ] [REMOTE ] LEA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:11.329478] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-01. [2025-10-20T19:40:11.341585] [INFO ] [REMOTE ] LC: Full fetch complete (70701 records). [2025-10-20T19:40:11.362615] [DEBUG] [VALIDATOR ] LC: Validating primary data. [2025-10-20T19:40:11.369834] [INFO ] [REMOTE ] LDP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:11.456556] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-01. [2025-10-20T19:40:11.833049] [DEBUG] [VALIDATOR ] LC: All expected trading days are present. [2025-10-20T19:40:11.833197] [DEBUG] [VALIDATOR ] LC: Primary data validation passed. [2025-10-20T19:40:11.835137] [INFO ] [MAIN ] LC: Fetched primary data is valid. [2025-10-20T19:40:11.835346] [DEBUG] [LOCAL ] LC: Saving primary data. [2025-10-20T19:40:11.854094] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T19:40:11.860855] [INFO ] [REMOTE ] LDOS: Full fetch complete (66980 records). [2025-10-20T19:40:11.861199] [DEBUG] [VALIDATOR ] LDOS: Validating primary data. [2025-10-20T19:40:11.861102] [DEBUG] [LOCAL ] LC: Trimmed 70701 records to 48258 (keeping 200 days of data) [2025-10-20T19:40:12.556761] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:40:12.585596] [DEBUG] [VALIDATOR ] LDOS: All expected trading days are present. [2025-10-20T19:40:12.605095] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-02. [2025-10-20T19:40:12.600997] [DEBUG] [VALIDATOR ] LDOS: Primary data validation passed. [2025-10-20T19:40:12.606797] [INFO ] [MAIN ] LDOS: Fetched primary data is valid. [2025-10-20T19:40:12.641735] [DEBUG] [LOCAL ] LDOS: Saving primary data. [2025-10-20T19:40:12.659206] [DEBUG] [LOCAL ] LC: Primary data saved (48258 records). [2025-10-20T19:40:12.659526] [INFO ] [MAIN ] LC: Full history saved. [2025-10-20T19:40:12.659891] [INFO ] [MAIN ] Update Progress: 1243/2367 symbols completed. LC took 28.1s. Estimated remaining: 2h 24m 48s. Rate: 7.8 symbols/min [2025-10-20T19:40:12.660114] [DEBUG] [LOCAL ] LDOS: Trimmed 66980 records to 44754 (keeping 200 days of data) [2025-10-20T19:40:12.972618] [INFO ] [MAIN ] LEG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:13.127686] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T19:40:13.137999] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-02. [2025-10-20T19:40:13.191987] [DEBUG] [LOCAL ] LEG: Loaded 50873 existing records from primary storage. [2025-10-20T19:40:13.206099] [INFO ] [MAIN ] LEG: Loaded 50873 existing records. [2025-10-20T19:40:13.319305] [DEBUG] [LOCAL ] LDOS: Primary data saved (44754 records). [2025-10-20T19:40:13.322213] [INFO ] [MAIN ] LDOS: Full history saved. [2025-10-20T19:40:13.322555] [INFO ] [MAIN ] Update Progress: 1244/2367 symbols completed. LDOS took 28.7s. Estimated remaining: 2h 24m 34s. Rate: 7.8 symbols/min [2025-10-20T19:40:13.540843] [WARN ] [MAIN ] LEG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:13.551464] [INFO ] [MAIN ] LEG: Wiping existing primary data before full refresh. [2025-10-20T19:40:13.584229] [INFO ] [LOCAL ] LEG: Starting deletion of primary data. [2025-10-20T19:40:13.605693] [DEBUG] [LOCAL ] LEG: Deleted primary data file. [2025-10-20T19:40:13.605855] [INFO ] [MAIN ] LEG: Starting full primary refresh. [2025-10-20T19:40:13.605602] [INFO ] [MAIN ] LEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:13.721864] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T19:40:13.858126] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-03. [2025-10-20T19:40:14.019280] [INFO ] [REMOTE ] LEG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:14.020915] [DEBUG] [LOCAL ] LEN: Loaded 53973 existing records from primary storage. [2025-10-20T19:40:14.021324] [INFO ] [MAIN ] LEN: Loaded 53973 existing records. [2025-10-20T19:40:14.201508] [WARN ] [MAIN ] LEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:14.201771] [INFO ] [MAIN ] LEN: Wiping existing primary data before full refresh. [2025-10-20T19:40:14.201820] [INFO ] [LOCAL ] LEN: Starting deletion of primary data. [2025-10-20T19:40:14.202035] [DEBUG] [LOCAL ] LEN: Deleted primary data file. [2025-10-20T19:40:14.202089] [INFO ] [MAIN ] LEN: Starting full primary refresh. [2025-10-20T19:40:14.573510] [INFO ] [REMOTE ] LEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:14.651354] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-01. [2025-10-20T19:40:14.723583] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T19:40:14.883465] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T19:40:15.451375] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-04. [2025-10-20T19:40:16.219810] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T19:40:16.251564] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:40:16.251687] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-05. [2025-10-20T19:40:17.051372] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-01. [2025-10-20T19:40:17.230359] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T19:40:17.851392] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-02. [2025-10-20T19:40:18.651432] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-06. [2025-10-20T19:40:18.870897] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T19:40:19.451419] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-02. [2025-10-20T19:40:19.540167] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T19:40:19.567865] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T19:40:20.251428] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-07. [2025-10-20T19:40:21.051345] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-03. [2025-10-20T19:40:21.131310] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T19:40:21.249305] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T19:40:21.851406] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-03. [2025-10-20T19:40:22.651351] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-03. [2025-10-20T19:40:22.727320] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T19:40:23.451373] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-04. [2025-10-20T19:40:23.482186] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:40:24.251361] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-04. [2025-10-20T19:40:24.312553] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T19:40:24.809427] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T19:40:25.051376] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-08. [2025-10-20T19:40:25.789372] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T19:40:25.851341] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-05. [2025-10-20T19:40:25.888399] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T19:40:26.651384] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-05. [2025-10-20T19:40:27.290540] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T19:40:27.451410] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-06. [2025-10-20T19:40:28.260733] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-04. [2025-10-20T19:40:28.265793] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:40:29.007926] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:40:29.051334] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-07. [2025-10-20T19:40:29.851352] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-06. [2025-10-20T19:40:30.245782] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T19:40:30.548522] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:40:30.651339] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-08. [2025-10-20T19:40:31.461330] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-05. [2025-10-20T19:40:31.485919] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T19:40:32.084042] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T19:40:32.251408] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-09. [2025-10-20T19:40:33.051413] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-07. [2025-10-20T19:40:33.404455] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T19:40:33.852288] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-06. [2025-10-20T19:40:33.859829] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T19:40:34.651372] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-10. [2025-10-20T19:40:34.753568] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:40:35.451382] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-08. [2025-10-20T19:40:35.695535] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T19:40:35.829756] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T19:40:35.835761] [INFO ] [REMOTE ] LEA: Full fetch complete (60683 records). [2025-10-20T19:40:35.835877] [DEBUG] [VALIDATOR ] LEA: Validating primary data. [2025-10-20T19:40:36.207857] [DEBUG] [VALIDATOR ] LEA: All expected trading days are present. [2025-10-20T19:40:36.207938] [DEBUG] [VALIDATOR ] LEA: Primary data validation passed. [2025-10-20T19:40:36.208809] [INFO ] [MAIN ] LEA: Fetched primary data is valid. [2025-10-20T19:40:36.208871] [DEBUG] [LOCAL ] LEA: Saving primary data. [2025-10-20T19:40:36.210069] [DEBUG] [LOCAL ] LEA: Trimmed 60683 records to 41313 (keeping 200 days of data) [2025-10-20T19:40:36.262476] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-07. [2025-10-20T19:40:36.462046] [DEBUG] [LOCAL ] LEA: Primary data saved (41313 records). [2025-10-20T19:40:36.462170] [INFO ] [MAIN ] LEA: Full history saved. [2025-10-20T19:40:36.462235] [INFO ] [MAIN ] Update Progress: 1245/2367 symbols completed. LEA took 26.2s. Estimated remaining: 2h 24m 40s. Rate: 7.8 symbols/min [2025-10-20T19:40:36.629284] [INFO ] [MAIN ] LEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:36.646747] [DEBUG] [LOCAL ] LEO: Loaded 16814 existing records from primary storage. [2025-10-20T19:40:36.646986] [INFO ] [MAIN ] LEO: Loaded 16814 existing records. [2025-10-20T19:40:36.850298] [WARN ] [MAIN ] LEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:36.850621] [INFO ] [MAIN ] LEO: Wiping existing primary data before full refresh. [2025-10-20T19:40:36.850773] [INFO ] [LOCAL ] LEO: Starting deletion of primary data. [2025-10-20T19:40:36.851073] [DEBUG] [LOCAL ] LEO: Deleted primary data file. [2025-10-20T19:40:36.851135] [INFO ] [MAIN ] LEO: Starting full primary refresh. [2025-10-20T19:40:37.040160] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:40:37.076405] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-09. [2025-10-20T19:40:37.266909] [INFO ] [REMOTE ] LEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:37.851367] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-01. [2025-10-20T19:40:37.963150] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T19:40:38.290218] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T19:40:38.651349] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-10. [2025-10-20T19:40:38.895198] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:40:39.451384] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-08. [2025-10-20T19:40:39.534054] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T19:40:39.536966] [INFO ] [REMOTE ] LDP: Full fetch complete (18779 records). [2025-10-20T19:40:39.537067] [DEBUG] [VALIDATOR ] LDP: Validating primary data. [2025-10-20T19:40:39.916991] [DEBUG] [VALIDATOR ] LDP: All expected trading days are present. [2025-10-20T19:40:39.917080] [DEBUG] [VALIDATOR ] LDP: Primary data validation passed. [2025-10-20T19:40:39.917532] [INFO ] [MAIN ] LDP: Fetched primary data is valid. [2025-10-20T19:40:39.917577] [DEBUG] [LOCAL ] LDP: Saving primary data. [2025-10-20T19:40:39.918261] [DEBUG] [LOCAL ] LDP: Trimmed 18779 records to 12697 (keeping 200 days of data) [2025-10-20T19:40:40.002051] [DEBUG] [LOCAL ] LDP: Primary data saved (12697 records). [2025-10-20T19:40:40.002195] [INFO ] [MAIN ] LDP: Full history saved. [2025-10-20T19:40:40.002277] [INFO ] [MAIN ] Update Progress: 1246/2367 symbols completed. LDP took 30.4s. Estimated remaining: 2h 24m 29s. Rate: 7.8 symbols/min [2025-10-20T19:40:40.184287] [INFO ] [MAIN ] LEVI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:40.238444] [DEBUG] [LOCAL ] LEVI: Loaded 52559 existing records from primary storage. [2025-10-20T19:40:40.238854] [INFO ] [MAIN ] LEVI: Loaded 52559 existing records. [2025-10-20T19:40:40.259941] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-02. [2025-10-20T19:40:40.427271] [WARN ] [MAIN ] LEVI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:40.427524] [INFO ] [MAIN ] LEVI: Wiping existing primary data before full refresh. [2025-10-20T19:40:40.427572] [INFO ] [LOCAL ] LEVI: Starting deletion of primary data. [2025-10-20T19:40:40.428086] [DEBUG] [LOCAL ] LEVI: Deleted primary data file. [2025-10-20T19:40:40.428143] [INFO ] [MAIN ] LEVI: Starting full primary refresh. [2025-10-20T19:40:40.782306] [INFO ] [REMOTE ] LEVI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:41.051364] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-09. [2025-10-20T19:40:41.352398] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T19:40:41.533541] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T19:40:41.851372] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-03. [2025-10-20T19:40:42.651389] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-09. [2025-10-20T19:40:42.833914] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:40:42.863340] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:40:43.451359] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-04. [2025-10-20T19:40:44.251406] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-01. [2025-10-20T19:40:44.478181] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T19:40:44.954947] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-09 (2.30 seconds). [2025-10-20T19:40:45.051366] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-05. [2025-10-20T19:40:45.851410] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-10. [2025-10-20T19:40:45.976080] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T19:40:45.975516] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T19:40:46.651377] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-10. [2025-10-20T19:40:47.451388] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-06. [2025-10-20T19:40:47.654881] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T19:40:47.662316] [INFO ] [REMOTE ] LEG: Full fetch complete (72913 records). [2025-10-20T19:40:47.662466] [DEBUG] [VALIDATOR ] LEG: Validating primary data. [2025-10-20T19:40:48.053439] [DEBUG] [VALIDATOR ] LEG: All expected trading days are present. [2025-10-20T19:40:48.053529] [DEBUG] [VALIDATOR ] LEG: Primary data validation passed. [2025-10-20T19:40:48.054475] [INFO ] [MAIN ] LEG: Fetched primary data is valid. [2025-10-20T19:40:48.054537] [DEBUG] [LOCAL ] LEG: Saving primary data. [2025-10-20T19:40:48.055966] [DEBUG] [LOCAL ] LEG: Trimmed 72913 records to 50094 (keeping 200 days of data) [2025-10-20T19:40:48.260296] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-02. [2025-10-20T19:40:48.383256] [DEBUG] [LOCAL ] LEG: Primary data saved (50094 records). [2025-10-20T19:40:48.383437] [INFO ] [MAIN ] LEG: Full history saved. [2025-10-20T19:40:48.383814] [INFO ] [MAIN ] Update Progress: 1247/2367 symbols completed. LEG took 35.7s. Estimated remaining: 2h 24m 22s. Rate: 7.8 symbols/min [2025-10-20T19:40:48.500894] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T19:40:48.577827] [INFO ] [MAIN ] LFT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:48.596984] [DEBUG] [LOCAL ] LFT: Loaded 18518 existing records from primary storage. [2025-10-20T19:40:48.597247] [INFO ] [MAIN ] LFT: Loaded 18518 existing records. [2025-10-20T19:40:48.776476] [WARN ] [MAIN ] LFT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:48.776742] [INFO ] [MAIN ] LFT: Wiping existing primary data before full refresh. [2025-10-20T19:40:48.776794] [INFO ] [LOCAL ] LFT: Starting deletion of primary data. [2025-10-20T19:40:48.777046] [DEBUG] [LOCAL ] LFT: Deleted primary data file. [2025-10-20T19:40:48.777086] [INFO ] [MAIN ] LFT: Starting full primary refresh. [2025-10-20T19:40:49.054773] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-07. [2025-10-20T19:40:49.148691] [INFO ] [REMOTE ] LFT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:49.170120] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-10 (2.52 seconds). [2025-10-20T19:40:49.177941] [INFO ] [REMOTE ] LEN: Full fetch complete (77214 records). [2025-10-20T19:40:49.178089] [DEBUG] [VALIDATOR ] LEN: Validating primary data. [2025-10-20T19:40:49.560612] [DEBUG] [VALIDATOR ] LEN: All expected trading days are present. [2025-10-20T19:40:49.560864] [DEBUG] [VALIDATOR ] LEN: Primary data validation passed. [2025-10-20T19:40:49.561890] [INFO ] [MAIN ] LEN: Fetched primary data is valid. [2025-10-20T19:40:49.561953] [DEBUG] [LOCAL ] LEN: Saving primary data. [2025-10-20T19:40:49.563422] [DEBUG] [LOCAL ] LEN: Trimmed 77214 records to 53202 (keeping 200 days of data) [2025-10-20T19:40:49.857486] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-01. [2025-10-20T19:40:49.914976] [DEBUG] [LOCAL ] LEN: Primary data saved (53202 records). [2025-10-20T19:40:49.915299] [INFO ] [MAIN ] LEN: Full history saved. [2025-10-20T19:40:49.915489] [INFO ] [MAIN ] Update Progress: 1248/2367 symbols completed. LEN took 36.6s. Estimated remaining: 2h 24m 8s. Rate: 7.8 symbols/min [2025-10-20T19:40:50.138251] [INFO ] [MAIN ] LGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:50.147959] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T19:40:50.148470] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T19:40:50.174572] [DEBUG] [LOCAL ] LGI: Loaded 10171 existing records from primary storage. [2025-10-20T19:40:50.174859] [INFO ] [MAIN ] LGI: Loaded 10171 existing records. [2025-10-20T19:40:50.382895] [WARN ] [MAIN ] LGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:50.383154] [INFO ] [MAIN ] LGI: Wiping existing primary data before full refresh. [2025-10-20T19:40:50.383229] [INFO ] [LOCAL ] LGI: Starting deletion of primary data. [2025-10-20T19:40:50.383484] [DEBUG] [LOCAL ] LGI: Deleted primary data file. [2025-10-20T19:40:50.383552] [INFO ] [MAIN ] LGI: Starting full primary refresh. [2025-10-20T19:40:50.652911] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-03. [2025-10-20T19:40:50.798437] [INFO ] [REMOTE ] LGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:50.858531] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T19:40:51.451403] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-02. [2025-10-20T19:40:52.251365] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-01. [2025-10-20T19:40:52.273847] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T19:40:52.469669] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T19:40:53.011983] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T19:40:53.051340] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-02. [2025-10-20T19:40:53.796096] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T19:40:53.851438] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-03. [2025-10-20T19:40:54.624892] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T19:40:54.651379] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-08. [2025-10-20T19:40:55.451360] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-04. [2025-10-20T19:40:55.562480] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T19:40:56.240882] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:40:56.251354] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-05. [2025-10-20T19:40:57.041824] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T19:40:57.051339] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-06. [2025-10-20T19:40:57.771675] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T19:40:57.851376] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-03. [2025-10-20T19:40:58.651362] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-04. [2025-10-20T19:40:58.739573] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T19:40:59.451377] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-09. [2025-10-20T19:41:00.251376] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-07. [2025-10-20T19:41:00.423851] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:41:00.526078] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T19:41:01.051417] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-05. [2025-10-20T19:41:01.249959] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T19:41:01.851395] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-08. [2025-10-20T19:41:02.651362] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-04. [2025-10-20T19:41:02.871558] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T19:41:03.083419] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T19:41:03.451372] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-06. [2025-10-20T19:41:03.779346] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T19:41:04.251422] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-09. [2025-10-20T19:41:04.988167] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:41:05.059150] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-10. [2025-10-20T19:41:05.077568] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T19:41:05.851353] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-05. [2025-10-20T19:41:05.982412] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T19:41:05.984500] [INFO ] [REMOTE ] LGI: Full fetch complete (14465 records). [2025-10-20T19:41:05.984571] [DEBUG] [VALIDATOR ] LGI: Validating primary data. [2025-10-20T19:41:06.358991] [DEBUG] [VALIDATOR ] LGI: All expected trading days are present. [2025-10-20T19:41:06.359094] [DEBUG] [VALIDATOR ] LGI: Primary data validation passed. [2025-10-20T19:41:06.359458] [INFO ] [MAIN ] LGI: Fetched primary data is valid. [2025-10-20T19:41:06.359499] [DEBUG] [LOCAL ] LGI: Saving primary data. [2025-10-20T19:41:06.360201] [DEBUG] [LOCAL ] LGI: Trimmed 14465 records to 10042 (keeping 200 days of data) [2025-10-20T19:41:06.422061] [DEBUG] [LOCAL ] LGI: Primary data saved (10042 records). [2025-10-20T19:41:06.422196] [INFO ] [MAIN ] LGI: Full history saved. [2025-10-20T19:41:06.422274] [INFO ] [MAIN ] Update Progress: 1249/2367 symbols completed. LGI took 16.5s. Estimated remaining: 2h 24m 9s. Rate: 7.8 symbols/min [2025-10-20T19:41:06.601955] [INFO ] [MAIN ] LH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:06.649388] [DEBUG] [LOCAL ] LH: Loaded 38017 existing records from primary storage. [2025-10-20T19:41:06.649685] [INFO ] [MAIN ] LH: Loaded 38017 existing records. [2025-10-20T19:41:06.660826] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-10. [2025-10-20T19:41:06.833273] [WARN ] [MAIN ] LH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:06.833596] [INFO ] [MAIN ] LH: Wiping existing primary data before full refresh. [2025-10-20T19:41:06.833735] [INFO ] [LOCAL ] LH: Starting deletion of primary data. [2025-10-20T19:41:06.833999] [DEBUG] [LOCAL ] LH: Deleted primary data file. [2025-10-20T19:41:06.834043] [INFO ] [MAIN ] LH: Starting full primary refresh. [2025-10-20T19:41:06.953574] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T19:41:07.208836] [INFO ] [REMOTE ] LH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:07.451390] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-01. [2025-10-20T19:41:07.631409] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T19:41:07.634348] [INFO ] [REMOTE ] LEO: Full fetch complete (26292 records). [2025-10-20T19:41:07.634423] [DEBUG] [VALIDATOR ] LEO: Validating primary data. [2025-10-20T19:41:08.025331] [DEBUG] [VALIDATOR ] LEO: All expected trading days are present. [2025-10-20T19:41:08.025436] [DEBUG] [VALIDATOR ] LEO: Primary data validation passed. [2025-10-20T19:41:08.026017] [INFO ] [MAIN ] LEO: Fetched primary data is valid. [2025-10-20T19:41:08.026079] [DEBUG] [LOCAL ] LEO: Saving primary data. [2025-10-20T19:41:08.026860] [DEBUG] [LOCAL ] LEO: Trimmed 26292 records to 16617 (keeping 200 days of data) [2025-10-20T19:41:08.122337] [DEBUG] [LOCAL ] LEO: Primary data saved (16617 records). [2025-10-20T19:41:08.122484] [INFO ] [MAIN ] LEO: Full history saved. [2025-10-20T19:41:08.122563] [INFO ] [MAIN ] Update Progress: 1250/2367 symbols completed. LEO took 31.7s. Estimated remaining: 2h 23m 55s. Rate: 7.8 symbols/min [2025-10-20T19:41:08.259123] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-07. [2025-10-20T19:41:08.291429] [INFO ] [MAIN ] LHX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:08.336112] [DEBUG] [LOCAL ] LHX: Loaded 49301 existing records from primary storage. [2025-10-20T19:41:08.336420] [INFO ] [MAIN ] LHX: Loaded 49301 existing records. [2025-10-20T19:41:08.509437] [WARN ] [MAIN ] LHX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:08.509686] [INFO ] [MAIN ] LHX: Wiping existing primary data before full refresh. [2025-10-20T19:41:08.509752] [INFO ] [LOCAL ] LHX: Starting deletion of primary data. [2025-10-20T19:41:08.510421] [DEBUG] [LOCAL ] LHX: Deleted primary data file. [2025-10-20T19:41:08.510478] [INFO ] [MAIN ] LHX: Starting full primary refresh. [2025-10-20T19:41:08.872586] [INFO ] [REMOTE ] LHX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:09.040332] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T19:41:09.051336] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-02. [2025-10-20T19:41:09.851495] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-01. [2025-10-20T19:41:10.143913] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T19:41:10.240804] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T19:41:10.651394] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-03. [2025-10-20T19:41:11.451368] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-08. [2025-10-20T19:41:11.550677] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T19:41:12.256905] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-06. [2025-10-20T19:41:12.281950] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:41:13.051346] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-02. [2025-10-20T19:41:13.113046] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:41:13.439394] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T19:41:13.851434] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-04. [2025-10-20T19:41:14.651387] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-07. [2025-10-20T19:41:14.845682] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:41:15.451386] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-09. [2025-10-20T19:41:15.502269] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T19:41:15.556243] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:41:16.251428] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-05. [2025-10-20T19:41:17.051421] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-03. [2025-10-20T19:41:17.111087] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T19:41:17.851377] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-10. [2025-10-20T19:41:17.878629] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T19:41:18.651362] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-06. [2025-10-20T19:41:18.938737] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T19:41:19.451398] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-04. [2025-10-20T19:41:19.931518] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T19:41:19.939742] [INFO ] [REMOTE ] LEVI: Full fetch complete (74962 records). [2025-10-20T19:41:19.939913] [DEBUG] [VALIDATOR ] LEVI: Validating primary data. [2025-10-20T19:41:20.262385] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-08. [2025-10-20T19:41:20.435952] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T19:41:20.467126] [DEBUG] [VALIDATOR ] LEVI: All expected trading days are present. [2025-10-20T19:41:20.467542] [DEBUG] [VALIDATOR ] LEVI: Primary data validation passed. [2025-10-20T19:41:20.468586] [INFO ] [MAIN ] LEVI: Fetched primary data is valid. [2025-10-20T19:41:20.468672] [DEBUG] [LOCAL ] LEVI: Saving primary data. [2025-10-20T19:41:20.470017] [DEBUG] [LOCAL ] LEVI: Trimmed 74962 records to 51821 (keeping 200 days of data) [2025-10-20T19:41:20.780232] [DEBUG] [LOCAL ] LEVI: Primary data saved (51821 records). [2025-10-20T19:41:20.780374] [INFO ] [MAIN ] LEVI: Full history saved. [2025-10-20T19:41:20.780454] [INFO ] [MAIN ] Update Progress: 1251/2367 symbols completed. LEVI took 40.8s. Estimated remaining: 2h 23m 52s. Rate: 7.8 symbols/min [2025-10-20T19:41:20.951200] [INFO ] [MAIN ] LII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:20.992617] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T19:41:21.013960] [DEBUG] [LOCAL ] LII: Loaded 28209 existing records from primary storage. [2025-10-20T19:41:21.014256] [INFO ] [MAIN ] LII: Loaded 28209 existing records. [2025-10-20T19:41:21.056478] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-07. [2025-10-20T19:41:21.203473] [WARN ] [MAIN ] LII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:21.203705] [INFO ] [MAIN ] LII: Wiping existing primary data before full refresh. [2025-10-20T19:41:21.203763] [INFO ] [LOCAL ] LII: Starting deletion of primary data. [2025-10-20T19:41:21.203935] [DEBUG] [LOCAL ] LII: Deleted primary data file. [2025-10-20T19:41:21.203974] [INFO ] [MAIN ] LII: Starting full primary refresh. [2025-10-20T19:41:21.441902] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T19:41:21.581250] [INFO ] [REMOTE ] LII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:21.851443] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-05. [2025-10-20T19:41:22.595563] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:41:22.651357] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-01. [2025-10-20T19:41:23.451419] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-08. [2025-10-20T19:41:23.716374] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T19:41:24.026109] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:41:24.251437] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-06. [2025-10-20T19:41:24.822342] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T19:41:25.051387] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-09. [2025-10-20T19:41:25.851389] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-09. [2025-10-20T19:41:26.138561] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:41:26.305954] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T19:41:26.651403] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-10. [2025-10-20T19:41:26.828122] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T19:41:27.451406] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-10. [2025-10-20T19:41:28.093374] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T19:41:28.098441] [INFO ] [REMOTE ] LH: Full fetch complete (54915 records). [2025-10-20T19:41:28.098544] [DEBUG] [VALIDATOR ] LH: Validating primary data. [2025-10-20T19:41:28.272292] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-07. [2025-10-20T19:41:28.488523] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T19:41:28.508485] [DEBUG] [VALIDATOR ] LH: All expected trading days are present. [2025-10-20T19:41:28.508594] [INFO ] [REMOTE ] LFT: Full fetch complete (23879 records). [2025-10-20T19:41:28.508714] [DEBUG] [VALIDATOR ] LH: Primary data validation passed. [2025-10-20T19:41:28.508801] [DEBUG] [VALIDATOR ] LFT: Validating primary data. [2025-10-20T19:41:28.509801] [INFO ] [MAIN ] LH: Fetched primary data is valid. [2025-10-20T19:41:28.509933] [DEBUG] [LOCAL ] LH: Saving primary data. [2025-10-20T19:41:28.611819] [DEBUG] [LOCAL ] LH: Trimmed 54915 records to 37275 (keeping 200 days of data) [2025-10-20T19:41:29.014422] [DEBUG] [VALIDATOR ] LFT: All expected trading days are present. [2025-10-20T19:41:29.014510] [DEBUG] [VALIDATOR ] LFT: Primary data validation passed. [2025-10-20T19:41:29.061398] [INFO ] [MAIN ] LFT: Fetched primary data is valid. [2025-10-20T19:41:29.060762] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-02. [2025-10-20T19:41:29.062486] [DEBUG] [LOCAL ] LFT: Saving primary data. [2025-10-20T19:41:29.071954] [DEBUG] [LOCAL ] LFT: Trimmed 23879 records to 18230 (keeping 200 days of data) [2025-10-20T19:41:29.202515] [DEBUG] [LOCAL ] LH: Primary data saved (37275 records). [2025-10-20T19:41:29.203316] [INFO ] [MAIN ] LH: Full history saved. [2025-10-20T19:41:29.215725] [INFO ] [MAIN ] Update Progress: 1252/2367 symbols completed. LH took 22.8s. Estimated remaining: 2h 23m 45s. Rate: 7.8 symbols/min [2025-10-20T19:41:29.254233] [DEBUG] [LOCAL ] LFT: Primary data saved (18230 records). [2025-10-20T19:41:29.298143] [INFO ] [MAIN ] LFT: Full history saved. [2025-10-20T19:41:29.309836] [INFO ] [MAIN ] Update Progress: 1253/2367 symbols completed. LFT took 40.9s. Estimated remaining: 2h 23m 31s. Rate: 7.8 symbols/min [2025-10-20T19:41:29.550938] [INFO ] [MAIN ] LION: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:29.597598] [INFO ] [MAIN ] LITB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:29.605778] [DEBUG] [LOCAL ] LITB: Loaded 4239 existing records from primary storage. [2025-10-20T19:41:29.606027] [INFO ] [MAIN ] LITB: Loaded 4239 existing records. [2025-10-20T19:41:29.835426] [WARN ] [MAIN ] LITB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:29.836549] [DEBUG] [LOCAL ] LION: Loaded 47066 existing records from primary storage. [2025-10-20T19:41:29.836672] [INFO ] [MAIN ] LITB: Wiping existing primary data before full refresh. [2025-10-20T19:41:29.836909] [INFO ] [MAIN ] LION: Loaded 47066 existing records. [2025-10-20T19:41:29.836832] [INFO ] [LOCAL ] LITB: Starting deletion of primary data. [2025-10-20T19:41:29.860847] [DEBUG] [LOCAL ] LITB: Deleted primary data file. [2025-10-20T19:41:29.904739] [INFO ] [MAIN ] LITB: Starting full primary refresh. [2025-10-20T19:41:30.157375] [WARN ] [MAIN ] LION: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:30.181798] [INFO ] [MAIN ] LION: Wiping existing primary data before full refresh. [2025-10-20T19:41:30.192160] [INFO ] [LOCAL ] LION: Starting deletion of primary data. [2025-10-20T19:41:30.266137] [DEBUG] [LOCAL ] LION: Deleted primary data file. [2025-10-20T19:41:30.306444] [INFO ] [MAIN ] LION: Starting full primary refresh. [2025-10-20T19:41:30.546741] [INFO ] [REMOTE ] LITB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:30.557994] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-01. [2025-10-20T19:41:30.796047] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:41:30.840093] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-07 (2.57 seconds). [2025-10-20T19:41:30.846544] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-03. [2025-10-20T19:41:31.034347] [INFO ] [REMOTE ] LION: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:31.116215] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:41:31.451427] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-02. [2025-10-20T19:41:31.889865] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T19:41:32.256449] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-01. [2025-10-20T19:41:32.266914] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T19:41:33.041997] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:41:33.051339] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-03. [2025-10-20T19:41:33.671367] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T19:41:33.851405] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-04. [2025-10-20T19:41:34.587435] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T19:41:34.651407] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-02. [2025-10-20T19:41:35.451348] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-05. [2025-10-20T19:41:35.767288] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T19:41:35.927771] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T19:41:36.251367] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-03. [2025-10-20T19:41:37.051350] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-06. [2025-10-20T19:41:37.301809] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:41:37.489845] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T19:41:37.851362] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-04. [2025-10-20T19:41:38.651411] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-08. [2025-10-20T19:41:39.393107] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T19:41:39.451353] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-05. [2025-10-20T19:41:40.251351] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-04. [2025-10-20T19:41:40.434055] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T19:41:40.774733] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T19:41:41.051403] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-06. [2025-10-20T19:41:41.134738] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T19:41:41.851377] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-07. [2025-10-20T19:41:42.192540] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:41:42.299682] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T19:41:42.651382] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-09. [2025-10-20T19:41:43.451373] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-08. [2025-10-20T19:41:43.916141] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T19:41:44.251382] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-07. [2025-10-20T19:41:44.415575] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T19:41:45.051359] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-09. [2025-10-20T19:41:45.556672] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:41:45.569297] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-10-20T19:41:45.851373] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-08. [2025-10-20T19:41:46.651341] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-05. [2025-10-20T19:41:47.039312] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T19:41:47.451361] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-09. [2025-10-20T19:41:48.205418] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:41:48.251391] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-06. [2025-10-20T19:41:48.598141] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:41:49.051395] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-10. [2025-10-20T19:41:49.627744] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T19:41:49.628902] [INFO ] [REMOTE ] LITB: Full fetch complete (5746 records). [2025-10-20T19:41:49.628969] [DEBUG] [VALIDATOR ] LITB: Validating primary data. [2025-10-20T19:41:49.856684] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-10. [2025-10-20T19:41:50.055368] [DEBUG] [VALIDATOR ] LITB: All expected trading days are present. [2025-10-20T19:41:50.055615] [DEBUG] [VALIDATOR ] LITB: Primary data validation passed. [2025-10-20T19:41:50.057845] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T19:41:50.056238] [INFO ] [MAIN ] LITB: Fetched primary data is valid. [2025-10-20T19:41:50.057973] [DEBUG] [LOCAL ] LITB: Saving primary data. [2025-10-20T19:41:50.068714] [DEBUG] [LOCAL ] LITB: Trimmed 5746 records to 3871 (keeping 200 days of data) [2025-10-20T19:41:50.090303] [DEBUG] [LOCAL ] LITB: Primary data saved (3871 records). [2025-10-20T19:41:50.090431] [INFO ] [MAIN ] LITB: Full history saved. [2025-10-20T19:41:50.090511] [INFO ] [MAIN ] Update Progress: 1254/2367 symbols completed. LITB took 20.8s. Estimated remaining: 2h 23m 34s. Rate: 7.8 symbols/min [2025-10-20T19:41:50.273078] [INFO ] [MAIN ] LLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:50.328534] [DEBUG] [LOCAL ] LLY: Loaded 54022 existing records from primary storage. [2025-10-20T19:41:50.329071] [INFO ] [MAIN ] LLY: Loaded 54022 existing records. [2025-10-20T19:41:50.526580] [WARN ] [MAIN ] LLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:50.526879] [INFO ] [MAIN ] LLY: Wiping existing primary data before full refresh. [2025-10-20T19:41:50.526931] [INFO ] [LOCAL ] LLY: Starting deletion of primary data. [2025-10-20T19:41:50.527491] [DEBUG] [LOCAL ] LLY: Deleted primary data file. [2025-10-20T19:41:50.527553] [INFO ] [MAIN ] LLY: Starting full primary refresh. [2025-10-20T19:41:50.656723] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-10. [2025-10-20T19:41:50.902627] [INFO ] [REMOTE ] LLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:51.242543] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T19:41:51.247686] [INFO ] [REMOTE ] LII: Full fetch complete (41554 records). [2025-10-20T19:41:51.247840] [DEBUG] [VALIDATOR ] LII: Validating primary data. [2025-10-20T19:41:51.454500] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-07. [2025-10-20T19:41:51.694794] [DEBUG] [VALIDATOR ] LII: All expected trading days are present. [2025-10-20T19:41:51.694882] [DEBUG] [VALIDATOR ] LII: Primary data validation passed. [2025-10-20T19:41:51.695751] [INFO ] [MAIN ] LII: Fetched primary data is valid. [2025-10-20T19:41:51.695817] [DEBUG] [LOCAL ] LII: Saving primary data. [2025-10-20T19:41:51.697013] [DEBUG] [LOCAL ] LII: Trimmed 41554 records to 27796 (keeping 200 days of data) [2025-10-20T19:41:51.893821] [DEBUG] [LOCAL ] LII: Primary data saved (27796 records). [2025-10-20T19:41:51.893967] [INFO ] [MAIN ] LII: Full history saved. [2025-10-20T19:41:51.894056] [INFO ] [MAIN ] Update Progress: 1255/2367 symbols completed. LII took 31.1s. Estimated remaining: 2h 23m 21s. Rate: 7.8 symbols/min [2025-10-20T19:41:52.069310] [INFO ] [MAIN ] LMND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:52.117678] [DEBUG] [LOCAL ] LMND: Loaded 52841 existing records from primary storage. [2025-10-20T19:41:52.117994] [INFO ] [MAIN ] LMND: Loaded 52841 existing records. [2025-10-20T19:41:52.258365] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-01. [2025-10-20T19:41:52.301951] [WARN ] [MAIN ] LMND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:52.302497] [INFO ] [MAIN ] LMND: Wiping existing primary data before full refresh. [2025-10-20T19:41:52.302854] [INFO ] [LOCAL ] LMND: Starting deletion of primary data. [2025-10-20T19:41:52.303211] [DEBUG] [LOCAL ] LMND: Deleted primary data file. [2025-10-20T19:41:52.303274] [INFO ] [MAIN ] LMND: Starting full primary refresh. [2025-10-20T19:41:52.612943] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T19:41:52.641794] [INFO ] [REMOTE ] LHX: Full fetch complete (70784 records). [2025-10-20T19:41:52.652245] [DEBUG] [VALIDATOR ] LHX: Validating primary data. [2025-10-20T19:41:52.817092] [INFO ] [REMOTE ] LMND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:53.060113] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-01. [2025-10-20T19:41:53.118445] [DEBUG] [VALIDATOR ] LHX: All expected trading days are present. [2025-10-20T19:41:53.118553] [DEBUG] [VALIDATOR ] LHX: Primary data validation passed. [2025-10-20T19:41:53.119562] [INFO ] [MAIN ] LHX: Fetched primary data is valid. [2025-10-20T19:41:53.119735] [DEBUG] [LOCAL ] LHX: Saving primary data. [2025-10-20T19:41:53.121543] [DEBUG] [LOCAL ] LHX: Trimmed 70784 records to 48528 (keeping 200 days of data) [2025-10-20T19:41:53.249754] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T19:41:53.475815] [DEBUG] [LOCAL ] LHX: Primary data saved (48528 records). [2025-10-20T19:41:53.475956] [INFO ] [MAIN ] LHX: Full history saved. [2025-10-20T19:41:53.476037] [INFO ] [MAIN ] Update Progress: 1256/2367 symbols completed. LHX took 45.4s. Estimated remaining: 2h 23m 8s. Rate: 7.8 symbols/min [2025-10-20T19:41:53.664183] [INFO ] [MAIN ] LMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:53.664381] [DEBUG] [LOCAL ] LMT: No existing data file found in primary storage. [2025-10-20T19:41:53.664421] [WARN ] [MAIN ] LMT: No existing data found, performing full refresh instead. [2025-10-20T19:41:53.664534] [INFO ] [MAIN ] LMT: Wiping existing primary data before full refresh. [2025-10-20T19:41:53.664576] [INFO ] [LOCAL ] LMT: Starting deletion of primary data. [2025-10-20T19:41:53.664619] [DEBUG] [LOCAL ] LMT: No primary data file found to delete. [2025-10-20T19:41:53.664683] [INFO ] [MAIN ] LMT: Starting full primary refresh. [2025-10-20T19:41:53.854534] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-08. [2025-10-20T19:41:54.062748] [INFO ] [REMOTE ] LMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:54.651405] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-01. [2025-10-20T19:41:54.753397] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-01 (2.50 seconds). [2025-10-20T19:41:54.930634] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T19:41:55.446032] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:41:55.455773] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-02. [2025-10-20T19:41:56.251346] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-02. [2025-10-20T19:41:56.576470] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T19:41:57.051347] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-02. [2025-10-20T19:41:57.369803] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T19:41:57.851398] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-09. [2025-10-20T19:41:58.651383] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-03. [2025-10-20T19:41:58.788236] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-02 (2.54 seconds). [2025-10-20T19:41:59.059142] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T19:41:59.451346] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-03. [2025-10-20T19:41:59.538761] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T19:42:00.251357] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-10. [2025-10-20T19:42:00.526666] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T19:42:01.051449] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-04. [2025-10-20T19:42:01.851511] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-03. [2025-10-20T19:42:02.192084] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:42:02.217950] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-03 (2.77 seconds). [2025-10-20T19:42:02.215766] [INFO ] [REMOTE ] LION: Full fetch complete (57669 records). [2025-10-20T19:42:02.228636] [DEBUG] [VALIDATOR ] LION: Validating primary data. [2025-10-20T19:42:02.600974] [DEBUG] [VALIDATOR ] LION: All expected trading days are present. [2025-10-20T19:42:02.601061] [DEBUG] [VALIDATOR ] LION: Primary data validation passed. [2025-10-20T19:42:02.601943] [INFO ] [MAIN ] LION: Fetched primary data is valid. [2025-10-20T19:42:02.602621] [DEBUG] [LOCAL ] LION: Saving primary data. [2025-10-20T19:42:02.604315] [DEBUG] [LOCAL ] LION: Trimmed 57669 records to 46794 (keeping 200 days of data) [2025-10-20T19:42:02.656476] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-04. [2025-10-20T19:42:02.829053] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T19:42:02.911464] [DEBUG] [LOCAL ] LION: Primary data saved (46794 records). [2025-10-20T19:42:02.911630] [INFO ] [MAIN ] LION: Full history saved. [2025-10-20T19:42:02.912541] [INFO ] [MAIN ] Update Progress: 1257/2367 symbols completed. LION took 33.7s. Estimated remaining: 2h 23m 2s. Rate: 7.8 symbols/min [2025-10-20T19:42:03.085967] [INFO ] [MAIN ] LNC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:03.139365] [DEBUG] [LOCAL ] LNC: Loaded 52301 existing records from primary storage. [2025-10-20T19:42:03.139714] [INFO ] [MAIN ] LNC: Loaded 52301 existing records. [2025-10-20T19:42:03.329027] [WARN ] [MAIN ] LNC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:03.329280] [INFO ] [MAIN ] LNC: Wiping existing primary data before full refresh. [2025-10-20T19:42:03.329341] [INFO ] [LOCAL ] LNC: Starting deletion of primary data. [2025-10-20T19:42:03.329572] [DEBUG] [LOCAL ] LNC: Deleted primary data file. [2025-10-20T19:42:03.329630] [INFO ] [MAIN ] LNC: Starting full primary refresh. [2025-10-20T19:42:03.456601] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-05. [2025-10-20T19:42:03.687428] [INFO ] [REMOTE ] LNC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:04.063927] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-03 (2.21 seconds). [2025-10-20T19:42:04.251428] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-01. [2025-10-20T19:42:05.051356] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-04. [2025-10-20T19:42:05.209628] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-04 (2.55 seconds). [2025-10-20T19:42:05.358327] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:42:05.825672] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T19:42:05.851369] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-02. [2025-10-20T19:42:06.651392] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-05. [2025-10-20T19:42:07.173460] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T19:42:07.451364] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-06. [2025-10-20T19:42:07.494367] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:42:08.251397] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-05. [2025-10-20T19:42:09.051365] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-03. [2025-10-20T19:42:09.250196] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-05 (2.60 seconds). [2025-10-20T19:42:09.544213] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-06 (2.09 seconds). [2025-10-20T19:42:09.851376] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-07. [2025-10-20T19:42:10.037794] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:42:10.653010] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:42:10.651970] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-06. [2025-10-20T19:42:11.451377] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-04. [2025-10-20T19:42:12.009664] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T19:42:12.251412] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-06. [2025-10-20T19:42:13.056551] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-08. [2025-10-20T19:42:13.069895] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-06 (2.42 seconds). [2025-10-20T19:42:13.195932] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T19:42:13.851369] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-05. [2025-10-20T19:42:14.651372] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-07. [2025-10-20T19:42:14.694683] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-06 (2.44 seconds). [2025-10-20T19:42:15.252522] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-10-20T19:42:15.300830] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:42:15.451370] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-07. [2025-10-20T19:42:16.251394] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-06. [2025-10-20T19:42:16.678781] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T19:42:17.051417] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-08. [2025-10-20T19:42:17.850581] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:42:17.854472] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-09. [2025-10-20T19:42:18.067936] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-07 (2.62 seconds). [2025-10-20T19:42:18.651435] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-08. [2025-10-20T19:42:19.261733] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T19:42:19.451459] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-07. [2025-10-20T19:42:19.965017] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-10-20T19:42:20.251391] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-10. [2025-10-20T19:42:21.051422] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-09. [2025-10-20T19:42:21.247884] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T19:42:21.325453] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-08 (2.67 seconds). [2025-10-20T19:42:21.851423] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-08. [2025-10-20T19:42:22.502087] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-10 (2.25 seconds). [2025-10-20T19:42:22.509848] [INFO ] [REMOTE ] LMND: Full fetch complete (75472 records). [2025-10-20T19:42:22.510006] [DEBUG] [VALIDATOR ] LMND: Validating primary data. [2025-10-20T19:42:22.657688] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-09. [2025-10-20T19:42:22.966082] [DEBUG] [VALIDATOR ] LMND: All expected trading days are present. [2025-10-20T19:42:22.966176] [DEBUG] [VALIDATOR ] LMND: Primary data validation passed. [2025-10-20T19:42:22.967377] [INFO ] [MAIN ] LMND: Fetched primary data is valid. [2025-10-20T19:42:22.967449] [DEBUG] [LOCAL ] LMND: Saving primary data. [2025-10-20T19:42:22.969210] [DEBUG] [LOCAL ] LMND: Trimmed 75472 records to 52135 (keeping 200 days of data) [2025-10-20T19:42:23.314990] [DEBUG] [LOCAL ] LMND: Primary data saved (52135 records). [2025-10-20T19:42:23.315517] [INFO ] [MAIN ] LMND: Full history saved. [2025-10-20T19:42:23.315878] [INFO ] [MAIN ] Update Progress: 1258/2367 symbols completed. LMND took 31.4s. Estimated remaining: 2h 23m 5s. Rate: 7.8 symbols/min [2025-10-20T19:42:23.560925] [INFO ] [MAIN ] LND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:23.568160] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-09 (2.52 seconds). [2025-10-20T19:42:23.592078] [DEBUG] [LOCAL ] LND: Loaded 5027 existing records from primary storage. [2025-10-20T19:42:23.592384] [INFO ] [MAIN ] LND: Loaded 5027 existing records. [2025-10-20T19:42:23.593013] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-10. [2025-10-20T19:42:23.900824] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T19:42:23.933362] [WARN ] [MAIN ] LND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:23.948133] [INFO ] [MAIN ] LND: Wiping existing primary data before full refresh. [2025-10-20T19:42:23.948249] [INFO ] [LOCAL ] LND: Starting deletion of primary data. [2025-10-20T19:42:23.948931] [DEBUG] [LOCAL ] LND: Deleted primary data file. [2025-10-20T19:42:23.949000] [INFO ] [MAIN ] LND: Starting full primary refresh. [2025-10-20T19:42:24.262952] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-09. [2025-10-20T19:42:24.456163] [INFO ] [REMOTE ] LND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:25.051403] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-01. [2025-10-20T19:42:25.375968] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-09 (2.72 seconds). [2025-10-20T19:42:25.611454] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:42:25.851421] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-02. [2025-10-20T19:42:26.232636] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:42:26.331236] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T19:42:26.355635] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-10 (2.76 seconds). [2025-10-20T19:42:26.362927] [INFO ] [REMOTE ] LMT: Full fetch complete (72494 records). [2025-10-20T19:42:26.363047] [DEBUG] [VALIDATOR ] LMT: Validating primary data. [2025-10-20T19:42:26.659122] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-10. [2025-10-20T19:42:26.757860] [DEBUG] [VALIDATOR ] LMT: All expected trading days are present. [2025-10-20T19:42:26.757942] [DEBUG] [VALIDATOR ] LMT: Primary data validation passed. [2025-10-20T19:42:26.758911] [INFO ] [MAIN ] LMT: Fetched primary data is valid. [2025-10-20T19:42:26.759017] [DEBUG] [LOCAL ] LMT: Saving primary data. [2025-10-20T19:42:26.760380] [DEBUG] [LOCAL ] LMT: Trimmed 72494 records to 49622 (keeping 200 days of data) [2025-10-20T19:42:27.057217] [DEBUG] [LOCAL ] LMT: Primary data saved (49622 records). [2025-10-20T19:42:27.057347] [INFO ] [MAIN ] LMT: Full history saved. [2025-10-20T19:42:27.057414] [INFO ] [MAIN ] Update Progress: 1259/2367 symbols completed. LMT took 33.6s. Estimated remaining: 2h 22m 54s. Rate: 7.8 symbols/min [2025-10-20T19:42:27.227403] [INFO ] [MAIN ] LNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:27.274639] [DEBUG] [LOCAL ] LNG: Loaded 51953 existing records from primary storage. [2025-10-20T19:42:27.274997] [INFO ] [MAIN ] LNG: Loaded 51953 existing records. [2025-10-20T19:42:27.455314] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-10. [2025-10-20T19:42:27.455207] [WARN ] [MAIN ] LNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:27.456625] [INFO ] [MAIN ] LNG: Wiping existing primary data before full refresh. [2025-10-20T19:42:27.456901] [INFO ] [LOCAL ] LNG: Starting deletion of primary data. [2025-10-20T19:42:27.457213] [DEBUG] [LOCAL ] LNG: Deleted primary data file. [2025-10-20T19:42:27.457270] [INFO ] [MAIN ] LNG: Starting full primary refresh. [2025-10-20T19:42:27.832460] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T19:42:27.837819] [INFO ] [REMOTE ] LNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:27.845684] [INFO ] [REMOTE ] LLY: Full fetch complete (77237 records). [2025-10-20T19:42:27.845808] [DEBUG] [VALIDATOR ] LLY: Validating primary data. [2025-10-20T19:42:28.236332] [DEBUG] [VALIDATOR ] LLY: All expected trading days are present. [2025-10-20T19:42:28.236419] [DEBUG] [VALIDATOR ] LLY: Primary data validation passed. [2025-10-20T19:42:28.237396] [INFO ] [MAIN ] LLY: Fetched primary data is valid. [2025-10-20T19:42:28.237459] [DEBUG] [LOCAL ] LLY: Saving primary data. [2025-10-20T19:42:28.238932] [DEBUG] [LOCAL ] LLY: Trimmed 77237 records to 53228 (keeping 200 days of data) [2025-10-20T19:42:28.295347] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-01. [2025-10-20T19:42:28.574877] [DEBUG] [LOCAL ] LLY: Primary data saved (53228 records). [2025-10-20T19:42:28.575020] [INFO ] [MAIN ] LLY: Full history saved. [2025-10-20T19:42:28.575564] [INFO ] [MAIN ] Update Progress: 1260/2367 symbols completed. LLY took 38.5s. Estimated remaining: 2h 22m 41s. Rate: 7.8 symbols/min [2025-10-20T19:42:28.746596] [INFO ] [MAIN ] LNN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:28.760075] [DEBUG] [LOCAL ] LNN: Loaded 11624 existing records from primary storage. [2025-10-20T19:42:28.760295] [INFO ] [MAIN ] LNN: Loaded 11624 existing records. [2025-10-20T19:42:28.944308] [WARN ] [MAIN ] LNN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:28.944579] [INFO ] [MAIN ] LNN: Wiping existing primary data before full refresh. [2025-10-20T19:42:28.944818] [INFO ] [LOCAL ] LNN: Starting deletion of primary data. [2025-10-20T19:42:28.945246] [DEBUG] [LOCAL ] LNN: Deleted primary data file. [2025-10-20T19:42:28.945315] [INFO ] [MAIN ] LNN: Starting full primary refresh. [2025-10-20T19:42:29.051674] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-03. [2025-10-20T19:42:29.254815] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T19:42:29.295177] [INFO ] [REMOTE ] LNC: Full fetch complete (74559 records). [2025-10-20T19:42:29.342742] [DEBUG] [VALIDATOR ] LNC: Validating primary data. [2025-10-20T19:42:29.363991] [INFO ] [REMOTE ] LNN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:29.677617] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-10-20T19:42:29.774512] [DEBUG] [VALIDATOR ] LNC: All expected trading days are present. [2025-10-20T19:42:29.774601] [DEBUG] [VALIDATOR ] LNC: Primary data validation passed. [2025-10-20T19:42:29.775551] [INFO ] [MAIN ] LNC: Fetched primary data is valid. [2025-10-20T19:42:29.775616] [DEBUG] [LOCAL ] LNC: Saving primary data. [2025-10-20T19:42:29.777081] [DEBUG] [LOCAL ] LNC: Trimmed 74559 records to 51511 (keeping 200 days of data) [2025-10-20T19:42:29.864197] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-04. [2025-10-20T19:42:30.088311] [DEBUG] [LOCAL ] LNC: Primary data saved (51511 records). [2025-10-20T19:42:30.088520] [INFO ] [MAIN ] LNC: Full history saved. [2025-10-20T19:42:30.089303] [INFO ] [MAIN ] Update Progress: 1261/2367 symbols completed. LNC took 27.2s. Estimated remaining: 2h 22m 28s. Rate: 7.8 symbols/min [2025-10-20T19:42:30.259864] [INFO ] [MAIN ] LOAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:30.291566] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T19:42:30.334290] [DEBUG] [LOCAL ] LOAR: Loaded 42593 existing records from primary storage. [2025-10-20T19:42:30.334587] [INFO ] [MAIN ] LOAR: Loaded 42593 existing records. [2025-10-20T19:42:30.514113] [WARN ] [MAIN ] LOAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:30.514309] [INFO ] [MAIN ] LOAR: Wiping existing primary data before full refresh. [2025-10-20T19:42:30.514354] [INFO ] [LOCAL ] LOAR: Starting deletion of primary data. [2025-10-20T19:42:30.514769] [DEBUG] [LOCAL ] LOAR: Deleted primary data file. [2025-10-20T19:42:30.514849] [INFO ] [MAIN ] LOAR: Starting full primary refresh. [2025-10-20T19:42:30.620447] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T19:42:30.661442] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-02. [2025-10-20T19:42:30.875436] [INFO ] [REMOTE ] LOAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:31.451349] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-01. [2025-10-20T19:42:32.251358] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-05. [2025-10-20T19:42:32.279954] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T19:42:32.663239] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-10-20T19:42:32.857715] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T19:42:33.051430] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-02. [2025-10-20T19:42:33.812121] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T19:42:33.851450] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-03. [2025-10-20T19:42:34.651389] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-06. [2025-10-20T19:42:34.700412] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T19:42:35.268272] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T19:42:35.451371] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-04. [2025-10-20T19:42:36.251391] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-07. [2025-10-20T19:42:36.389051] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T19:42:36.843701] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T19:42:37.051384] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-08. [2025-10-20T19:42:37.625472] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-10-20T19:42:37.851363] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-09. [2025-10-20T19:42:38.489269] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T19:42:38.651378] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-01. [2025-10-20T19:42:39.451366] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-10. [2025-10-20T19:42:39.867747] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T19:42:40.251346] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-03. [2025-10-20T19:42:40.393131] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:42:40.395262] [INFO ] [REMOTE ] LND: Full fetch complete (7834 records). [2025-10-20T19:42:40.395384] [DEBUG] [VALIDATOR ] LND: Validating primary data. [2025-10-20T19:42:40.773713] [DEBUG] [VALIDATOR ] LND: All expected trading days are present. [2025-10-20T19:42:40.773796] [DEBUG] [VALIDATOR ] LND: Primary data validation passed. [2025-10-20T19:42:40.774021] [INFO ] [MAIN ] LND: Fetched primary data is valid. [2025-10-20T19:42:40.774060] [DEBUG] [LOCAL ] LND: Saving primary data. [2025-10-20T19:42:40.774447] [DEBUG] [LOCAL ] LND: Trimmed 7834 records to 4966 (keeping 200 days of data) [2025-10-20T19:42:40.800869] [DEBUG] [LOCAL ] LND: Primary data saved (4966 records). [2025-10-20T19:42:40.800945] [INFO ] [MAIN ] LND: Full history saved. [2025-10-20T19:42:40.801016] [INFO ] [MAIN ] Update Progress: 1262/2367 symbols completed. LND took 17.5s. Estimated remaining: 2h 22m 23s. Rate: 7.8 symbols/min [2025-10-20T19:42:40.965750] [INFO ] [MAIN ] LOB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:40.991414] [DEBUG] [LOCAL ] LOB: Loaded 27202 existing records from primary storage. [2025-10-20T19:42:40.991720] [INFO ] [MAIN ] LOB: Loaded 27202 existing records. [2025-10-20T19:42:41.056596] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-05. [2025-10-20T19:42:41.168188] [WARN ] [MAIN ] LOB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:41.168411] [INFO ] [MAIN ] LOB: Wiping existing primary data before full refresh. [2025-10-20T19:42:41.168458] [INFO ] [LOCAL ] LOB: Starting deletion of primary data. [2025-10-20T19:42:41.168812] [DEBUG] [LOCAL ] LOB: Deleted primary data file. [2025-10-20T19:42:41.168868] [INFO ] [MAIN ] LOB: Starting full primary refresh. [2025-10-20T19:42:41.516355] [INFO ] [REMOTE ] LOB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:41.851469] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-01. [2025-10-20T19:42:41.854532] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T19:42:42.189801] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T19:42:42.514854] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-10-20T19:42:42.651387] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-06. [2025-10-20T19:42:43.459034] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-02. [2025-10-20T19:42:43.460058] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T19:42:44.251394] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-04. [2025-10-20T19:42:44.729415] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:42:45.051501] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-07. [2025-10-20T19:42:45.851346] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-02. [2025-10-20T19:42:46.040737] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T19:42:46.239540] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T19:42:46.651401] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-05. [2025-10-20T19:42:46.962237] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:42:47.451370] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-03. [2025-10-20T19:42:48.251363] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-03. [2025-10-20T19:42:48.506711] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T19:42:48.787094] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T19:42:49.051355] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-04. [2025-10-20T19:42:49.212724] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T19:42:49.851388] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-04. [2025-10-20T19:42:50.453894] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:42:50.651348] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-08. [2025-10-20T19:42:51.357818] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:42:51.397778] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T19:42:51.451417] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-05. [2025-10-20T19:42:52.251440] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-09. [2025-10-20T19:42:53.051365] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-06. [2025-10-20T19:42:53.093870] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T19:42:53.157815] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T19:42:53.851369] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-06. [2025-10-20T19:42:54.651350] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-10. [2025-10-20T19:42:54.841352] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T19:42:55.387221] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:42:55.451389] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-07. [2025-10-20T19:42:55.693316] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T19:42:55.696615] [INFO ] [REMOTE ] LNN: Full fetch complete (15911 records). [2025-10-20T19:42:55.696787] [DEBUG] [VALIDATOR ] LNN: Validating primary data. [2025-10-20T19:42:56.067614] [DEBUG] [VALIDATOR ] LNN: All expected trading days are present. [2025-10-20T19:42:56.067715] [DEBUG] [VALIDATOR ] LNN: Primary data validation passed. [2025-10-20T19:42:56.068037] [INFO ] [MAIN ] LNN: Fetched primary data is valid. [2025-10-20T19:42:56.068074] [DEBUG] [LOCAL ] LNN: Saving primary data. [2025-10-20T19:42:56.068570] [DEBUG] [LOCAL ] LNN: Trimmed 15911 records to 11478 (keeping 200 days of data) [2025-10-20T19:42:56.140361] [DEBUG] [LOCAL ] LNN: Primary data saved (11478 records). [2025-10-20T19:42:56.140510] [INFO ] [MAIN ] LNN: Full history saved. [2025-10-20T19:42:56.140599] [INFO ] [MAIN ] Update Progress: 1263/2367 symbols completed. LNN took 27.6s. Estimated remaining: 2h 22m 22s. Rate: 7.8 symbols/min [2025-10-20T19:42:56.256606] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-05. [2025-10-20T19:42:56.309782] [INFO ] [MAIN ] LOCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:56.314793] [DEBUG] [LOCAL ] LOCL: Loaded 3843 existing records from primary storage. [2025-10-20T19:42:56.314947] [INFO ] [MAIN ] LOCL: Loaded 3843 existing records. [2025-10-20T19:42:56.494095] [WARN ] [MAIN ] LOCL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:56.494332] [INFO ] [MAIN ] LOCL: Wiping existing primary data before full refresh. [2025-10-20T19:42:56.494400] [INFO ] [LOCAL ] LOCL: Starting deletion of primary data. [2025-10-20T19:42:56.494540] [DEBUG] [LOCAL ] LOCL: Deleted primary data file. [2025-10-20T19:42:56.494595] [INFO ] [MAIN ] LOCL: Starting full primary refresh. [2025-10-20T19:42:56.855430] [INFO ] [REMOTE ] LOCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:57.005086] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:42:57.051352] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-08. [2025-10-20T19:42:57.363313] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:42:57.851422] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-01. [2025-10-20T19:42:58.583773] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:42:58.618960] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:42:58.651340] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-09. [2025-10-20T19:42:59.451355] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-02. [2025-10-20T19:42:59.790255] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-10-20T19:43:00.129693] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T19:43:00.251335] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-07. [2025-10-20T19:43:01.051374] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-03. [2025-10-20T19:43:01.680464] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-10-20T19:43:01.851387] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-04. [2025-10-20T19:43:02.294828] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T19:43:02.642075] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:43:02.651324] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-05. [2025-10-20T19:43:03.093469] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T19:43:03.451450] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-08. [2025-10-20T19:43:04.251417] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-06. [2025-10-20T19:43:04.717723] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T19:43:05.051371] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-10. [2025-10-20T19:43:05.511613] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T19:43:05.851415] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-09. [2025-10-20T19:43:06.447765] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:43:06.453444] [INFO ] [REMOTE ] LOAR: Full fetch complete (55973 records). [2025-10-20T19:43:06.453565] [DEBUG] [VALIDATOR ] LOAR: Validating primary data. [2025-10-20T19:43:06.656746] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-07. [2025-10-20T19:43:06.839086] [DEBUG] [VALIDATOR ] LOAR: All expected trading days are present. [2025-10-20T19:43:06.839182] [DEBUG] [VALIDATOR ] LOAR: Primary data validation passed. [2025-10-20T19:43:06.840155] [INFO ] [MAIN ] LOAR: Fetched primary data is valid. [2025-10-20T19:43:06.840223] [DEBUG] [LOCAL ] LOAR: Saving primary data. [2025-10-20T19:43:06.841559] [DEBUG] [LOCAL ] LOAR: Trimmed 55973 records to 42010 (keeping 200 days of data) [2025-10-20T19:43:07.081835] [DEBUG] [LOCAL ] LOAR: Primary data saved (42010 records). [2025-10-20T19:43:07.081966] [INFO ] [MAIN ] LOAR: Full history saved. [2025-10-20T19:43:07.082045] [INFO ] [MAIN ] Update Progress: 1264/2367 symbols completed. LOAR took 37.0s. Estimated remaining: 2h 22m 17s. Rate: 7.8 symbols/min [2025-10-20T19:43:07.250317] [INFO ] [MAIN ] LOMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:07.283285] [DEBUG] [LOCAL ] LOMA: Loaded 33150 existing records from primary storage. [2025-10-20T19:43:07.283546] [INFO ] [MAIN ] LOMA: Loaded 33150 existing records. [2025-10-20T19:43:07.351946] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T19:43:07.464818] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-06. [2025-10-20T19:43:07.471017] [WARN ] [MAIN ] LOMA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:07.471224] [INFO ] [MAIN ] LOMA: Wiping existing primary data before full refresh. [2025-10-20T19:43:07.472009] [INFO ] [LOCAL ] LOMA: Starting deletion of primary data. [2025-10-20T19:43:07.472228] [DEBUG] [LOCAL ] LOMA: Deleted primary data file. [2025-10-20T19:43:07.472287] [INFO ] [MAIN ] LOMA: Starting full primary refresh. [2025-10-20T19:43:07.835424] [INFO ] [REMOTE ] LOMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:07.864363] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T19:43:08.251379] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-08. [2025-10-20T19:43:08.619405] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T19:43:08.726901] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T19:43:09.051409] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-09. [2025-10-20T19:43:09.653517] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T19:43:09.851451] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-10. [2025-10-20T19:43:10.651397] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-01. [2025-10-20T19:43:10.709870] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:43:10.710946] [INFO ] [REMOTE ] LOCL: Full fetch complete (4539 records). [2025-10-20T19:43:10.711013] [DEBUG] [VALIDATOR ] LOCL: Validating primary data. [2025-10-20T19:43:11.065694] [DEBUG] [VALIDATOR ] LOCL: All expected trading days are present. [2025-10-20T19:43:11.065778] [DEBUG] [VALIDATOR ] LOCL: Primary data validation passed. [2025-10-20T19:43:11.065915] [INFO ] [MAIN ] LOCL: Fetched primary data is valid. [2025-10-20T19:43:11.065952] [DEBUG] [LOCAL ] LOCL: Saving primary data. [2025-10-20T19:43:11.066311] [DEBUG] [LOCAL ] LOCL: Trimmed 4539 records to 3814 (keeping 200 days of data) [2025-10-20T19:43:11.087229] [DEBUG] [LOCAL ] LOCL: Primary data saved (3814 records). [2025-10-20T19:43:11.087359] [INFO ] [MAIN ] LOCL: Full history saved. [2025-10-20T19:43:11.087436] [INFO ] [MAIN ] Update Progress: 1265/2367 symbols completed. LOCL took 14.9s. Estimated remaining: 2h 22m 6s. Rate: 7.8 symbols/min [2025-10-20T19:43:11.258690] [INFO ] [MAIN ] LOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:11.309571] [DEBUG] [LOCAL ] LOW: Loaded 53762 existing records from primary storage. [2025-10-20T19:43:11.309942] [INFO ] [MAIN ] LOW: Loaded 53762 existing records. [2025-10-20T19:43:11.465219] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-07. [2025-10-20T19:43:11.505758] [WARN ] [MAIN ] LOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:11.505914] [INFO ] [MAIN ] LOW: Wiping existing primary data before full refresh. [2025-10-20T19:43:11.505960] [INFO ] [LOCAL ] LOW: Starting deletion of primary data. [2025-10-20T19:43:11.506132] [DEBUG] [LOCAL ] LOW: Deleted primary data file. [2025-10-20T19:43:11.506168] [INFO ] [MAIN ] LOW: Starting full primary refresh. [2025-10-20T19:43:11.854908] [INFO ] [REMOTE ] LOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:12.015921] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T19:43:12.251396] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-02. [2025-10-20T19:43:12.447962] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:43:13.051349] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-10. [2025-10-20T19:43:13.489528] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T19:43:13.564862] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T19:43:13.572113] [INFO ] [REMOTE ] LNG: Full fetch complete (74771 records). [2025-10-20T19:43:13.572252] [DEBUG] [VALIDATOR ] LNG: Validating primary data. [2025-10-20T19:43:13.856579] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-03. [2025-10-20T19:43:13.964706] [DEBUG] [VALIDATOR ] LNG: All expected trading days are present. [2025-10-20T19:43:13.964808] [DEBUG] [VALIDATOR ] LNG: Primary data validation passed. [2025-10-20T19:43:13.965783] [INFO ] [MAIN ] LNG: Fetched primary data is valid. [2025-10-20T19:43:13.965849] [DEBUG] [LOCAL ] LNG: Saving primary data. [2025-10-20T19:43:13.967297] [DEBUG] [LOCAL ] LNG: Trimmed 74771 records to 51168 (keeping 200 days of data) [2025-10-20T19:43:14.271742] [DEBUG] [LOCAL ] LNG: Primary data saved (51168 records). [2025-10-20T19:43:14.271873] [INFO ] [MAIN ] LNG: Full history saved. [2025-10-20T19:43:14.271947] [INFO ] [MAIN ] Update Progress: 1266/2367 symbols completed. LNG took 47.2s. Estimated remaining: 2h 21m 54s. Rate: 7.8 symbols/min [2025-10-20T19:43:14.436783] [INFO ] [MAIN ] LPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:14.476617] [DEBUG] [LOCAL ] LPG: Loaded 40105 existing records from primary storage. [2025-10-20T19:43:14.476915] [INFO ] [MAIN ] LPG: Loaded 40105 existing records. [2025-10-20T19:43:14.649879] [WARN ] [MAIN ] LPG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:14.650088] [INFO ] [MAIN ] LPG: Wiping existing primary data before full refresh. [2025-10-20T19:43:14.650133] [INFO ] [LOCAL ] LPG: Starting deletion of primary data. [2025-10-20T19:43:14.650489] [DEBUG] [LOCAL ] LPG: Deleted primary data file. [2025-10-20T19:43:14.650543] [INFO ] [MAIN ] LPG: Starting full primary refresh. [2025-10-20T19:43:14.661478] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-01. [2025-10-20T19:43:15.000427] [INFO ] [REMOTE ] LPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:15.217513] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:43:15.451373] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-01. [2025-10-20T19:43:16.251402] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-08. [2025-10-20T19:43:16.672607] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T19:43:17.051402] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-02. [2025-10-20T19:43:17.179676] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T19:43:17.384084] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T19:43:17.851360] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-04. [2025-10-20T19:43:18.651430] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-09. [2025-10-20T19:43:18.708756] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:43:19.122152] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T19:43:19.451397] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-05. [2025-10-20T19:43:19.491045] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T19:43:20.251379] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-10. [2025-10-20T19:43:20.695724] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:43:21.051413] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-03. [2025-10-20T19:43:21.438780] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T19:43:21.442895] [INFO ] [REMOTE ] LOB: Full fetch complete (38479 records). [2025-10-20T19:43:21.443017] [DEBUG] [VALIDATOR ] LOB: Validating primary data. [2025-10-20T19:43:21.809252] [DEBUG] [VALIDATOR ] LOB: All expected trading days are present. [2025-10-20T19:43:21.809328] [DEBUG] [VALIDATOR ] LOB: Primary data validation passed. [2025-10-20T19:43:21.809982] [INFO ] [MAIN ] LOB: Fetched primary data is valid. [2025-10-20T19:43:21.810042] [DEBUG] [LOCAL ] LOB: Saving primary data. [2025-10-20T19:43:21.810879] [DEBUG] [LOCAL ] LOB: Trimmed 38479 records to 26844 (keeping 200 days of data) [2025-10-20T19:43:21.862279] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-06. [2025-10-20T19:43:21.974266] [DEBUG] [LOCAL ] LOB: Primary data saved (26844 records). [2025-10-20T19:43:21.974400] [INFO ] [MAIN ] LOB: Full history saved. [2025-10-20T19:43:21.974479] [INFO ] [MAIN ] Update Progress: 1267/2367 symbols completed. LOB took 41.2s. Estimated remaining: 2h 21m 46s. Rate: 7.8 symbols/min [2025-10-20T19:43:22.138184] [INFO ] [MAIN ] LPL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:22.165075] [DEBUG] [LOCAL ] LPL: Loaded 28863 existing records from primary storage. [2025-10-20T19:43:22.165324] [INFO ] [MAIN ] LPL: Loaded 28863 existing records. [2025-10-20T19:43:22.337292] [WARN ] [MAIN ] LPL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:22.337497] [INFO ] [MAIN ] LPL: Wiping existing primary data before full refresh. [2025-10-20T19:43:22.337543] [INFO ] [LOCAL ] LPL: Starting deletion of primary data. [2025-10-20T19:43:22.337763] [DEBUG] [LOCAL ] LPL: Deleted primary data file. [2025-10-20T19:43:22.337819] [INFO ] [MAIN ] LPL: Starting full primary refresh. [2025-10-20T19:43:22.660996] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-02. [2025-10-20T19:43:22.692711] [INFO ] [REMOTE ] LPL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:22.956945] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:43:23.140619] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T19:43:23.451417] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-07. [2025-10-20T19:43:24.200044] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T19:43:24.251424] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-01. [2025-10-20T19:43:24.760480] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:43:25.051406] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-03. [2025-10-20T19:43:25.272722] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T19:43:25.851390] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-02. [2025-10-20T19:43:26.594951] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:43:26.651365] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-04. [2025-10-20T19:43:26.815928] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T19:43:27.451369] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-03. [2025-10-20T19:43:28.215231] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T19:43:28.251422] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-05. [2025-10-20T19:43:28.514235] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T19:43:29.051394] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-08. [2025-10-20T19:43:29.594985] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T19:43:29.851412] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-06. [2025-10-20T19:43:30.333145] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T19:43:30.651378] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-04. [2025-10-20T19:43:31.416159] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:43:31.451351] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-04. [2025-10-20T19:43:31.760169] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:43:32.251373] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-07. [2025-10-20T19:43:33.051387] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-05. [2025-10-20T19:43:33.335787] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T19:43:33.710382] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:43:33.851389] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-05. [2025-10-20T19:43:34.128088] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:43:34.651365] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-06. [2025-10-20T19:43:35.451357] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-08. [2025-10-20T19:43:35.661747] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T19:43:35.783683] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T19:43:36.251429] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-07. [2025-10-20T19:43:36.873026] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:43:37.051370] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-09. [2025-10-20T19:43:37.419972] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T19:43:37.851391] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-09. [2025-10-20T19:43:38.541548] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:43:38.651364] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-08. [2025-10-20T19:43:39.318359] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:43:39.451380] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-10. [2025-10-20T19:43:40.071031] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:43:40.251367] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-09. [2025-10-20T19:43:40.879585] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T19:43:40.884981] [INFO ] [REMOTE ] LPG: Full fetch complete (59041 records). [2025-10-20T19:43:40.885114] [DEBUG] [VALIDATOR ] LPG: Validating primary data. [2025-10-20T19:43:41.062765] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-10. [2025-10-20T19:43:41.284687] [DEBUG] [VALIDATOR ] LPG: All expected trading days are present. [2025-10-20T19:43:41.284794] [DEBUG] [VALIDATOR ] LPG: Primary data validation passed. [2025-10-20T19:43:41.285754] [INFO ] [MAIN ] LPG: Fetched primary data is valid. [2025-10-20T19:43:41.285820] [DEBUG] [LOCAL ] LPG: Saving primary data. [2025-10-20T19:43:41.287070] [DEBUG] [LOCAL ] LPG: Trimmed 59041 records to 39541 (keeping 200 days of data) [2025-10-20T19:43:41.520960] [DEBUG] [LOCAL ] LPG: Primary data saved (39541 records). [2025-10-20T19:43:41.521122] [INFO ] [MAIN ] LPG: Full history saved. [2025-10-20T19:43:41.521201] [INFO ] [MAIN ] Update Progress: 1268/2367 symbols completed. LPG took 27.2s. Estimated remaining: 2h 21m 49s. Rate: 7.7 symbols/min [2025-10-20T19:43:41.688050] [INFO ] [MAIN ] LPX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:41.725152] [DEBUG] [LOCAL ] LPX: Loaded 40820 existing records from primary storage. [2025-10-20T19:43:41.725428] [INFO ] [MAIN ] LPX: Loaded 40820 existing records. [2025-10-20T19:43:41.856379] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-06. [2025-10-20T19:43:41.908761] [WARN ] [MAIN ] LPX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:41.909073] [INFO ] [MAIN ] LPX: Wiping existing primary data before full refresh. [2025-10-20T19:43:41.909140] [INFO ] [LOCAL ] LPX: Starting deletion of primary data. [2025-10-20T19:43:41.909371] [DEBUG] [LOCAL ] LPX: Deleted primary data file. [2025-10-20T19:43:41.909427] [INFO ] [MAIN ] LPX: Starting full primary refresh. [2025-10-20T19:43:41.995785] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T19:43:42.286125] [INFO ] [REMOTE ] LPX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:42.408281] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T19:43:42.413708] [INFO ] [REMOTE ] LOMA: Full fetch complete (47886 records). [2025-10-20T19:43:42.413793] [DEBUG] [VALIDATOR ] LOMA: Validating primary data. [2025-10-20T19:43:42.652478] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-10. [2025-10-20T19:43:42.788986] [DEBUG] [VALIDATOR ] LOMA: All expected trading days are present. [2025-10-20T19:43:42.789314] [DEBUG] [VALIDATOR ] LOMA: Primary data validation passed. [2025-10-20T19:43:42.790084] [INFO ] [MAIN ] LOMA: Fetched primary data is valid. [2025-10-20T19:43:42.790155] [DEBUG] [LOCAL ] LOMA: Saving primary data. [2025-10-20T19:43:42.791265] [DEBUG] [LOCAL ] LOMA: Trimmed 47886 records to 32769 (keeping 200 days of data) [2025-10-20T19:43:42.970740] [DEBUG] [LOCAL ] LOMA: Primary data saved (32769 records). [2025-10-20T19:43:42.970860] [INFO ] [MAIN ] LOMA: Full history saved. [2025-10-20T19:43:42.970934] [INFO ] [MAIN ] Update Progress: 1269/2367 symbols completed. LOMA took 35.9s. Estimated remaining: 2h 21m 35s. Rate: 7.8 symbols/min [2025-10-20T19:43:43.136813] [INFO ] [MAIN ] LRN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:43.177581] [DEBUG] [LOCAL ] LRN: Loaded 40127 existing records from primary storage. [2025-10-20T19:43:43.177875] [INFO ] [MAIN ] LRN: Loaded 40127 existing records. [2025-10-20T19:43:43.355397] [WARN ] [MAIN ] LRN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:43.355725] [INFO ] [MAIN ] LRN: Wiping existing primary data before full refresh. [2025-10-20T19:43:43.355793] [INFO ] [LOCAL ] LRN: Starting deletion of primary data. [2025-10-20T19:43:43.356347] [DEBUG] [LOCAL ] LRN: Deleted primary data file. [2025-10-20T19:43:43.356403] [INFO ] [MAIN ] LRN: Starting full primary refresh. [2025-10-20T19:43:43.453946] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-01. [2025-10-20T19:43:43.761773] [INFO ] [REMOTE ] LRN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:43.766107] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T19:43:43.843762] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T19:43:43.849590] [INFO ] [REMOTE ] LPL: Full fetch complete (37230 records). [2025-10-20T19:43:43.849692] [DEBUG] [VALIDATOR ] LPL: Validating primary data. [2025-10-20T19:43:44.210491] [DEBUG] [VALIDATOR ] LPL: All expected trading days are present. [2025-10-20T19:43:44.210562] [DEBUG] [VALIDATOR ] LPL: Primary data validation passed. [2025-10-20T19:43:44.211151] [INFO ] [MAIN ] LPL: Fetched primary data is valid. [2025-10-20T19:43:44.211210] [DEBUG] [LOCAL ] LPL: Saving primary data. [2025-10-20T19:43:44.211996] [DEBUG] [LOCAL ] LPL: Trimmed 37230 records to 28753 (keeping 200 days of data) [2025-10-20T19:43:44.254914] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-01. [2025-10-20T19:43:44.370864] [DEBUG] [LOCAL ] LPL: Primary data saved (28753 records). [2025-10-20T19:43:44.370986] [INFO ] [MAIN ] LPL: Full history saved. [2025-10-20T19:43:44.371059] [INFO ] [MAIN ] Update Progress: 1270/2367 symbols completed. LPL took 22.4s. Estimated remaining: 2h 21m 22s. Rate: 7.8 symbols/min [2025-10-20T19:43:44.534467] [INFO ] [MAIN ] LSPD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:44.572980] [DEBUG] [LOCAL ] LSPD: Loaded 43208 existing records from primary storage. [2025-10-20T19:43:44.573254] [INFO ] [MAIN ] LSPD: Loaded 43208 existing records. [2025-10-20T19:43:44.747963] [WARN ] [MAIN ] LSPD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:44.748239] [INFO ] [MAIN ] LSPD: Wiping existing primary data before full refresh. [2025-10-20T19:43:44.748305] [INFO ] [LOCAL ] LSPD: Starting deletion of primary data. [2025-10-20T19:43:44.748602] [DEBUG] [LOCAL ] LSPD: Deleted primary data file. [2025-10-20T19:43:44.748683] [INFO ] [MAIN ] LSPD: Starting full primary refresh. [2025-10-20T19:43:45.058617] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-07. [2025-10-20T19:43:45.111459] [INFO ] [REMOTE ] LSPD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:45.137263] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T19:43:45.859672] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-01. [2025-10-20T19:43:45.864326] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T19:43:46.651349] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-02. [2025-10-20T19:43:47.120863] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T19:43:47.159043] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T19:43:47.451403] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-02. [2025-10-20T19:43:48.102307] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T19:43:48.251384] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-02. [2025-10-20T19:43:48.969430] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T19:43:49.051345] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-03. [2025-10-20T19:43:49.700255] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T19:43:49.851365] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-03. [2025-10-20T19:43:50.662133] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-03. [2025-10-20T19:43:50.667785] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:43:51.418239] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T19:43:51.451345] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-04. [2025-10-20T19:43:52.129242] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T19:43:52.251401] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-04. [2025-10-20T19:43:53.043032] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T19:43:53.053856] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-04. [2025-10-20T19:43:53.783176] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T19:43:53.851358] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-05. [2025-10-20T19:43:54.651452] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-08. [2025-10-20T19:43:54.692425] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:43:55.357351] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:43:55.451382] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-06. [2025-10-20T19:43:56.251416] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-05. [2025-10-20T19:43:56.744147] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-10-20T19:43:56.823361] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T19:43:57.051431] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-09. [2025-10-20T19:43:57.800141] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:43:57.851402] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-06. [2025-10-20T19:43:58.651389] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-05. [2025-10-20T19:43:58.912776] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T19:43:59.430279] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:43:59.451346] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-07. [2025-10-20T19:43:59.734190] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:44:00.251393] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-07. [2025-10-20T19:44:00.929144] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T19:44:01.051415] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-08. [2025-10-20T19:44:01.742197] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T19:44:01.851385] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-08. [2025-10-20T19:44:02.627285] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:44:02.651366] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-09. [2025-10-20T19:44:03.428637] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:44:03.451349] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-10. [2025-10-20T19:44:04.251392] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-06. [2025-10-20T19:44:04.312426] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T19:44:04.356914] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T19:44:04.364727] [INFO ] [REMOTE ] LOW: Full fetch complete (76732 records). [2025-10-20T19:44:04.364883] [DEBUG] [VALIDATOR ] LOW: Validating primary data. [2025-10-20T19:44:04.751246] [DEBUG] [VALIDATOR ] LOW: All expected trading days are present. [2025-10-20T19:44:04.751337] [DEBUG] [VALIDATOR ] LOW: Primary data validation passed. [2025-10-20T19:44:04.752328] [INFO ] [MAIN ] LOW: Fetched primary data is valid. [2025-10-20T19:44:04.752395] [DEBUG] [LOCAL ] LOW: Saving primary data. [2025-10-20T19:44:04.754030] [DEBUG] [LOCAL ] LOW: Trimmed 76732 records to 52993 (keeping 200 days of data) [2025-10-20T19:44:05.061951] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-10. [2025-10-20T19:44:05.080742] [DEBUG] [LOCAL ] LOW: Primary data saved (52993 records). [2025-10-20T19:44:05.080864] [INFO ] [MAIN ] LOW: Full history saved. [2025-10-20T19:44:05.081162] [INFO ] [MAIN ] Update Progress: 1271/2367 symbols completed. LOW took 54.0s. Estimated remaining: 2h 21m 26s. Rate: 7.7 symbols/min [2025-10-20T19:44:05.248925] [INFO ] [MAIN ] LTC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:05.280619] [DEBUG] [LOCAL ] LTC: Loaded 31063 existing records from primary storage. [2025-10-20T19:44:05.280976] [INFO ] [MAIN ] LTC: Loaded 31063 existing records. [2025-10-20T19:44:05.456250] [WARN ] [MAIN ] LTC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:05.456450] [INFO ] [MAIN ] LTC: Wiping existing primary data before full refresh. [2025-10-20T19:44:05.456496] [INFO ] [LOCAL ] LTC: Starting deletion of primary data. [2025-10-20T19:44:05.456696] [DEBUG] [LOCAL ] LTC: Deleted primary data file. [2025-10-20T19:44:05.456752] [INFO ] [MAIN ] LTC: Starting full primary refresh. [2025-10-20T19:44:05.814808] [INFO ] [REMOTE ] LTC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:05.851342] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-01. [2025-10-20T19:44:05.980476] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T19:44:06.651346] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-09. [2025-10-20T19:44:06.817043] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T19:44:06.822696] [INFO ] [REMOTE ] LRN: Full fetch complete (58438 records). [2025-10-20T19:44:06.823021] [DEBUG] [VALIDATOR ] LRN: Validating primary data. [2025-10-20T19:44:07.213549] [DEBUG] [VALIDATOR ] LRN: All expected trading days are present. [2025-10-20T19:44:07.213628] [DEBUG] [VALIDATOR ] LRN: Primary data validation passed. [2025-10-20T19:44:07.214552] [INFO ] [MAIN ] LRN: Fetched primary data is valid. [2025-10-20T19:44:07.214616] [DEBUG] [LOCAL ] LRN: Saving primary data. [2025-10-20T19:44:07.215797] [DEBUG] [LOCAL ] LRN: Trimmed 58438 records to 39505 (keeping 200 days of data) [2025-10-20T19:44:07.455425] [DEBUG] [LOCAL ] LRN: Primary data saved (39505 records). [2025-10-20T19:44:07.452580] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-07. [2025-10-20T19:44:07.455617] [INFO ] [MAIN ] LRN: Full history saved. [2025-10-20T19:44:07.456842] [INFO ] [MAIN ] Update Progress: 1272/2367 symbols completed. LRN took 24.5s. Estimated remaining: 2h 21m 13s. Rate: 7.8 symbols/min [2025-10-20T19:44:07.633077] [INFO ] [MAIN ] LTH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:07.650763] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T19:44:07.703116] [DEBUG] [LOCAL ] LTH: Loaded 52497 existing records from primary storage. [2025-10-20T19:44:07.703428] [INFO ] [MAIN ] LTH: Loaded 52497 existing records. [2025-10-20T19:44:07.878518] [WARN ] [MAIN ] LTH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:07.878910] [INFO ] [MAIN ] LTH: Wiping existing primary data before full refresh. [2025-10-20T19:44:07.879058] [INFO ] [LOCAL ] LTH: Starting deletion of primary data. [2025-10-20T19:44:07.879400] [DEBUG] [LOCAL ] LTH: Deleted primary data file. [2025-10-20T19:44:07.879612] [INFO ] [MAIN ] LTH: Starting full primary refresh. [2025-10-20T19:44:08.124268] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:44:08.266472] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-10. [2025-10-20T19:44:08.271110] [INFO ] [REMOTE ] LTH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:08.679769] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T19:44:09.051438] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-08. [2025-10-20T19:44:09.851445] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-01. [2025-10-20T19:44:10.634361] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:44:10.651333] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-09. [2025-10-20T19:44:11.410827] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:44:11.451353] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-02. [2025-10-20T19:44:12.066355] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:44:12.251360] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-02. [2025-10-20T19:44:13.051453] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-10. [2025-10-20T19:44:13.099736] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:44:13.273573] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-10 (5.01 seconds). [2025-10-20T19:44:13.280498] [INFO ] [REMOTE ] LSPD: Full fetch complete (62183 records). [2025-10-20T19:44:13.280734] [DEBUG] [VALIDATOR ] LSPD: Validating primary data. [2025-10-20T19:44:13.724177] [DEBUG] [VALIDATOR ] LSPD: All expected trading days are present. [2025-10-20T19:44:13.720808] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:44:13.724474] [DEBUG] [VALIDATOR ] LSPD: Primary data validation passed. [2025-10-20T19:44:13.734577] [INFO ] [MAIN ] LSPD: Fetched primary data is valid. [2025-10-20T19:44:13.734736] [DEBUG] [LOCAL ] LSPD: Saving primary data. [2025-10-20T19:44:13.737597] [DEBUG] [LOCAL ] LSPD: Trimmed 62183 records to 42440 (keeping 200 days of data) [2025-10-20T19:44:13.851450] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-03. [2025-10-20T19:44:13.981440] [DEBUG] [LOCAL ] LSPD: Primary data saved (42440 records). [2025-10-20T19:44:13.981571] [INFO ] [MAIN ] LSPD: Full history saved. [2025-10-20T19:44:13.981670] [INFO ] [MAIN ] Update Progress: 1273/2367 symbols completed. LSPD took 29.6s. Estimated remaining: 2h 21m 5s. Rate: 7.8 symbols/min [2025-10-20T19:44:14.149903] [INFO ] [MAIN ] LTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:14.186150] [DEBUG] [LOCAL ] LTM: Loaded 39389 existing records from primary storage. [2025-10-20T19:44:14.186462] [INFO ] [MAIN ] LTM: Loaded 39389 existing records. [2025-10-20T19:44:14.368207] [WARN ] [MAIN ] LTM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:14.368437] [INFO ] [MAIN ] LTM: Wiping existing primary data before full refresh. [2025-10-20T19:44:14.368483] [INFO ] [LOCAL ] LTM: Starting deletion of primary data. [2025-10-20T19:44:14.368771] [DEBUG] [LOCAL ] LTM: Deleted primary data file. [2025-10-20T19:44:14.368827] [INFO ] [MAIN ] LTM: Starting full primary refresh. [2025-10-20T19:44:14.662441] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-03. [2025-10-20T19:44:14.733443] [INFO ] [REMOTE ] LTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:14.806085] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T19:44:14.812347] [INFO ] [REMOTE ] LPX: Full fetch complete (58861 records). [2025-10-20T19:44:14.812482] [DEBUG] [VALIDATOR ] LPX: Validating primary data. [2025-10-20T19:44:15.250963] [DEBUG] [VALIDATOR ] LPX: All expected trading days are present. [2025-10-20T19:44:15.251060] [DEBUG] [VALIDATOR ] LPX: Primary data validation passed. [2025-10-20T19:44:15.251990] [INFO ] [MAIN ] LPX: Fetched primary data is valid. [2025-10-20T19:44:15.252056] [DEBUG] [LOCAL ] LPX: Saving primary data. [2025-10-20T19:44:15.253338] [DEBUG] [LOCAL ] LPX: Trimmed 58861 records to 40152 (keeping 200 days of data) [2025-10-20T19:44:15.460740] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-01. [2025-10-20T19:44:15.499608] [DEBUG] [LOCAL ] LPX: Primary data saved (40152 records). [2025-10-20T19:44:15.499775] [INFO ] [MAIN ] LPX: Full history saved. [2025-10-20T19:44:15.499856] [INFO ] [MAIN ] Update Progress: 1274/2367 symbols completed. LPX took 34.0s. Estimated remaining: 2h 20m 52s. Rate: 7.8 symbols/min [2025-10-20T19:44:15.670342] [INFO ] [MAIN ] LU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:15.703985] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T19:44:15.751227] [DEBUG] [LOCAL ] LU: Loaded 49804 existing records from primary storage. [2025-10-20T19:44:15.751555] [INFO ] [MAIN ] LU: Loaded 49804 existing records. [2025-10-20T19:44:15.932578] [WARN ] [MAIN ] LU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:15.932972] [INFO ] [MAIN ] LU: Wiping existing primary data before full refresh. [2025-10-20T19:44:15.933060] [INFO ] [LOCAL ] LU: Starting deletion of primary data. [2025-10-20T19:44:15.933555] [DEBUG] [LOCAL ] LU: Deleted primary data file. [2025-10-20T19:44:15.933616] [INFO ] [MAIN ] LU: Starting full primary refresh. [2025-10-20T19:44:16.271680] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-04. [2025-10-20T19:44:16.321352] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T19:44:16.325577] [INFO ] [REMOTE ] LU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:16.810399] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T19:44:17.051439] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-04. [2025-10-20T19:44:17.851421] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-01. [2025-10-20T19:44:18.075335] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:44:18.453361] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:44:18.651377] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-05. [2025-10-20T19:44:19.451385] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-02. [2025-10-20T19:44:19.522030] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T19:44:20.071437] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T19:44:20.251405] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-02. [2025-10-20T19:44:20.697013] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T19:44:21.051358] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-03. [2025-10-20T19:44:21.851432] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-06. [2025-10-20T19:44:21.884534] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T19:44:21.952404] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T19:44:22.651429] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-03. [2025-10-20T19:44:22.771615] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T19:44:23.451374] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-07. [2025-10-20T19:44:24.086581] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:44:24.251377] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-05. [2025-10-20T19:44:24.911789] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:44:25.051391] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-04. [2025-10-20T19:44:25.851410] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-04. [2025-10-20T19:44:25.921810] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T19:44:26.648119] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T19:44:26.657569] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-06. [2025-10-20T19:44:27.081160] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T19:44:27.451370] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-08. [2025-10-20T19:44:28.251413] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-05. [2025-10-20T19:44:28.295825] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:44:28.871554] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:44:29.051425] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-05. [2025-10-20T19:44:29.851365] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-09. [2025-10-20T19:44:29.978105] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T19:44:30.417256] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:44:30.651383] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-06. [2025-10-20T19:44:31.196909] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:44:31.451401] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-10. [2025-10-20T19:44:31.557503] [DEBUG] [REMOTE ] LTC: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T19:44:31.557856] [WARN ] [REMOTE ] LTC: No data returned for month 2025-10 [2025-10-20T19:44:31.557944] [FATAL] [REMOTE ] LTC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:44:31.558045] [FATAL] [MAIN ] LTC: Failed to fetch primary data. [2025-10-20T19:44:31.558124] [WARN ] [MAIN ] LTC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:44:31.558208] [INFO ] [LOCAL ] LTC: Starting deletion of primary data. [2025-10-20T19:44:31.558290] [DEBUG] [LOCAL ] LTC: No primary data file found to delete. [2025-10-20T19:44:31.558339] [INFO ] [LOCAL ] LTC: Starting deletion of derivative data. [2025-10-20T19:44:31.558518] [DEBUG] [LOCAL ] LTC: Deleted derivative data file. [2025-10-20T19:44:31.558922] [INFO ] [MAIN ] Update Progress: 1275/2367 symbols completed. LTC took 26.5s. Estimated remaining: 2h 20m 51s. Rate: 7.8 symbols/min [2025-10-20T19:44:31.742696] [INFO ] [MAIN ] LUCK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:31.770180] [DEBUG] [LOCAL ] LUCK: Loaded 24521 existing records from primary storage. [2025-10-20T19:44:31.770483] [INFO ] [MAIN ] LUCK: Loaded 24521 existing records. [2025-10-20T19:44:31.960474] [WARN ] [MAIN ] LUCK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:31.960861] [INFO ] [MAIN ] LUCK: Wiping existing primary data before full refresh. [2025-10-20T19:44:31.960936] [INFO ] [LOCAL ] LUCK: Starting deletion of primary data. [2025-10-20T19:44:31.961166] [DEBUG] [LOCAL ] LUCK: Deleted primary data file. [2025-10-20T19:44:31.961232] [INFO ] [MAIN ] LUCK: Starting full primary refresh. [2025-10-20T19:44:32.253885] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-06. [2025-10-20T19:44:32.368231] [INFO ] [REMOTE ] LUCK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:32.396387] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T19:44:33.051376] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-01. [2025-10-20T19:44:33.641581] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T19:44:33.851405] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-07. [2025-10-20T19:44:34.499752] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T19:44:34.651432] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-07. [2025-10-20T19:44:35.451444] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-07. [2025-10-20T19:44:35.625259] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:44:36.092456] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T19:44:36.251408] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-08. [2025-10-20T19:44:37.051356] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-08. [2025-10-20T19:44:37.120571] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T19:44:37.817496] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:44:37.851379] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-09. [2025-10-20T19:44:38.651374] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-08. [2025-10-20T19:44:38.754024] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T19:44:39.442561] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:44:39.452819] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-02. [2025-10-20T19:44:40.157078] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:44:40.251343] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-09. [2025-10-20T19:44:40.415809] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T19:44:41.051375] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-09. [2025-10-20T19:44:41.846243] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:44:41.856357] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-03. [2025-10-20T19:44:42.651338] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-10. [2025-10-20T19:44:42.852485] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:44:43.082436] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T19:44:43.451394] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-10. [2025-10-20T19:44:44.210772] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T19:44:44.217811] [INFO ] [REMOTE ] LTM: Full fetch complete (50815 records). [2025-10-20T19:44:44.217900] [DEBUG] [VALIDATOR ] LTM: Validating primary data. [2025-10-20T19:44:44.260399] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-10. [2025-10-20T19:44:44.598892] [DEBUG] [VALIDATOR ] LTM: All expected trading days are present. [2025-10-20T19:44:44.598991] [DEBUG] [VALIDATOR ] LTM: Primary data validation passed. [2025-10-20T19:44:44.599807] [INFO ] [MAIN ] LTM: Fetched primary data is valid. [2025-10-20T19:44:44.599868] [DEBUG] [LOCAL ] LTM: Saving primary data. [2025-10-20T19:44:44.600978] [DEBUG] [LOCAL ] LTM: Trimmed 50815 records to 39040 (keeping 200 days of data) [2025-10-20T19:44:44.822712] [DEBUG] [LOCAL ] LTM: Primary data saved (39040 records). [2025-10-20T19:44:44.822844] [INFO ] [MAIN ] LTM: Full history saved. [2025-10-20T19:44:44.822921] [INFO ] [MAIN ] Update Progress: 1276/2367 symbols completed. LTM took 30.8s. Estimated remaining: 2h 20m 48s. Rate: 7.7 symbols/min [2025-10-20T19:44:44.991059] [INFO ] [MAIN ] LUMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:45.040153] [DEBUG] [LOCAL ] LUMN: Loaded 54426 existing records from primary storage. [2025-10-20T19:44:45.040451] [INFO ] [MAIN ] LUMN: Loaded 54426 existing records. [2025-10-20T19:44:45.071221] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-04. [2025-10-20T19:44:45.242764] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T19:44:45.246483] [WARN ] [MAIN ] LUMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:45.254010] [INFO ] [MAIN ] LUMN: Wiping existing primary data before full refresh. [2025-10-20T19:44:45.254131] [INFO ] [LOCAL ] LUMN: Starting deletion of primary data. [2025-10-20T19:44:45.254296] [INFO ] [REMOTE ] LTH: Full fetch complete (74768 records). [2025-10-20T19:44:45.254449] [DEBUG] [LOCAL ] LUMN: Deleted primary data file. [2025-10-20T19:44:45.254531] [INFO ] [MAIN ] LUMN: Starting full primary refresh. [2025-10-20T19:44:45.254413] [DEBUG] [VALIDATOR ] LTH: Validating primary data. [2025-10-20T19:44:45.947217] [INFO ] [REMOTE ] LUMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:45.950085] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-01. [2025-10-20T19:44:46.049662] [DEBUG] [VALIDATOR ] LTH: All expected trading days are present. [2025-10-20T19:44:46.049823] [DEBUG] [VALIDATOR ] LTH: Primary data validation passed. [2025-10-20T19:44:46.051133] [INFO ] [MAIN ] LTH: Fetched primary data is valid. [2025-10-20T19:44:46.051213] [DEBUG] [LOCAL ] LTH: Saving primary data. [2025-10-20T19:44:46.052786] [DEBUG] [LOCAL ] LTH: Trimmed 74768 records to 51686 (keeping 200 days of data) [2025-10-20T19:44:46.273094] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T19:44:46.302618] [INFO ] [REMOTE ] LU: Full fetch complete (70916 records). [2025-10-20T19:44:46.314741] [DEBUG] [VALIDATOR ] LU: Validating primary data. [2025-10-20T19:44:46.687909] [DEBUG] [LOCAL ] LTH: Primary data saved (51686 records). [2025-10-20T19:44:46.695731] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:44:46.701580] [INFO ] [MAIN ] LTH: Full history saved. [2025-10-20T19:44:46.712560] [INFO ] [MAIN ] Update Progress: 1277/2367 symbols completed. LTH took 39.3s. Estimated remaining: 2h 20m 35s. Rate: 7.8 symbols/min [2025-10-20T19:44:46.729394] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-05. [2025-10-20T19:44:46.821407] [DEBUG] [VALIDATOR ] LU: All expected trading days are present. [2025-10-20T19:44:46.821482] [DEBUG] [VALIDATOR ] LU: Primary data validation passed. [2025-10-20T19:44:46.822398] [INFO ] [MAIN ] LU: Fetched primary data is valid. [2025-10-20T19:44:46.822459] [DEBUG] [LOCAL ] LU: Saving primary data. [2025-10-20T19:44:46.823777] [DEBUG] [LOCAL ] LU: Trimmed 70916 records to 49163 (keeping 200 days of data) [2025-10-20T19:44:47.088908] [DEBUG] [LOCAL ] LU: Primary data saved (49163 records). [2025-10-20T19:44:47.089047] [INFO ] [MAIN ] LU: Full history saved. [2025-10-20T19:44:47.089126] [INFO ] [MAIN ] Update Progress: 1278/2367 symbols completed. LU took 31.6s. Estimated remaining: 2h 20m 21s. Rate: 7.8 symbols/min [2025-10-20T19:44:47.662559] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T19:44:47.679522] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-02. [2025-10-20T19:44:47.706080] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T19:44:48.251387] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-06. [2025-10-20T19:44:49.456777] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T19:44:49.463402] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-07. [2025-10-20T19:44:49.583582] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T19:44:49.851434] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-03. [2025-10-20T19:44:50.404928] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T19:44:50.651425] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-08. [2025-10-20T19:44:51.553348] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T19:44:51.559526] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-09. [2025-10-20T19:44:51.606818] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T19:44:52.251377] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-04. [2025-10-20T19:44:52.340949] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:44:53.051488] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-10. [2025-10-20T19:44:54.070882] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T19:44:54.074027] [INFO ] [REMOTE ] LUCK: Full fetch complete (39606 records). [2025-10-20T19:44:54.074116] [DEBUG] [VALIDATOR ] LUCK: Validating primary data. [2025-10-20T19:44:54.431306] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T19:44:54.489439] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-05. [2025-10-20T19:44:54.529965] [DEBUG] [VALIDATOR ] LUCK: All expected trading days are present. [2025-10-20T19:44:54.530041] [DEBUG] [VALIDATOR ] LUCK: Primary data validation passed. [2025-10-20T19:44:54.530830] [INFO ] [MAIN ] LUCK: Fetched primary data is valid. [2025-10-20T19:44:54.530892] [DEBUG] [LOCAL ] LUCK: Saving primary data. [2025-10-20T19:44:54.531776] [DEBUG] [LOCAL ] LUCK: Trimmed 39606 records to 24157 (keeping 200 days of data) [2025-10-20T19:44:54.672372] [DEBUG] [LOCAL ] LUCK: Primary data saved (24157 records). [2025-10-20T19:44:54.672493] [INFO ] [MAIN ] LUCK: Full history saved. [2025-10-20T19:44:54.672569] [INFO ] [MAIN ] Update Progress: 1279/2367 symbols completed. LUCK took 23.1s. Estimated remaining: 2h 20m 13s. Rate: 7.8 symbols/min [2025-10-20T19:44:56.469937] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T19:44:56.481153] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-06. [2025-10-20T19:44:58.349674] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T19:44:58.360712] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-07. [2025-10-20T19:45:00.327006] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T19:45:00.338567] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-08. [2025-10-20T19:45:02.452003] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T19:45:02.464015] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-09. [2025-10-20T19:45:04.678701] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T19:45:04.690234] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-10. [2025-10-20T19:45:07.101028] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-10 (2.41 seconds). [2025-10-20T19:45:07.112927] [INFO ] [REMOTE ] LUMN: Full fetch complete (77778 records). [2025-10-20T19:45:07.113112] [DEBUG] [VALIDATOR ] LUMN: Validating primary data. [2025-10-20T19:45:07.507367] [DEBUG] [VALIDATOR ] LUMN: All expected trading days are present. [2025-10-20T19:45:07.507447] [DEBUG] [VALIDATOR ] LUMN: Primary data validation passed. [2025-10-20T19:45:07.508444] [INFO ] [MAIN ] LUMN: Fetched primary data is valid. [2025-10-20T19:45:07.508507] [DEBUG] [LOCAL ] LUMN: Saving primary data. [2025-10-20T19:45:07.509967] [DEBUG] [LOCAL ] LUMN: Trimmed 77778 records to 53645 (keeping 200 days of data) [2025-10-20T19:45:07.803950] [DEBUG] [LOCAL ] LUMN: Primary data saved (53645 records). [2025-10-20T19:45:07.804085] [INFO ] [MAIN ] LUMN: Full history saved. [2025-10-20T19:45:07.804161] [INFO ] [MAIN ] Update Progress: 1280/2367 symbols completed. LUMN took 23.0s. Estimated remaining: 2h 20m 10s. Rate: 7.8 symbols/min [2025-10-20T19:45:08.409601] [INFO ] [MAIN ] LUXE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:08.434347] [INFO ] [MAIN ] LUV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:08.491562] [DEBUG] [LOCAL ] LUXE: No existing data file found in primary storage. [2025-10-20T19:45:08.511423] [INFO ] [MAIN ] LVS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:08.494436] [WARN ] [MAIN ] LUXE: No existing data found, performing full refresh instead. [2025-10-20T19:45:08.516475] [INFO ] [MAIN ] LUXE: Wiping existing primary data before full refresh. [2025-10-20T19:45:08.516564] [INFO ] [LOCAL ] LUXE: Starting deletion of primary data. [2025-10-20T19:45:08.516636] [DEBUG] [LOCAL ] LUXE: No primary data file found to delete. [2025-10-20T19:45:08.517220] [INFO ] [MAIN ] LUXE: Starting full primary refresh. [2025-10-20T19:45:08.516150] [INFO ] [MAIN ] LVWR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:08.672919] [DEBUG] [LOCAL ] LVWR: Loaded 18573 existing records from primary storage. [2025-10-20T19:45:08.686885] [INFO ] [MAIN ] LVWR: Loaded 18573 existing records. [2025-10-20T19:45:09.035343] [DEBUG] [LOCAL ] LVS: Loaded 54350 existing records from primary storage. [2025-10-20T19:45:09.090432] [INFO ] [MAIN ] LVS: Loaded 54350 existing records. [2025-10-20T19:45:09.090287] [WARN ] [MAIN ] LVWR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:09.107257] [DEBUG] [LOCAL ] LUV: Loaded 54469 existing records from primary storage. [2025-10-20T19:45:09.124071] [INFO ] [MAIN ] LVWR: Wiping existing primary data before full refresh. [2025-10-20T19:45:09.181326] [INFO ] [MAIN ] LUV: Loaded 54469 existing records. [2025-10-20T19:45:09.238112] [INFO ] [REMOTE ] LUXE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:09.199783] [INFO ] [LOCAL ] LVWR: Starting deletion of primary data. [2025-10-20T19:45:09.259263] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-01. [2025-10-20T19:45:09.274892] [DEBUG] [LOCAL ] LVWR: Deleted primary data file. [2025-10-20T19:45:09.280460] [INFO ] [MAIN ] LVWR: Starting full primary refresh. [2025-10-20T19:45:09.579916] [WARN ] [MAIN ] LVS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:09.653502] [WARN ] [MAIN ] LUV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:09.649823] [INFO ] [MAIN ] LVS: Wiping existing primary data before full refresh. [2025-10-20T19:45:09.658888] [INFO ] [LOCAL ] LVS: Starting deletion of primary data. [2025-10-20T19:45:09.664130] [INFO ] [MAIN ] LUV: Wiping existing primary data before full refresh. [2025-10-20T19:45:09.688496] [DEBUG] [LOCAL ] LVS: Deleted primary data file. [2025-10-20T19:45:09.683333] [INFO ] [LOCAL ] LUV: Starting deletion of primary data. [2025-10-20T19:45:09.697671] [INFO ] [MAIN ] LVS: Starting full primary refresh. [2025-10-20T19:45:09.697730] [DEBUG] [LOCAL ] LUV: Deleted primary data file. [2025-10-20T19:45:09.768178] [INFO ] [MAIN ] LUV: Starting full primary refresh. [2025-10-20T19:45:09.768116] [DEBUG] [REMOTE ] LUXE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:45:09.861723] [WARN ] [REMOTE ] LUXE: No data returned for month 2025-01 [2025-10-20T19:45:09.997808] [FATAL] [REMOTE ] LUXE: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:45:10.036853] [FATAL] [MAIN ] LUXE: Failed to fetch primary data. [2025-10-20T19:45:10.060358] [WARN ] [MAIN ] LUXE: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:45:10.103590] [INFO ] [LOCAL ] LUXE: Starting deletion of primary data. [2025-10-20T19:45:10.209737] [DEBUG] [LOCAL ] LUXE: No primary data file found to delete. [2025-10-20T19:45:10.246955] [INFO ] [LOCAL ] LUXE: Starting deletion of derivative data. [2025-10-20T19:45:10.270006] [INFO ] [REMOTE ] LVWR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:10.302055] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-01. [2025-10-20T19:45:10.358040] [DEBUG] [LOCAL ] LUXE: No derivative data file found to delete. [2025-10-20T19:45:10.426776] [INFO ] [MAIN ] Update Progress: 1281/2367 symbols completed. LUXE took 2.6s. Estimated remaining: 2h 19m 58s. Rate: 7.8 symbols/min [2025-10-20T19:45:10.877758] [INFO ] [REMOTE ] LUV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:10.859430] [INFO ] [REMOTE ] LVS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:10.885121] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-01. [2025-10-20T19:45:10.884889] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-01. [2025-10-20T19:45:10.916292] [INFO ] [MAIN ] LW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:10.967409] [DEBUG] [LOCAL ] LW: Loaded 51966 existing records from primary storage. [2025-10-20T19:45:10.967793] [INFO ] [MAIN ] LW: Loaded 51966 existing records. [2025-10-20T19:45:11.092395] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:45:11.137258] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-02. [2025-10-20T19:45:11.159761] [WARN ] [MAIN ] LW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:11.160463] [INFO ] [MAIN ] LW: Wiping existing primary data before full refresh. [2025-10-20T19:45:11.160521] [INFO ] [LOCAL ] LW: Starting deletion of primary data. [2025-10-20T19:45:11.160751] [DEBUG] [LOCAL ] LW: Deleted primary data file. [2025-10-20T19:45:11.160808] [INFO ] [MAIN ] LW: Starting full primary refresh. [2025-10-20T19:45:11.533473] [INFO ] [REMOTE ] LW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:11.533853] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-01. [2025-10-20T19:45:11.739847] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T19:45:11.741536] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-03. [2025-10-20T19:45:12.208566] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T19:45:12.210201] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-04. [2025-10-20T19:45:12.714149] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T19:45:12.724974] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-02. [2025-10-20T19:45:12.990105] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-10-20T19:45:13.004996] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-02. [2025-10-20T19:45:13.014439] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T19:45:13.016276] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-05. [2025-10-20T19:45:13.385536] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T19:45:13.395988] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-02. [2025-10-20T19:45:13.916306] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T19:45:13.922322] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-06. [2025-10-20T19:45:14.481205] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:45:14.491472] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-03. [2025-10-20T19:45:14.815161] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T19:45:14.826422] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-03. [2025-10-20T19:45:14.978200] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T19:45:14.988290] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-03. [2025-10-20T19:45:15.391609] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:45:15.400616] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-07. [2025-10-20T19:45:16.322618] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T19:45:16.327486] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-08. [2025-10-20T19:45:16.455265] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T19:45:16.466325] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-04. [2025-10-20T19:45:16.637797] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:45:16.648924] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-04. [2025-10-20T19:45:16.739217] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:45:16.750344] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-04. [2025-10-20T19:45:16.855026] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T19:45:16.858173] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-09. [2025-10-20T19:45:17.841114] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T19:45:17.845628] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-10. [2025-10-20T19:45:18.350756] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:45:18.363283] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-05. [2025-10-20T19:45:18.434001] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T19:45:18.446721] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-05. [2025-10-20T19:45:18.562071] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T19:45:18.651490] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-05. [2025-10-20T19:45:18.766309] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T19:45:18.769169] [INFO ] [REMOTE ] LVWR: Full fetch complete (20660 records). [2025-10-20T19:45:18.769327] [DEBUG] [VALIDATOR ] LVWR: Validating primary data. [2025-10-20T19:45:19.212958] [DEBUG] [VALIDATOR ] LVWR: All expected trading days are present. [2025-10-20T19:45:19.213052] [DEBUG] [VALIDATOR ] LVWR: Primary data validation passed. [2025-10-20T19:45:19.213597] [INFO ] [MAIN ] LVWR: Fetched primary data is valid. [2025-10-20T19:45:19.213702] [DEBUG] [LOCAL ] LVWR: Saving primary data. [2025-10-20T19:45:19.214736] [DEBUG] [LOCAL ] LVWR: Trimmed 20660 records to 18569 (keeping 200 days of data) [2025-10-20T19:45:19.341353] [DEBUG] [LOCAL ] LVWR: Primary data saved (18569 records). [2025-10-20T19:45:19.341505] [INFO ] [MAIN ] LVWR: Full history saved. [2025-10-20T19:45:19.341590] [INFO ] [MAIN ] Update Progress: 1282/2367 symbols completed. LVWR took 11.5s. Estimated remaining: 2h 19m 51s. Rate: 7.8 symbols/min [2025-10-20T19:45:19.526921] [INFO ] [MAIN ] LXFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:19.546007] [DEBUG] [LOCAL ] LXFR: Loaded 16810 existing records from primary storage. [2025-10-20T19:45:19.546309] [INFO ] [MAIN ] LXFR: Loaded 16810 existing records. [2025-10-20T19:45:19.723796] [WARN ] [MAIN ] LXFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:19.723994] [INFO ] [MAIN ] LXFR: Wiping existing primary data before full refresh. [2025-10-20T19:45:19.724042] [INFO ] [LOCAL ] LXFR: Starting deletion of primary data. [2025-10-20T19:45:19.724201] [DEBUG] [LOCAL ] LXFR: Deleted primary data file. [2025-10-20T19:45:19.724252] [INFO ] [MAIN ] LXFR: Starting full primary refresh. [2025-10-20T19:45:20.081156] [INFO ] [REMOTE ] LXFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:20.081816] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-01. [2025-10-20T19:45:20.121604] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T19:45:20.179098] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:45:20.251336] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-06. [2025-10-20T19:45:20.404000] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T19:45:21.051392] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-06. [2025-10-20T19:45:21.076952] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T19:45:21.804364] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T19:45:21.851411] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-06. [2025-10-20T19:45:22.651356] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-07. [2025-10-20T19:45:22.812338] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:45:23.447934] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:45:23.457221] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-02. [2025-10-20T19:45:24.251364] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-07. [2025-10-20T19:45:24.475459] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T19:45:24.666071] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T19:45:25.051367] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-08. [2025-10-20T19:45:25.851355] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-03. [2025-10-20T19:45:26.173149] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T19:45:26.651414] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-07. [2025-10-20T19:45:26.860400] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:45:27.011358] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T19:45:27.451398] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-09. [2025-10-20T19:45:28.251374] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-08. [2025-10-20T19:45:28.536630] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:45:29.051434] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-04. [2025-10-20T19:45:29.337212] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T19:45:29.851361] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-10. [2025-10-20T19:45:29.891160] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T19:45:29.941566] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T19:45:30.651398] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-09. [2025-10-20T19:45:30.715781] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:45:30.727996] [INFO ] [REMOTE ] LUV: Full fetch complete (77865 records). [2025-10-20T19:45:30.728187] [DEBUG] [VALIDATOR ] LUV: Validating primary data. [2025-10-20T19:45:31.153358] [DEBUG] [VALIDATOR ] LUV: All expected trading days are present. [2025-10-20T19:45:31.153483] [DEBUG] [VALIDATOR ] LUV: Primary data validation passed. [2025-10-20T19:45:31.154495] [INFO ] [MAIN ] LUV: Fetched primary data is valid. [2025-10-20T19:45:31.154612] [DEBUG] [LOCAL ] LUV: Saving primary data. [2025-10-20T19:45:31.156603] [DEBUG] [LOCAL ] LUV: Trimmed 77865 records to 53689 (keeping 200 days of data) [2025-10-20T19:45:31.462215] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-05. [2025-10-20T19:45:31.493184] [DEBUG] [LOCAL ] LUV: Primary data saved (53689 records). [2025-10-20T19:45:31.493342] [INFO ] [MAIN ] LUV: Full history saved. [2025-10-20T19:45:31.493435] [INFO ] [MAIN ] Update Progress: 1283/2367 symbols completed. LUV took 23.7s. Estimated remaining: 2h 19m 47s. Rate: 7.8 symbols/min [2025-10-20T19:45:31.663411] [INFO ] [MAIN ] LXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:31.712509] [DEBUG] [LOCAL ] LXP: Loaded 51460 existing records from primary storage. [2025-10-20T19:45:31.712925] [INFO ] [MAIN ] LXP: Loaded 51460 existing records. [2025-10-20T19:45:31.889126] [WARN ] [MAIN ] LXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:31.889369] [INFO ] [MAIN ] LXP: Wiping existing primary data before full refresh. [2025-10-20T19:45:31.889417] [INFO ] [LOCAL ] LXP: Starting deletion of primary data. [2025-10-20T19:45:31.889639] [DEBUG] [LOCAL ] LXP: Deleted primary data file. [2025-10-20T19:45:31.889757] [INFO ] [MAIN ] LXP: Starting full primary refresh. [2025-10-20T19:45:32.254218] [INFO ] [REMOTE ] LXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:32.254302] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-08. [2025-10-20T19:45:32.401438] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:45:32.436788] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T19:45:33.051374] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-10. [2025-10-20T19:45:33.851364] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-06. [2025-10-20T19:45:34.090160] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T19:45:34.651351] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-01. [2025-10-20T19:45:34.817743] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T19:45:34.960878] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T19:45:34.968609] [INFO ] [REMOTE ] LW: Full fetch complete (74766 records). [2025-10-20T19:45:34.968792] [DEBUG] [VALIDATOR ] LW: Validating primary data. [2025-10-20T19:45:35.368785] [DEBUG] [VALIDATOR ] LW: All expected trading days are present. [2025-10-20T19:45:35.368879] [DEBUG] [VALIDATOR ] LW: Primary data validation passed. [2025-10-20T19:45:35.370194] [INFO ] [MAIN ] LW: Fetched primary data is valid. [2025-10-20T19:45:35.370258] [DEBUG] [LOCAL ] LW: Saving primary data. [2025-10-20T19:45:35.371737] [DEBUG] [LOCAL ] LW: Trimmed 74766 records to 51191 (keeping 200 days of data) [2025-10-20T19:45:35.462066] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-07. [2025-10-20T19:45:35.683507] [DEBUG] [LOCAL ] LW: Primary data saved (51191 records). [2025-10-20T19:45:35.683672] [INFO ] [MAIN ] LW: Full history saved. [2025-10-20T19:45:35.683785] [INFO ] [MAIN ] Update Progress: 1284/2367 symbols completed. LW took 25.2s. Estimated remaining: 2h 19m 37s. Rate: 7.8 symbols/min [2025-10-20T19:45:35.854601] [INFO ] [MAIN ] LXU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:35.886092] [DEBUG] [LOCAL ] LXU: Loaded 32174 existing records from primary storage. [2025-10-20T19:45:35.886345] [INFO ] [MAIN ] LXU: Loaded 32174 existing records. [2025-10-20T19:45:36.077337] [WARN ] [MAIN ] LXU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:36.077553] [INFO ] [MAIN ] LXU: Wiping existing primary data before full refresh. [2025-10-20T19:45:36.077602] [INFO ] [LOCAL ] LXU: Starting deletion of primary data. [2025-10-20T19:45:36.077850] [DEBUG] [LOCAL ] LXU: Deleted primary data file. [2025-10-20T19:45:36.077907] [INFO ] [MAIN ] LXU: Starting full primary refresh. [2025-10-20T19:45:36.259312] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-09. [2025-10-20T19:45:36.485227] [INFO ] [REMOTE ] LXU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:36.486839] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T19:45:36.510004] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T19:45:37.051366] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-01. [2025-10-20T19:45:37.851439] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-08. [2025-10-20T19:45:38.287676] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T19:45:38.388985] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T19:45:38.437620] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T19:45:38.651369] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-02. [2025-10-20T19:45:39.451321] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-10. [2025-10-20T19:45:40.148325] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T19:45:40.251346] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-03. [2025-10-20T19:45:41.051373] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-09. [2025-10-20T19:45:41.546929] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T19:45:41.558277] [INFO ] [REMOTE ] LVS: Full fetch complete (77722 records). [2025-10-20T19:45:41.558446] [DEBUG] [VALIDATOR ] LVS: Validating primary data. [2025-10-20T19:45:41.863375] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-02. [2025-10-20T19:45:41.967910] [DEBUG] [VALIDATOR ] LVS: All expected trading days are present. [2025-10-20T19:45:41.968073] [DEBUG] [VALIDATOR ] LVS: Primary data validation passed. [2025-10-20T19:45:41.969226] [INFO ] [MAIN ] LVS: Fetched primary data is valid. [2025-10-20T19:45:41.969405] [DEBUG] [LOCAL ] LVS: Saving primary data. [2025-10-20T19:45:42.009383] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T19:45:42.008940] [DEBUG] [LOCAL ] LVS: Trimmed 77722 records to 53570 (keeping 200 days of data) [2025-10-20T19:45:42.011909] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T19:45:42.347324] [DEBUG] [LOCAL ] LVS: Primary data saved (53570 records). [2025-10-20T19:45:42.347458] [INFO ] [MAIN ] LVS: Full history saved. [2025-10-20T19:45:42.347878] [INFO ] [MAIN ] Update Progress: 1285/2367 symbols completed. LVS took 34.5s. Estimated remaining: 2h 19m 28s. Rate: 7.8 symbols/min [2025-10-20T19:45:42.514013] [INFO ] [MAIN ] LYB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:42.564384] [DEBUG] [LOCAL ] LYB: Loaded 54398 existing records from primary storage. [2025-10-20T19:45:42.564737] [INFO ] [MAIN ] LYB: Loaded 54398 existing records. [2025-10-20T19:45:42.656532] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-04. [2025-10-20T19:45:42.743684] [WARN ] [MAIN ] LYB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:42.743923] [INFO ] [MAIN ] LYB: Wiping existing primary data before full refresh. [2025-10-20T19:45:42.743972] [INFO ] [LOCAL ] LYB: Starting deletion of primary data. [2025-10-20T19:45:42.744306] [DEBUG] [LOCAL ] LYB: Deleted primary data file. [2025-10-20T19:45:42.744364] [INFO ] [MAIN ] LYB: Starting full primary refresh. [2025-10-20T19:45:43.129037] [INFO ] [REMOTE ] LYB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:43.130500] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:45:43.451385] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-01. [2025-10-20T19:45:44.251378] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-10. [2025-10-20T19:45:44.430781] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:45:44.808175] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T19:45:45.051378] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-03. [2025-10-20T19:45:45.282108] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T19:45:45.284549] [INFO ] [REMOTE ] LXFR: Full fetch complete (24745 records). [2025-10-20T19:45:45.284621] [DEBUG] [VALIDATOR ] LXFR: Validating primary data. [2025-10-20T19:45:45.657967] [DEBUG] [VALIDATOR ] LXFR: All expected trading days are present. [2025-10-20T19:45:45.658054] [DEBUG] [VALIDATOR ] LXFR: Primary data validation passed. [2025-10-20T19:45:45.658584] [INFO ] [MAIN ] LXFR: Fetched primary data is valid. [2025-10-20T19:45:45.658667] [DEBUG] [LOCAL ] LXFR: Saving primary data. [2025-10-20T19:45:45.659287] [DEBUG] [LOCAL ] LXFR: Trimmed 24745 records to 16472 (keeping 200 days of data) [2025-10-20T19:45:45.756421] [DEBUG] [LOCAL ] LXFR: Primary data saved (16472 records). [2025-10-20T19:45:45.756553] [INFO ] [MAIN ] LXFR: Full history saved. [2025-10-20T19:45:45.756632] [INFO ] [MAIN ] Update Progress: 1286/2367 symbols completed. LXFR took 26.4s. Estimated remaining: 2h 19m 17s. Rate: 7.8 symbols/min [2025-10-20T19:45:45.856557] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-02. [2025-10-20T19:45:45.930814] [INFO ] [MAIN ] LYG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:45.979978] [DEBUG] [LOCAL ] LYG: Loaded 53435 existing records from primary storage. [2025-10-20T19:45:45.980312] [INFO ] [MAIN ] LYG: Loaded 53435 existing records. [2025-10-20T19:45:46.174012] [WARN ] [MAIN ] LYG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:46.174357] [INFO ] [MAIN ] LYG: Wiping existing primary data before full refresh. [2025-10-20T19:45:46.174491] [INFO ] [LOCAL ] LYG: Starting deletion of primary data. [2025-10-20T19:45:46.175026] [DEBUG] [LOCAL ] LYG: Deleted primary data file. [2025-10-20T19:45:46.175081] [INFO ] [MAIN ] LYG: Starting full primary refresh. [2025-10-20T19:45:46.434860] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T19:45:46.569379] [INFO ] [REMOTE ] LYG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:46.651400] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-04. [2025-10-20T19:45:47.451439] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-05. [2025-10-20T19:45:47.589356] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T19:45:48.212192] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T19:45:48.251336] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-05. [2025-10-20T19:45:49.051383] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-03. [2025-10-20T19:45:49.059091] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:45:49.213901] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T19:45:49.851360] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-06. [2025-10-20T19:45:50.651347] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-06. [2025-10-20T19:45:50.964830] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:45:51.326248] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:45:51.451334] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-07. [2025-10-20T19:45:52.251383] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-01. [2025-10-20T19:45:52.323463] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:45:52.387984] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T19:45:53.051429] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-04. [2025-10-20T19:45:53.851415] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-07. [2025-10-20T19:45:54.009184] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:45:54.651411] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-08. [2025-10-20T19:45:54.952897] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T19:45:55.451405] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-05. [2025-10-20T19:45:55.493594] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T19:45:55.609685] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T19:45:56.251418] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-08. [2025-10-20T19:45:57.051350] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-09. [2025-10-20T19:45:57.363401] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:45:57.851344] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-06. [2025-10-20T19:45:57.898475] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T19:45:58.324414] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T19:45:58.651420] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-02. [2025-10-20T19:45:59.451401] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-10. [2025-10-20T19:45:59.550555] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:46:00.251477] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-07. [2025-10-20T19:46:00.392323] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:46:00.429732] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:46:00.434191] [INFO ] [REMOTE ] LXU: Full fetch complete (45306 records). [2025-10-20T19:46:00.434298] [DEBUG] [VALIDATOR ] LXU: Validating primary data. [2025-10-20T19:46:00.824924] [DEBUG] [VALIDATOR ] LXU: All expected trading days are present. [2025-10-20T19:46:00.825004] [DEBUG] [VALIDATOR ] LXU: Primary data validation passed. [2025-10-20T19:46:00.825832] [INFO ] [MAIN ] LXU: Fetched primary data is valid. [2025-10-20T19:46:00.825894] [DEBUG] [LOCAL ] LXU: Saving primary data. [2025-10-20T19:46:00.826915] [DEBUG] [LOCAL ] LXU: Trimmed 45306 records to 31669 (keeping 200 days of data) [2025-10-20T19:46:01.013945] [DEBUG] [LOCAL ] LXU: Primary data saved (31669 records). [2025-10-20T19:46:01.014121] [INFO ] [MAIN ] LXU: Full history saved. [2025-10-20T19:46:01.014240] [INFO ] [MAIN ] Update Progress: 1287/2367 symbols completed. LXU took 25.3s. Estimated remaining: 2h 19m 15s. Rate: 7.8 symbols/min [2025-10-20T19:46:01.056472] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-03. [2025-10-20T19:46:01.193782] [INFO ] [MAIN ] LYV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:01.244565] [DEBUG] [LOCAL ] LYV: Loaded 51878 existing records from primary storage. [2025-10-20T19:46:01.244913] [INFO ] [MAIN ] LYV: Loaded 51878 existing records. [2025-10-20T19:46:01.419923] [WARN ] [MAIN ] LYV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:01.420121] [INFO ] [MAIN ] LYV: Wiping existing primary data before full refresh. [2025-10-20T19:46:01.420188] [INFO ] [LOCAL ] LYV: Starting deletion of primary data. [2025-10-20T19:46:01.420365] [DEBUG] [LOCAL ] LYV: Deleted primary data file. [2025-10-20T19:46:01.420402] [INFO ] [MAIN ] LYV: Starting full primary refresh. [2025-10-20T19:46:01.766004] [INFO ] [REMOTE ] LYV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:01.851333] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-01. [2025-10-20T19:46:02.109579] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T19:46:02.651400] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-08. [2025-10-20T19:46:03.124261] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T19:46:03.451472] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-04. [2025-10-20T19:46:03.499384] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T19:46:04.251438] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-02. [2025-10-20T19:46:05.051354] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-09. [2025-10-20T19:46:05.302740] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-08 (2.65 seconds). [2025-10-20T19:46:05.626336] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T19:46:05.851362] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-05. [2025-10-20T19:46:06.051312] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T19:46:06.651402] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-03. [2025-10-20T19:46:06.742144] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T19:46:07.451350] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-09. [2025-10-20T19:46:07.644563] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:46:08.251443] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-06. [2025-10-20T19:46:08.475176] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T19:46:09.051348] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-04. [2025-10-20T19:46:09.453310] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T19:46:09.851437] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-10. [2025-10-20T19:46:10.007540] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:46:10.651391] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-07. [2025-10-20T19:46:10.794816] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T19:46:11.451470] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-10. [2025-10-20T19:46:11.607130] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T19:46:11.620083] [INFO ] [REMOTE ] LXP: Full fetch complete (72854 records). [2025-10-20T19:46:11.620421] [DEBUG] [VALIDATOR ] LXP: Validating primary data. [2025-10-20T19:46:12.262951] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-05. [2025-10-20T19:46:12.271547] [DEBUG] [VALIDATOR ] LXP: All expected trading days are present. [2025-10-20T19:46:12.271746] [DEBUG] [VALIDATOR ] LXP: Primary data validation passed. [2025-10-20T19:46:12.273904] [INFO ] [MAIN ] LXP: Fetched primary data is valid. [2025-10-20T19:46:12.274240] [DEBUG] [LOCAL ] LXP: Saving primary data. [2025-10-20T19:46:12.276177] [DEBUG] [LOCAL ] LXP: Trimmed 72854 records to 50793 (keeping 200 days of data) [2025-10-20T19:46:12.534224] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T19:46:12.576484] [INFO ] [REMOTE ] LYB: Full fetch complete (77632 records). [2025-10-20T19:46:12.576726] [DEBUG] [VALIDATOR ] LYB: Validating primary data. [2025-10-20T19:46:12.701439] [DEBUG] [LOCAL ] LXP: Primary data saved (50793 records). [2025-10-20T19:46:12.727929] [INFO ] [MAIN ] LXP: Full history saved. [2025-10-20T19:46:12.747379] [INFO ] [MAIN ] Update Progress: 1288/2367 symbols completed. LXP took 41.3s. Estimated remaining: 2h 19m 11s. Rate: 7.8 symbols/min [2025-10-20T19:46:13.003834] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-10-20T19:46:13.187722] [INFO ] [MAIN ] LZB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:13.200423] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-08. [2025-10-20T19:46:13.306555] [DEBUG] [VALIDATOR ] LYB: All expected trading days are present. [2025-10-20T19:46:13.306771] [DEBUG] [VALIDATOR ] LYB: Primary data validation passed. [2025-10-20T19:46:13.308020] [INFO ] [MAIN ] LYB: Fetched primary data is valid. [2025-10-20T19:46:13.308153] [DEBUG] [LOCAL ] LYB: Saving primary data. [2025-10-20T19:46:13.310468] [DEBUG] [LOCAL ] LYB: Trimmed 77632 records to 53621 (keeping 200 days of data) [2025-10-20T19:46:13.447812] [DEBUG] [LOCAL ] LZB: Loaded 35693 existing records from primary storage. [2025-10-20T19:46:13.452797] [INFO ] [MAIN ] LZB: Loaded 35693 existing records. [2025-10-20T19:46:13.751232] [WARN ] [MAIN ] LZB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:13.763594] [INFO ] [MAIN ] LZB: Wiping existing primary data before full refresh. [2025-10-20T19:46:13.765760] [INFO ] [LOCAL ] LZB: Starting deletion of primary data. [2025-10-20T19:46:13.769042] [DEBUG] [LOCAL ] LZB: Deleted primary data file. [2025-10-20T19:46:13.769910] [INFO ] [MAIN ] LZB: Starting full primary refresh. [2025-10-20T19:46:14.158275] [DEBUG] [LOCAL ] LYB: Primary data saved (53621 records). [2025-10-20T19:46:14.158466] [INFO ] [MAIN ] LYB: Full history saved. [2025-10-20T19:46:14.158980] [INFO ] [MAIN ] Update Progress: 1289/2367 symbols completed. LYB took 31.8s. Estimated remaining: 2h 18m 58s. Rate: 7.8 symbols/min [2025-10-20T19:46:14.287378] [INFO ] [REMOTE ] LZB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:14.332941] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-01. [2025-10-20T19:46:14.434991] [INFO ] [MAIN ] LZM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:14.463309] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-05 (2.20 seconds). [2025-10-20T19:46:14.486938] [DEBUG] [LOCAL ] LZM: Loaded 14886 existing records from primary storage. [2025-10-20T19:46:14.487220] [INFO ] [MAIN ] LZM: Loaded 14886 existing records. [2025-10-20T19:46:14.663271] [WARN ] [MAIN ] LZM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:14.658686] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-06. [2025-10-20T19:46:14.664230] [INFO ] [MAIN ] LZM: Wiping existing primary data before full refresh. [2025-10-20T19:46:14.664284] [INFO ] [LOCAL ] LZM: Starting deletion of primary data. [2025-10-20T19:46:14.664432] [DEBUG] [LOCAL ] LZM: Deleted primary data file. [2025-10-20T19:46:14.664508] [INFO ] [MAIN ] LZM: Starting full primary refresh. [2025-10-20T19:46:15.051119] [INFO ] [REMOTE ] LZM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:15.046183] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-10-20T19:46:15.451331] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-09. [2025-10-20T19:46:15.686848] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T19:46:16.004578] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T19:46:16.251364] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-01. [2025-10-20T19:46:17.050751] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T19:46:17.052923] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-02. [2025-10-20T19:46:17.360407] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T19:46:17.851369] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-07. [2025-10-20T19:46:18.493137] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T19:46:18.651373] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-02. [2025-10-20T19:46:19.375597] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T19:46:19.451334] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-03. [2025-10-20T19:46:19.619728] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:46:20.251390] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-03. [2025-10-20T19:46:21.011812] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:46:21.051341] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-04. [2025-10-20T19:46:21.151737] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T19:46:21.851397] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-10. [2025-10-20T19:46:22.511827] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T19:46:22.651374] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-04. [2025-10-20T19:46:23.451355] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-05. [2025-10-20T19:46:23.702152] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:46:23.715964] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T19:46:23.725678] [INFO ] [REMOTE ] LYG: Full fetch complete (76212 records). [2025-10-20T19:46:23.725803] [DEBUG] [VALIDATOR ] LYG: Validating primary data. [2025-10-20T19:46:24.109049] [DEBUG] [VALIDATOR ] LYG: All expected trading days are present. [2025-10-20T19:46:24.109142] [DEBUG] [VALIDATOR ] LYG: Primary data validation passed. [2025-10-20T19:46:24.110126] [INFO ] [MAIN ] LYG: Fetched primary data is valid. [2025-10-20T19:46:24.110189] [DEBUG] [LOCAL ] LYG: Saving primary data. [2025-10-20T19:46:24.111594] [DEBUG] [LOCAL ] LYG: Trimmed 76212 records to 52666 (keeping 200 days of data) [2025-10-20T19:46:24.274693] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-05. [2025-10-20T19:46:24.414824] [DEBUG] [LOCAL ] LYG: Primary data saved (52666 records). [2025-10-20T19:46:24.414954] [INFO ] [MAIN ] LYG: Full history saved. [2025-10-20T19:46:24.415027] [INFO ] [MAIN ] Update Progress: 1290/2367 symbols completed. LYG took 38.7s. Estimated remaining: 2h 18m 52s. Rate: 7.8 symbols/min [2025-10-20T19:46:24.580810] [INFO ] [MAIN ] M: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:24.634380] [DEBUG] [LOCAL ] M: Loaded 54325 existing records from primary storage. [2025-10-20T19:46:24.634714] [INFO ] [MAIN ] M: Loaded 54325 existing records. [2025-10-20T19:46:24.827811] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T19:46:24.832484] [WARN ] [MAIN ] M: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:24.839673] [INFO ] [MAIN ] M: Wiping existing primary data before full refresh. [2025-10-20T19:46:24.839766] [INFO ] [LOCAL ] M: Starting deletion of primary data. [2025-10-20T19:46:24.840337] [DEBUG] [LOCAL ] M: Deleted primary data file. [2025-10-20T19:46:24.840397] [INFO ] [MAIN ] M: Starting full primary refresh. [2025-10-20T19:46:25.056878] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-06. [2025-10-20T19:46:25.167317] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T19:46:25.210073] [INFO ] [REMOTE ] M: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:25.851438] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-06. [2025-10-20T19:46:26.597188] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:46:26.612960] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T19:46:26.651404] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-07. [2025-10-20T19:46:27.451382] [INFO ] [REMOTE ] M: Fetching primary data for 2025-01. [2025-10-20T19:46:27.564870] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T19:46:28.251349] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-08. [2025-10-20T19:46:29.051428] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-08. [2025-10-20T19:46:29.354697] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T19:46:29.812885] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T19:46:29.851359] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-09. [2025-10-20T19:46:29.884901] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T19:46:30.651396] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-07. [2025-10-20T19:46:30.681221] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T19:46:31.451426] [INFO ] [REMOTE ] M: Fetching primary data for 2025-02. [2025-10-20T19:46:32.050987] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T19:46:32.251441] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-10. [2025-10-20T19:46:33.051373] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-09. [2025-10-20T19:46:33.248582] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T19:46:33.380384] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T19:46:33.384099] [INFO ] [REMOTE ] LZM: Full fetch complete (19736 records). [2025-10-20T19:46:33.384194] [DEBUG] [VALIDATOR ] LZM: Validating primary data. [2025-10-20T19:46:33.744092] [DEBUG] [VALIDATOR ] LZM: All expected trading days are present. [2025-10-20T19:46:33.744172] [DEBUG] [VALIDATOR ] LZM: Primary data validation passed. [2025-10-20T19:46:33.744566] [INFO ] [MAIN ] LZM: Fetched primary data is valid. [2025-10-20T19:46:33.744604] [DEBUG] [LOCAL ] LZM: Saving primary data. [2025-10-20T19:46:33.745213] [DEBUG] [LOCAL ] LZM: Trimmed 19736 records to 14708 (keeping 200 days of data) [2025-10-20T19:46:33.825057] [DEBUG] [LOCAL ] LZM: Primary data saved (14708 records). [2025-10-20T19:46:33.825231] [INFO ] [MAIN ] LZM: Full history saved. [2025-10-20T19:46:33.825315] [INFO ] [MAIN ] Update Progress: 1291/2367 symbols completed. LZM took 19.6s. Estimated remaining: 2h 18m 46s. Rate: 7.8 symbols/min [2025-10-20T19:46:33.856574] [INFO ] [REMOTE ] M: Fetching primary data for 2025-03. [2025-10-20T19:46:33.995965] [INFO ] [MAIN ] MA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:34.045365] [DEBUG] [LOCAL ] MA: Loaded 53252 existing records from primary storage. [2025-10-20T19:46:34.045692] [INFO ] [MAIN ] MA: Loaded 53252 existing records. [2025-10-20T19:46:34.221928] [WARN ] [MAIN ] MA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:34.222169] [INFO ] [MAIN ] MA: Wiping existing primary data before full refresh. [2025-10-20T19:46:34.222224] [INFO ] [LOCAL ] MA: Starting deletion of primary data. [2025-10-20T19:46:34.222811] [DEBUG] [LOCAL ] MA: Deleted primary data file. [2025-10-20T19:46:34.222879] [INFO ] [MAIN ] MA: Starting full primary refresh. [2025-10-20T19:46:34.584874] [INFO ] [REMOTE ] MA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:34.651328] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-01. [2025-10-20T19:46:34.826281] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:46:35.451397] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-10. [2025-10-20T19:46:35.755206] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T19:46:36.251361] [INFO ] [REMOTE ] M: Fetching primary data for 2025-04. [2025-10-20T19:46:36.286933] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T19:46:36.294832] [INFO ] [REMOTE ] LYV: Full fetch complete (74181 records). [2025-10-20T19:46:36.294943] [DEBUG] [VALIDATOR ] LYV: Validating primary data. [2025-10-20T19:46:36.689728] [DEBUG] [VALIDATOR ] LYV: All expected trading days are present. [2025-10-20T19:46:36.689835] [DEBUG] [VALIDATOR ] LYV: Primary data validation passed. [2025-10-20T19:46:36.690841] [INFO ] [MAIN ] LYV: Fetched primary data is valid. [2025-10-20T19:46:36.690901] [DEBUG] [LOCAL ] LYV: Saving primary data. [2025-10-20T19:46:36.692423] [DEBUG] [LOCAL ] LYV: Trimmed 74181 records to 51086 (keeping 200 days of data) [2025-10-20T19:46:36.810483] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-01 (2.16 seconds). [2025-10-20T19:46:37.026841] [DEBUG] [LOCAL ] LYV: Primary data saved (51086 records). [2025-10-20T19:46:37.026967] [INFO ] [MAIN ] LYV: Full history saved. [2025-10-20T19:46:37.027047] [INFO ] [MAIN ] Update Progress: 1292/2367 symbols completed. LYV took 36.0s. Estimated remaining: 2h 18m 34s. Rate: 7.8 symbols/min [2025-10-20T19:46:37.057519] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-08. [2025-10-20T19:46:37.198344] [INFO ] [MAIN ] MAA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:37.239685] [DEBUG] [LOCAL ] MAA: Loaded 43519 existing records from primary storage. [2025-10-20T19:46:37.239991] [INFO ] [MAIN ] MAA: Loaded 43519 existing records. [2025-10-20T19:46:37.428688] [WARN ] [MAIN ] MAA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:37.428915] [INFO ] [MAIN ] MAA: Wiping existing primary data before full refresh. [2025-10-20T19:46:37.428974] [INFO ] [LOCAL ] MAA: Starting deletion of primary data. [2025-10-20T19:46:37.429175] [DEBUG] [LOCAL ] MAA: Deleted primary data file. [2025-10-20T19:46:37.429246] [INFO ] [MAIN ] MAA: Starting full primary refresh. [2025-10-20T19:46:37.785802] [INFO ] [REMOTE ] MAA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:37.851335] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-01. [2025-10-20T19:46:38.219263] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T19:46:38.595977] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T19:46:38.651388] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-02. [2025-10-20T19:46:39.352093] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T19:46:39.451370] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-09. [2025-10-20T19:46:40.251361] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-02. [2025-10-20T19:46:40.487448] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T19:46:40.869989] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:46:41.051412] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-10. [2025-10-20T19:46:41.725026] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:46:41.851405] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-03. [2025-10-20T19:46:42.619748] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T19:46:42.624615] [INFO ] [REMOTE ] LZB: Full fetch complete (50965 records). [2025-10-20T19:46:42.624725] [DEBUG] [VALIDATOR ] LZB: Validating primary data. [2025-10-20T19:46:42.656494] [INFO ] [REMOTE ] M: Fetching primary data for 2025-05. [2025-10-20T19:46:42.996022] [DEBUG] [VALIDATOR ] LZB: All expected trading days are present. [2025-10-20T19:46:42.996099] [DEBUG] [VALIDATOR ] LZB: Primary data validation passed. [2025-10-20T19:46:42.996869] [INFO ] [MAIN ] LZB: Fetched primary data is valid. [2025-10-20T19:46:42.996931] [DEBUG] [LOCAL ] LZB: Saving primary data. [2025-10-20T19:46:42.997882] [DEBUG] [LOCAL ] LZB: Trimmed 50965 records to 35168 (keeping 200 days of data) [2025-10-20T19:46:43.202898] [DEBUG] [LOCAL ] LZB: Primary data saved (35168 records). [2025-10-20T19:46:43.203015] [INFO ] [MAIN ] LZB: Full history saved. [2025-10-20T19:46:43.203087] [INFO ] [MAIN ] Update Progress: 1293/2367 symbols completed. LZB took 30.5s. Estimated remaining: 2h 18m 25s. Rate: 7.8 symbols/min [2025-10-20T19:46:43.374099] [INFO ] [MAIN ] MAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:43.424563] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T19:46:43.449309] [DEBUG] [LOCAL ] MAC: Loaded 50453 existing records from primary storage. [2025-10-20T19:46:43.449591] [INFO ] [MAIN ] MAC: Loaded 50453 existing records. [2025-10-20T19:46:43.460711] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-04. [2025-10-20T19:46:43.625280] [WARN ] [MAIN ] MAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:43.625487] [INFO ] [MAIN ] MAC: Wiping existing primary data before full refresh. [2025-10-20T19:46:43.625534] [INFO ] [LOCAL ] MAC: Starting deletion of primary data. [2025-10-20T19:46:43.625760] [DEBUG] [LOCAL ] MAC: Deleted primary data file. [2025-10-20T19:46:43.625817] [INFO ] [MAIN ] MAC: Starting full primary refresh. [2025-10-20T19:46:43.972755] [INFO ] [REMOTE ] MAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:44.251372] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-01. [2025-10-20T19:46:44.641304] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T19:46:45.051411] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-03. [2025-10-20T19:46:45.136297] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:46:45.416054] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:46:45.851368] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-05. [2025-10-20T19:46:46.651347] [INFO ] [REMOTE ] M: Fetching primary data for 2025-06. [2025-10-20T19:46:46.874066] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T19:46:47.343058] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T19:46:47.451340] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-06. [2025-10-20T19:46:48.251346] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-02. [2025-10-20T19:46:48.516879] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T19:46:49.051407] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-04. [2025-10-20T19:46:49.113228] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:46:49.395079] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:46:49.851465] [INFO ] [REMOTE ] M: Fetching primary data for 2025-07. [2025-10-20T19:46:50.651400] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-07. [2025-10-20T19:46:51.011909] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T19:46:51.451346] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-03. [2025-10-20T19:46:51.741050] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:46:51.907032] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T19:46:52.251413] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-08. [2025-10-20T19:46:53.051459] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-05. [2025-10-20T19:46:53.197480] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:46:53.784639] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T19:46:53.851383] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-09. [2025-10-20T19:46:54.651429] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-04. [2025-10-20T19:46:55.451354] [INFO ] [REMOTE ] M: Fetching primary data for 2025-08. [2025-10-20T19:46:55.661313] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-05 (2.61 seconds). [2025-10-20T19:46:55.736158] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T19:46:56.251352] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-06. [2025-10-20T19:46:56.358520] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:46:57.051436] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-05. [2025-10-20T19:46:57.174158] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:46:57.851352] [INFO ] [REMOTE ] M: Fetching primary data for 2025-09. [2025-10-20T19:46:58.200509] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T19:46:58.388488] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T19:46:58.651448] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-06. [2025-10-20T19:46:59.451408] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-10. [2025-10-20T19:46:59.801994] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T19:47:00.251410] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-07. [2025-10-20T19:47:00.286826] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:47:01.051373] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-07. [2025-10-20T19:47:01.454103] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T19:47:01.460308] [INFO ] [REMOTE ] MAA: Full fetch complete (61664 records). [2025-10-20T19:47:01.460427] [DEBUG] [VALIDATOR ] MAA: Validating primary data. [2025-10-20T19:47:01.836488] [DEBUG] [VALIDATOR ] MAA: All expected trading days are present. [2025-10-20T19:47:01.836574] [DEBUG] [VALIDATOR ] MAA: Primary data validation passed. [2025-10-20T19:47:01.837454] [INFO ] [MAIN ] MAA: Fetched primary data is valid. [2025-10-20T19:47:01.837517] [DEBUG] [LOCAL ] MAA: Saving primary data. [2025-10-20T19:47:01.838822] [DEBUG] [LOCAL ] MAA: Trimmed 61664 records to 42908 (keeping 200 days of data) [2025-10-20T19:47:01.892616] [INFO ] [REMOTE ] M: Fetching primary data for 2025-10. [2025-10-20T19:47:02.106785] [DEBUG] [LOCAL ] MAA: Primary data saved (42908 records). [2025-10-20T19:47:02.106902] [INFO ] [MAIN ] MAA: Full history saved. [2025-10-20T19:47:02.106973] [INFO ] [MAIN ] Update Progress: 1294/2367 symbols completed. MAA took 25.1s. Estimated remaining: 2h 18m 27s. Rate: 7.8 symbols/min [2025-10-20T19:47:02.281365] [INFO ] [MAIN ] MAGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:02.316777] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T19:47:02.354105] [DEBUG] [LOCAL ] MAGN: Loaded 36703 existing records from primary storage. [2025-10-20T19:47:02.354386] [INFO ] [MAIN ] MAGN: Loaded 36703 existing records. [2025-10-20T19:47:02.550973] [WARN ] [MAIN ] MAGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:02.551163] [INFO ] [MAIN ] MAGN: Wiping existing primary data before full refresh. [2025-10-20T19:47:02.551209] [INFO ] [LOCAL ] MAGN: Starting deletion of primary data. [2025-10-20T19:47:02.551724] [DEBUG] [LOCAL ] MAGN: Deleted primary data file. [2025-10-20T19:47:02.551818] [INFO ] [MAIN ] MAGN: Starting full primary refresh. [2025-10-20T19:47:02.651484] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-08. [2025-10-20T19:47:02.931547] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T19:47:02.935744] [INFO ] [REMOTE ] MAGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:03.451335] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-08. [2025-10-20T19:47:03.933781] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T19:47:03.941553] [INFO ] [REMOTE ] M: Full fetch complete (77658 records). [2025-10-20T19:47:03.941713] [DEBUG] [VALIDATOR ] M: Validating primary data. [2025-10-20T19:47:04.256262] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-01. [2025-10-20T19:47:04.406070] [DEBUG] [VALIDATOR ] M: All expected trading days are present. [2025-10-20T19:47:04.406266] [DEBUG] [VALIDATOR ] M: Primary data validation passed. [2025-10-20T19:47:04.407834] [INFO ] [MAIN ] M: Fetched primary data is valid. [2025-10-20T19:47:04.408072] [DEBUG] [LOCAL ] M: Saving primary data. [2025-10-20T19:47:04.410516] [DEBUG] [LOCAL ] M: Trimmed 77658 records to 53544 (keeping 200 days of data) [2025-10-20T19:47:04.565757] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T19:47:04.877094] [DEBUG] [LOCAL ] M: Primary data saved (53544 records). [2025-10-20T19:47:04.877388] [INFO ] [MAIN ] M: Full history saved. [2025-10-20T19:47:04.877535] [INFO ] [MAIN ] Update Progress: 1295/2367 symbols completed. M took 40.5s. Estimated remaining: 2h 18m 15s. Rate: 7.8 symbols/min [2025-10-20T19:47:05.055157] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-09. [2025-10-20T19:47:05.128280] [INFO ] [MAIN ] MAIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:05.166260] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T19:47:05.219638] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T19:47:05.243886] [DEBUG] [LOCAL ] MAIN: Loaded 42467 existing records from primary storage. [2025-10-20T19:47:05.244389] [INFO ] [MAIN ] MAIN: Loaded 42467 existing records. [2025-10-20T19:47:05.437612] [WARN ] [MAIN ] MAIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:05.437885] [INFO ] [MAIN ] MAIN: Wiping existing primary data before full refresh. [2025-10-20T19:47:05.437952] [INFO ] [LOCAL ] MAIN: Starting deletion of primary data. [2025-10-20T19:47:05.438199] [DEBUG] [LOCAL ] MAIN: Deleted primary data file. [2025-10-20T19:47:05.438260] [INFO ] [MAIN ] MAIN: Starting full primary refresh. [2025-10-20T19:47:05.784559] [INFO ] [REMOTE ] MAIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:05.851331] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-01. [2025-10-20T19:47:06.651404] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-02. [2025-10-20T19:47:07.024671] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:47:07.451385] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-09. [2025-10-20T19:47:07.569078] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T19:47:07.983029] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T19:47:08.251425] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-03. [2025-10-20T19:47:09.051383] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-02. [2025-10-20T19:47:09.331771] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T19:47:09.851338] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-10. [2025-10-20T19:47:09.980339] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:47:10.651374] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-10. [2025-10-20T19:47:11.458292] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-04. [2025-10-20T19:47:11.460962] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-02 (2.41 seconds). [2025-10-20T19:47:11.831226] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T19:47:11.839298] [INFO ] [REMOTE ] MAC: Full fetch complete (71608 records). [2025-10-20T19:47:11.839463] [DEBUG] [VALIDATOR ] MAC: Validating primary data. [2025-10-20T19:47:12.253462] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-03. [2025-10-20T19:47:12.275903] [DEBUG] [VALIDATOR ] MAC: All expected trading days are present. [2025-10-20T19:47:12.275985] [DEBUG] [VALIDATOR ] MAC: Primary data validation passed. [2025-10-20T19:47:12.277053] [INFO ] [MAIN ] MAC: Fetched primary data is valid. [2025-10-20T19:47:12.277121] [DEBUG] [LOCAL ] MAC: Saving primary data. [2025-10-20T19:47:12.278788] [DEBUG] [LOCAL ] MAC: Trimmed 71608 records to 49690 (keeping 200 days of data) [2025-10-20T19:47:12.587684] [DEBUG] [LOCAL ] MAC: Primary data saved (49690 records). [2025-10-20T19:47:12.587954] [INFO ] [MAIN ] MAC: Full history saved. [2025-10-20T19:47:12.588101] [INFO ] [MAIN ] Update Progress: 1296/2367 symbols completed. MAC took 29.4s. Estimated remaining: 2h 18m 7s. Rate: 7.8 symbols/min [2025-10-20T19:47:12.750714] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T19:47:12.790717] [INFO ] [MAIN ] MAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:12.791088] [INFO ] [REMOTE ] MA: Full fetch complete (76410 records). [2025-10-20T19:47:12.791749] [DEBUG] [VALIDATOR ] MA: Validating primary data. [2025-10-20T19:47:13.163206] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:47:13.169256] [DEBUG] [LOCAL ] MAN: Loaded 44786 existing records from primary storage. [2025-10-20T19:47:13.210034] [INFO ] [MAIN ] MAN: Loaded 44786 existing records. [2025-10-20T19:47:13.264960] [DEBUG] [VALIDATOR ] MA: All expected trading days are present. [2025-10-20T19:47:13.278116] [DEBUG] [VALIDATOR ] MA: Primary data validation passed. [2025-10-20T19:47:13.298040] [INFO ] [MAIN ] MA: Fetched primary data is valid. [2025-10-20T19:47:13.278444] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-05. [2025-10-20T19:47:13.317599] [DEBUG] [LOCAL ] MA: Saving primary data. [2025-10-20T19:47:13.393957] [DEBUG] [LOCAL ] MA: Trimmed 76410 records to 52467 (keeping 200 days of data) [2025-10-20T19:47:13.541824] [WARN ] [MAIN ] MAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:13.568533] [INFO ] [MAIN ] MAN: Wiping existing primary data before full refresh. [2025-10-20T19:47:13.586575] [INFO ] [LOCAL ] MAN: Starting deletion of primary data. [2025-10-20T19:47:13.598008] [DEBUG] [LOCAL ] MAN: Deleted primary data file. [2025-10-20T19:47:13.600031] [INFO ] [MAIN ] MAN: Starting full primary refresh. [2025-10-20T19:47:14.186071] [INFO ] [REMOTE ] MAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:14.188757] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-01. [2025-10-20T19:47:14.203258] [DEBUG] [LOCAL ] MA: Primary data saved (52467 records). [2025-10-20T19:47:14.203669] [INFO ] [MAIN ] MA: Full history saved. [2025-10-20T19:47:14.204216] [INFO ] [MAIN ] Update Progress: 1297/2367 symbols completed. MA took 40.4s. Estimated remaining: 2h 17m 54s. Rate: 7.8 symbols/min [2025-10-20T19:47:14.331675] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T19:47:14.408341] [INFO ] [MAIN ] MANU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:14.436530] [DEBUG] [LOCAL ] MANU: Loaded 27565 existing records from primary storage. [2025-10-20T19:47:14.436809] [INFO ] [MAIN ] MANU: Loaded 27565 existing records. [2025-10-20T19:47:14.612738] [WARN ] [MAIN ] MANU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:14.612920] [INFO ] [MAIN ] MANU: Wiping existing primary data before full refresh. [2025-10-20T19:47:14.612966] [INFO ] [LOCAL ] MANU: Starting deletion of primary data. [2025-10-20T19:47:14.613138] [DEBUG] [LOCAL ] MANU: Deleted primary data file. [2025-10-20T19:47:14.613175] [INFO ] [MAIN ] MANU: Starting full primary refresh. [2025-10-20T19:47:14.655375] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-04. [2025-10-20T19:47:14.916173] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T19:47:14.999949] [INFO ] [REMOTE ] MANU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:15.451403] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-01. [2025-10-20T19:47:15.759604] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T19:47:16.251412] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-06. [2025-10-20T19:47:16.606625] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T19:47:16.643429] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:47:17.051390] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-02. [2025-10-20T19:47:17.540438] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T19:47:17.851389] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-05. [2025-10-20T19:47:18.449911] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T19:47:18.651444] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-03. [2025-10-20T19:47:19.448812] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T19:47:19.457426] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-02. [2025-10-20T19:47:20.251426] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-06. [2025-10-20T19:47:20.319052] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:47:20.627824] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T19:47:21.051396] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-07. [2025-10-20T19:47:21.851387] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-03. [2025-10-20T19:47:21.906439] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:47:22.523134] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:47:22.651671] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-07. [2025-10-20T19:47:23.125759] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T19:47:23.451382] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-04. [2025-10-20T19:47:24.248311] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:47:24.256993] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-04. [2025-10-20T19:47:24.606981] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T19:47:25.051450] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-08. [2025-10-20T19:47:25.859605] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-05. [2025-10-20T19:47:25.879955] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:47:26.350696] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T19:47:26.651481] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-09. [2025-10-20T19:47:27.003142] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T19:47:27.451412] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-08. [2025-10-20T19:47:28.251398] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-05. [2025-10-20T19:47:28.286588] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T19:47:29.051378] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-10. [2025-10-20T19:47:29.335292] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T19:47:29.851389] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-06. [2025-10-20T19:47:30.117326] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T19:47:30.651409] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-09. [2025-10-20T19:47:30.993154] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:47:31.000388] [INFO ] [REMOTE ] MAGN: Full fetch complete (51892 records). [2025-10-20T19:47:31.000540] [DEBUG] [VALIDATOR ] MAGN: Validating primary data. [2025-10-20T19:47:31.440008] [DEBUG] [VALIDATOR ] MAGN: All expected trading days are present. [2025-10-20T19:47:31.442673] [DEBUG] [VALIDATOR ] MAGN: Primary data validation passed. [2025-10-20T19:47:31.444944] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T19:47:31.443843] [INFO ] [MAIN ] MAGN: Fetched primary data is valid. [2025-10-20T19:47:31.451832] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-06. [2025-10-20T19:47:31.453772] [DEBUG] [LOCAL ] MAGN: Saving primary data. [2025-10-20T19:47:31.454983] [DEBUG] [LOCAL ] MAGN: Trimmed 51892 records to 36341 (keeping 200 days of data) [2025-10-20T19:47:31.667616] [DEBUG] [LOCAL ] MAGN: Primary data saved (36341 records). [2025-10-20T19:47:31.667800] [INFO ] [MAIN ] MAGN: Full history saved. [2025-10-20T19:47:31.667887] [INFO ] [MAIN ] Update Progress: 1298/2367 symbols completed. MAGN took 29.6s. Estimated remaining: 2h 17m 55s. Rate: 7.8 symbols/min [2025-10-20T19:47:31.843154] [INFO ] [MAIN ] MAS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:31.894680] [DEBUG] [LOCAL ] MAS: Loaded 53044 existing records from primary storage. [2025-10-20T19:47:31.895011] [INFO ] [MAIN ] MAS: Loaded 53044 existing records. [2025-10-20T19:47:32.077467] [WARN ] [MAIN ] MAS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:32.077743] [INFO ] [MAIN ] MAS: Wiping existing primary data before full refresh. [2025-10-20T19:47:32.077798] [INFO ] [LOCAL ] MAS: Starting deletion of primary data. [2025-10-20T19:47:32.077978] [DEBUG] [LOCAL ] MAS: Deleted primary data file. [2025-10-20T19:47:32.078016] [INFO ] [MAIN ] MAS: Starting full primary refresh. [2025-10-20T19:47:32.261424] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-07. [2025-10-20T19:47:32.443700] [INFO ] [REMOTE ] MAS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:32.469744] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T19:47:32.979011] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:47:33.051379] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-07. [2025-10-20T19:47:33.649835] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T19:47:33.851389] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-01. [2025-10-20T19:47:34.592587] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:47:34.651349] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-08. [2025-10-20T19:47:35.451351] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-08. [2025-10-20T19:47:35.590523] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T19:47:35.723701] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T19:47:36.251365] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-02. [2025-10-20T19:47:36.498386] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:47:37.051408] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-10. [2025-10-20T19:47:37.851468] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-09. [2025-10-20T19:47:37.907378] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:47:38.651477] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-03. [2025-10-20T19:47:38.938413] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T19:47:38.945293] [INFO ] [REMOTE ] MAIN: Full fetch complete (60425 records). [2025-10-20T19:47:38.945430] [DEBUG] [VALIDATOR ] MAIN: Validating primary data. [2025-10-20T19:47:39.340964] [DEBUG] [VALIDATOR ] MAIN: All expected trading days are present. [2025-10-20T19:47:39.341105] [DEBUG] [VALIDATOR ] MAIN: Primary data validation passed. [2025-10-20T19:47:39.341993] [INFO ] [MAIN ] MAIN: Fetched primary data is valid. [2025-10-20T19:47:39.342099] [DEBUG] [LOCAL ] MAIN: Saving primary data. [2025-10-20T19:47:39.343621] [DEBUG] [LOCAL ] MAIN: Trimmed 60425 records to 41956 (keeping 200 days of data) [2025-10-20T19:47:39.370535] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:47:39.465305] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-09. [2025-10-20T19:47:39.637787] [DEBUG] [LOCAL ] MAIN: Primary data saved (41956 records). [2025-10-20T19:47:39.637939] [INFO ] [MAIN ] MAIN: Full history saved. [2025-10-20T19:47:39.638020] [INFO ] [MAIN ] Update Progress: 1299/2367 symbols completed. MAIN took 34.8s. Estimated remaining: 2h 17m 47s. Rate: 7.8 symbols/min [2025-10-20T19:47:39.806272] [INFO ] [MAIN ] MATV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:39.839711] [DEBUG] [LOCAL ] MATV: Loaded 36069 existing records from primary storage. [2025-10-20T19:47:39.839980] [INFO ] [MAIN ] MATV: Loaded 36069 existing records. [2025-10-20T19:47:40.019475] [WARN ] [MAIN ] MATV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:40.019706] [INFO ] [MAIN ] MATV: Wiping existing primary data before full refresh. [2025-10-20T19:47:40.019765] [INFO ] [LOCAL ] MATV: Starting deletion of primary data. [2025-10-20T19:47:40.020011] [DEBUG] [LOCAL ] MATV: Deleted primary data file. [2025-10-20T19:47:40.020067] [INFO ] [MAIN ] MATV: Starting full primary refresh. [2025-10-20T19:47:40.252221] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-10. [2025-10-20T19:47:40.377543] [INFO ] [REMOTE ] MATV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:40.633955] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T19:47:40.927132] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:47:41.051398] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-04. [2025-10-20T19:47:41.851385] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-01. [2025-10-20T19:47:41.946486] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T19:47:41.953394] [INFO ] [REMOTE ] MAN: Full fetch complete (62781 records). [2025-10-20T19:47:41.953548] [DEBUG] [VALIDATOR ] MAN: Validating primary data. [2025-10-20T19:47:42.344049] [DEBUG] [VALIDATOR ] MAN: All expected trading days are present. [2025-10-20T19:47:42.344323] [DEBUG] [VALIDATOR ] MAN: Primary data validation passed. [2025-10-20T19:47:42.345221] [INFO ] [MAIN ] MAN: Fetched primary data is valid. [2025-10-20T19:47:42.345282] [DEBUG] [LOCAL ] MAN: Saving primary data. [2025-10-20T19:47:42.346479] [DEBUG] [LOCAL ] MAN: Trimmed 62781 records to 44330 (keeping 200 days of data) [2025-10-20T19:47:42.600635] [DEBUG] [LOCAL ] MAN: Primary data saved (44330 records). [2025-10-20T19:47:42.600790] [INFO ] [MAIN ] MAN: Full history saved. [2025-10-20T19:47:42.600877] [INFO ] [MAIN ] Update Progress: 1300/2367 symbols completed. MAN took 30.0s. Estimated remaining: 2h 17m 35s. Rate: 7.8 symbols/min [2025-10-20T19:47:42.663860] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-10. [2025-10-20T19:47:42.773914] [INFO ] [MAIN ] MATX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:42.807242] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T19:47:42.843819] [DEBUG] [LOCAL ] MATX: Loaded 32418 existing records from primary storage. [2025-10-20T19:47:42.844085] [INFO ] [MAIN ] MATX: Loaded 32418 existing records. [2025-10-20T19:47:43.018562] [WARN ] [MAIN ] MATX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:43.018792] [INFO ] [MAIN ] MATX: Wiping existing primary data before full refresh. [2025-10-20T19:47:43.018839] [INFO ] [LOCAL ] MATX: Starting deletion of primary data. [2025-10-20T19:47:43.019216] [DEBUG] [LOCAL ] MATX: Deleted primary data file. [2025-10-20T19:47:43.019272] [INFO ] [MAIN ] MATX: Starting full primary refresh. [2025-10-20T19:47:43.258866] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:47:43.396744] [INFO ] [REMOTE ] MATX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:43.451328] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-01. [2025-10-20T19:47:44.132599] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T19:47:44.137373] [INFO ] [REMOTE ] MANU: Full fetch complete (39239 records). [2025-10-20T19:47:44.137448] [DEBUG] [VALIDATOR ] MANU: Validating primary data. [2025-10-20T19:47:44.255670] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-05. [2025-10-20T19:47:44.517226] [DEBUG] [VALIDATOR ] MANU: All expected trading days are present. [2025-10-20T19:47:44.517360] [DEBUG] [VALIDATOR ] MANU: Primary data validation passed. [2025-10-20T19:47:44.518202] [INFO ] [MAIN ] MANU: Fetched primary data is valid. [2025-10-20T19:47:44.518334] [DEBUG] [LOCAL ] MANU: Saving primary data. [2025-10-20T19:47:44.553153] [DEBUG] [LOCAL ] MANU: Trimmed 39239 records to 27281 (keeping 200 days of data) [2025-10-20T19:47:44.553048] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:47:44.708622] [DEBUG] [LOCAL ] MANU: Primary data saved (27281 records). [2025-10-20T19:47:44.708766] [INFO ] [MAIN ] MANU: Full history saved. [2025-10-20T19:47:44.708846] [INFO ] [MAIN ] Update Progress: 1301/2367 symbols completed. MANU took 30.5s. Estimated remaining: 2h 17m 23s. Rate: 7.8 symbols/min [2025-10-20T19:47:44.875177] [INFO ] [MAIN ] MAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:44.905881] [DEBUG] [LOCAL ] MAX: Loaded 31450 existing records from primary storage. [2025-10-20T19:47:44.906141] [INFO ] [MAIN ] MAX: Loaded 31450 existing records. [2025-10-20T19:47:45.061759] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-02. [2025-10-20T19:47:45.082684] [WARN ] [MAIN ] MAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:45.082933] [INFO ] [MAIN ] MAX: Wiping existing primary data before full refresh. [2025-10-20T19:47:45.082982] [INFO ] [LOCAL ] MAX: Starting deletion of primary data. [2025-10-20T19:47:45.083159] [DEBUG] [LOCAL ] MAX: Deleted primary data file. [2025-10-20T19:47:45.083214] [INFO ] [MAIN ] MAX: Starting full primary refresh. [2025-10-20T19:47:45.436811] [INFO ] [REMOTE ] MAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:45.851352] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-01. [2025-10-20T19:47:46.009072] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T19:47:46.651400] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-06. [2025-10-20T19:47:46.863892] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T19:47:47.364132] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T19:47:47.451381] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-03. [2025-10-20T19:47:48.028692] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:47:48.251443] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-02. [2025-10-20T19:47:49.051448] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-07. [2025-10-20T19:47:49.163879] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T19:47:49.475346] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T19:47:49.851398] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-03. [2025-10-20T19:47:50.651379] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-02. [2025-10-20T19:47:50.845975] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:47:51.150134] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:47:51.451427] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-04. [2025-10-20T19:47:51.885709] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:47:52.251398] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-04. [2025-10-20T19:47:52.776100] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T19:47:53.051396] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-03. [2025-10-20T19:47:53.373821] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T19:47:53.851474] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-05. [2025-10-20T19:47:54.536486] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:47:54.651398] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-05. [2025-10-20T19:47:55.283322] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T19:47:55.451396] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-06. [2025-10-20T19:47:56.135428] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T19:47:56.251397] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-06. [2025-10-20T19:47:56.874006] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T19:47:57.051525] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-07. [2025-10-20T19:47:57.085374] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T19:47:57.851436] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-08. [2025-10-20T19:47:58.479376] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T19:47:58.651400] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-08. [2025-10-20T19:47:59.451380] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-04. [2025-10-20T19:48:00.251366] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-07. [2025-10-20T19:48:00.432795] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-08 (2.58 seconds). [2025-10-20T19:48:00.988971] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-08 (2.34 seconds). [2025-10-20T19:48:01.051362] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-09. [2025-10-20T19:48:01.644906] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T19:48:01.851347] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-05. [2025-10-20T19:48:02.651350] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-09. [2025-10-20T19:48:02.800915] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-07 (2.55 seconds). [2025-10-20T19:48:03.089243] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:48:03.181221] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-09 (2.13 seconds). [2025-10-20T19:48:03.451369] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-08. [2025-10-20T19:48:04.251400] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-10. [2025-10-20T19:48:04.642823] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T19:48:04.762561] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:48:05.051374] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-06. [2025-10-20T19:48:05.651724] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:48:05.656966] [INFO ] [REMOTE ] MATV: Full fetch complete (52266 records). [2025-10-20T19:48:05.657076] [DEBUG] [VALIDATOR ] MATV: Validating primary data. [2025-10-20T19:48:05.856538] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-10. [2025-10-20T19:48:06.097305] [DEBUG] [VALIDATOR ] MATV: All expected trading days are present. [2025-10-20T19:48:06.097386] [DEBUG] [VALIDATOR ] MATV: Primary data validation passed. [2025-10-20T19:48:06.098262] [INFO ] [MAIN ] MATV: Fetched primary data is valid. [2025-10-20T19:48:06.098326] [DEBUG] [LOCAL ] MATV: Saving primary data. [2025-10-20T19:48:06.099484] [DEBUG] [LOCAL ] MATV: Trimmed 52266 records to 35365 (keeping 200 days of data) [2025-10-20T19:48:06.205770] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T19:48:06.332216] [DEBUG] [LOCAL ] MATV: Primary data saved (35365 records). [2025-10-20T19:48:06.332348] [INFO ] [MAIN ] MATV: Full history saved. [2025-10-20T19:48:06.332424] [INFO ] [MAIN ] Update Progress: 1302/2367 symbols completed. MATV took 26.7s. Estimated remaining: 2h 17m 27s. Rate: 7.7 symbols/min [2025-10-20T19:48:06.499601] [INFO ] [MAIN ] MBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:06.540691] [DEBUG] [LOCAL ] MBC: Loaded 44551 existing records from primary storage. [2025-10-20T19:48:06.540986] [INFO ] [MAIN ] MBC: Loaded 44551 existing records. [2025-10-20T19:48:06.656442] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-09. [2025-10-20T19:48:06.716463] [WARN ] [MAIN ] MBC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:06.716667] [INFO ] [MAIN ] MBC: Wiping existing primary data before full refresh. [2025-10-20T19:48:06.716727] [INFO ] [LOCAL ] MBC: Starting deletion of primary data. [2025-10-20T19:48:06.716911] [DEBUG] [LOCAL ] MBC: Deleted primary data file. [2025-10-20T19:48:06.716948] [INFO ] [MAIN ] MBC: Starting full primary refresh. [2025-10-20T19:48:07.063205] [INFO ] [REMOTE ] MBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:07.451377] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-01. [2025-10-20T19:48:07.783827] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-10-20T19:48:07.791620] [INFO ] [REMOTE ] MAS: Full fetch complete (75351 records). [2025-10-20T19:48:07.791781] [DEBUG] [VALIDATOR ] MAS: Validating primary data. [2025-10-20T19:48:08.012140] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:48:08.261044] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-07. [2025-10-20T19:48:08.289751] [DEBUG] [VALIDATOR ] MAS: All expected trading days are present. [2025-10-20T19:48:08.289883] [DEBUG] [VALIDATOR ] MAS: Primary data validation passed. [2025-10-20T19:48:08.291952] [INFO ] [MAIN ] MAS: Fetched primary data is valid. [2025-10-20T19:48:08.292058] [DEBUG] [LOCAL ] MAS: Saving primary data. [2025-10-20T19:48:08.294625] [DEBUG] [LOCAL ] MAS: Trimmed 75351 records to 52281 (keeping 200 days of data) [2025-10-20T19:48:08.653268] [DEBUG] [LOCAL ] MAS: Primary data saved (52281 records). [2025-10-20T19:48:08.653473] [INFO ] [MAIN ] MAS: Full history saved. [2025-10-20T19:48:08.653588] [INFO ] [MAIN ] Update Progress: 1303/2367 symbols completed. MAS took 37.0s. Estimated remaining: 2h 17m 15s. Rate: 7.8 symbols/min [2025-10-20T19:48:08.844827] [INFO ] [MAIN ] MBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:08.869223] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:48:08.901309] [DEBUG] [LOCAL ] MBI: Loaded 24689 existing records from primary storage. [2025-10-20T19:48:08.901708] [INFO ] [MAIN ] MBI: Loaded 24689 existing records. [2025-10-20T19:48:09.054571] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-10. [2025-10-20T19:48:09.147727] [WARN ] [MAIN ] MBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:09.148080] [INFO ] [MAIN ] MBI: Wiping existing primary data before full refresh. [2025-10-20T19:48:09.148136] [INFO ] [LOCAL ] MBI: Starting deletion of primary data. [2025-10-20T19:48:09.148415] [DEBUG] [LOCAL ] MBI: Deleted primary data file. [2025-10-20T19:48:09.148471] [INFO ] [MAIN ] MBI: Starting full primary refresh. [2025-10-20T19:48:09.534444] [INFO ] [REMOTE ] MBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:09.551724] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T19:48:09.851359] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-01. [2025-10-20T19:48:10.318055] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T19:48:10.322906] [INFO ] [REMOTE ] MATX: Full fetch complete (43895 records). [2025-10-20T19:48:10.322980] [DEBUG] [VALIDATOR ] MATX: Validating primary data. [2025-10-20T19:48:10.659586] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-08. [2025-10-20T19:48:10.692282] [DEBUG] [VALIDATOR ] MATX: All expected trading days are present. [2025-10-20T19:48:10.692416] [DEBUG] [VALIDATOR ] MATX: Primary data validation passed. [2025-10-20T19:48:10.693190] [INFO ] [MAIN ] MATX: Fetched primary data is valid. [2025-10-20T19:48:10.693263] [DEBUG] [LOCAL ] MATX: Saving primary data. [2025-10-20T19:48:10.694582] [DEBUG] [LOCAL ] MATX: Trimmed 43895 records to 32025 (keeping 200 days of data) [2025-10-20T19:48:10.891903] [DEBUG] [LOCAL ] MATX: Primary data saved (32025 records). [2025-10-20T19:48:10.892055] [INFO ] [MAIN ] MATX: Full history saved. [2025-10-20T19:48:10.892172] [INFO ] [MAIN ] Update Progress: 1304/2367 symbols completed. MATX took 28.3s. Estimated remaining: 2h 17m 2s. Rate: 7.8 symbols/min [2025-10-20T19:48:11.034151] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T19:48:11.081302] [INFO ] [MAIN ] MC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:11.120368] [DEBUG] [LOCAL ] MC: Loaded 44544 existing records from primary storage. [2025-10-20T19:48:11.120675] [INFO ] [MAIN ] MC: Loaded 44544 existing records. [2025-10-20T19:48:11.291975] [WARN ] [MAIN ] MC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:11.292181] [INFO ] [MAIN ] MC: Wiping existing primary data before full refresh. [2025-10-20T19:48:11.292227] [INFO ] [LOCAL ] MC: Starting deletion of primary data. [2025-10-20T19:48:11.292425] [DEBUG] [LOCAL ] MC: Deleted primary data file. [2025-10-20T19:48:11.292481] [INFO ] [MAIN ] MC: Starting full primary refresh. [2025-10-20T19:48:11.459574] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-02. [2025-10-20T19:48:11.643661] [INFO ] [REMOTE ] MC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:12.041957] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T19:48:12.251380] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-09. [2025-10-20T19:48:13.051398] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-02. [2025-10-20T19:48:13.114423] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:48:13.710879] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:48:13.851383] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-10. [2025-10-20T19:48:14.174533] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T19:48:14.651438] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-01. [2025-10-20T19:48:15.195582] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T19:48:15.201154] [INFO ] [REMOTE ] MAX: Full fetch complete (45363 records). [2025-10-20T19:48:15.201301] [DEBUG] [VALIDATOR ] MAX: Validating primary data. [2025-10-20T19:48:15.456624] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-03. [2025-10-20T19:48:15.602054] [DEBUG] [VALIDATOR ] MAX: All expected trading days are present. [2025-10-20T19:48:15.602134] [DEBUG] [VALIDATOR ] MAX: Primary data validation passed. [2025-10-20T19:48:15.602876] [INFO ] [MAIN ] MAX: Fetched primary data is valid. [2025-10-20T19:48:15.602939] [DEBUG] [LOCAL ] MAX: Saving primary data. [2025-10-20T19:48:15.603897] [DEBUG] [LOCAL ] MAX: Trimmed 45363 records to 30945 (keeping 200 days of data) [2025-10-20T19:48:15.781003] [DEBUG] [LOCAL ] MAX: Primary data saved (30945 records). [2025-10-20T19:48:15.781227] [INFO ] [MAIN ] MAX: Full history saved. [2025-10-20T19:48:15.781627] [INFO ] [MAIN ] Update Progress: 1305/2367 symbols completed. MAX took 31.1s. Estimated remaining: 2h 16m 52s. Rate: 7.8 symbols/min [2025-10-20T19:48:15.957494] [INFO ] [MAIN ] MCB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:15.971074] [DEBUG] [LOCAL ] MCB: Loaded 11549 existing records from primary storage. [2025-10-20T19:48:15.971315] [INFO ] [MAIN ] MCB: Loaded 11549 existing records. [2025-10-20T19:48:16.164445] [WARN ] [MAIN ] MCB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:16.164700] [INFO ] [MAIN ] MCB: Wiping existing primary data before full refresh. [2025-10-20T19:48:16.164771] [INFO ] [LOCAL ] MCB: Starting deletion of primary data. [2025-10-20T19:48:16.172830] [DEBUG] [LOCAL ] MCB: Deleted primary data file. [2025-10-20T19:48:16.185732] [INFO ] [MAIN ] MCB: Starting full primary refresh. [2025-10-20T19:48:16.220224] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T19:48:16.253160] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-02. [2025-10-20T19:48:16.569227] [INFO ] [REMOTE ] MCB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:16.808960] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T19:48:17.051399] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-04. [2025-10-20T19:48:17.857696] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-01. [2025-10-20T19:48:17.862600] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T19:48:18.204195] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:48:18.651397] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-03. [2025-10-20T19:48:18.703141] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T19:48:19.451408] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-02. [2025-10-20T19:48:20.231539] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:48:20.254202] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-03. [2025-10-20T19:48:20.265984] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T19:48:21.051400] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-05. [2025-10-20T19:48:21.746527] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:48:21.856448] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-03. [2025-10-20T19:48:21.865686] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T19:48:22.651354] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-06. [2025-10-20T19:48:22.730839] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:48:23.451400] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-04. [2025-10-20T19:48:23.733562] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T19:48:24.251402] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-07. [2025-10-20T19:48:24.965204] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:48:25.051370] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-04. [2025-10-20T19:48:25.370613] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T19:48:25.851395] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-08. [2025-10-20T19:48:26.657961] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-05. [2025-10-20T19:48:26.663994] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:48:27.213208] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T19:48:27.451403] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-09. [2025-10-20T19:48:28.108952] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:48:28.251348] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-04. [2025-10-20T19:48:28.544112] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T19:48:29.051343] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-06. [2025-10-20T19:48:29.314683] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T19:48:29.851382] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-05. [2025-10-20T19:48:30.445615] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T19:48:30.653981] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-05. [2025-10-20T19:48:30.655076] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T19:48:31.451419] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-10. [2025-10-20T19:48:32.254280] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-07. [2025-10-20T19:48:32.259972] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:48:32.576382] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T19:48:32.580210] [INFO ] [REMOTE ] MBI: Full fetch complete (36199 records). [2025-10-20T19:48:32.580322] [DEBUG] [VALIDATOR ] MBI: Validating primary data. [2025-10-20T19:48:32.953185] [DEBUG] [VALIDATOR ] MBI: All expected trading days are present. [2025-10-20T19:48:32.953269] [DEBUG] [VALIDATOR ] MBI: Primary data validation passed. [2025-10-20T19:48:32.953959] [INFO ] [MAIN ] MBI: Fetched primary data is valid. [2025-10-20T19:48:32.954023] [DEBUG] [LOCAL ] MBI: Saving primary data. [2025-10-20T19:48:32.955059] [DEBUG] [LOCAL ] MBI: Trimmed 36199 records to 24304 (keeping 200 days of data) [2025-10-20T19:48:33.052277] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-06. [2025-10-20T19:48:33.085953] [DEBUG] [LOCAL ] MBI: Primary data saved (24304 records). [2025-10-20T19:48:33.086057] [INFO ] [MAIN ] MBI: Full history saved. [2025-10-20T19:48:33.086128] [INFO ] [MAIN ] Update Progress: 1306/2367 symbols completed. MBI took 24.4s. Estimated remaining: 2h 16m 52s. Rate: 7.8 symbols/min [2025-10-20T19:48:33.251253] [INFO ] [MAIN ] MCD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:33.251385] [DEBUG] [LOCAL ] MCD: No existing data file found in primary storage. [2025-10-20T19:48:33.251423] [WARN ] [MAIN ] MCD: No existing data found, performing full refresh instead. [2025-10-20T19:48:33.251529] [INFO ] [MAIN ] MCD: Wiping existing primary data before full refresh. [2025-10-20T19:48:33.251573] [INFO ] [LOCAL ] MCD: Starting deletion of primary data. [2025-10-20T19:48:33.251682] [DEBUG] [LOCAL ] MCD: No primary data file found to delete. [2025-10-20T19:48:33.251747] [INFO ] [MAIN ] MCD: Starting full primary refresh. [2025-10-20T19:48:33.607763] [INFO ] [REMOTE ] MCD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:33.692197] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T19:48:33.851343] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-01. [2025-10-20T19:48:34.546204] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T19:48:34.651380] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-07. [2025-10-20T19:48:34.754716] [DEBUG] [REMOTE ] MC: Error fetching primary data for 2025-07: 'API call returned an empty response.' [2025-10-20T19:48:34.755037] [WARN ] [REMOTE ] MC: No data returned for month 2025-07 [2025-10-20T19:48:34.755128] [FATAL] [REMOTE ] MC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:48:34.755204] [FATAL] [MAIN ] MC: Failed to fetch primary data. [2025-10-20T19:48:34.755240] [WARN ] [MAIN ] MC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:48:34.755277] [INFO ] [LOCAL ] MC: Starting deletion of primary data. [2025-10-20T19:48:34.755330] [DEBUG] [LOCAL ] MC: No primary data file found to delete. [2025-10-20T19:48:34.755364] [INFO ] [LOCAL ] MC: Starting deletion of derivative data. [2025-10-20T19:48:34.755528] [DEBUG] [LOCAL ] MC: Deleted derivative data file. [2025-10-20T19:48:34.755607] [INFO ] [MAIN ] Update Progress: 1307/2367 symbols completed. MC took 23.9s. Estimated remaining: 2h 16m 40s. Rate: 7.8 symbols/min [2025-10-20T19:48:34.925968] [INFO ] [MAIN ] MCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:34.934508] [DEBUG] [LOCAL ] MCI: Loaded 6457 existing records from primary storage. [2025-10-20T19:48:34.934713] [INFO ] [MAIN ] MCI: Loaded 6457 existing records. [2025-10-20T19:48:35.108378] [WARN ] [MAIN ] MCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:35.108561] [INFO ] [MAIN ] MCI: Wiping existing primary data before full refresh. [2025-10-20T19:48:35.108614] [INFO ] [LOCAL ] MCI: Starting deletion of primary data. [2025-10-20T19:48:35.108771] [DEBUG] [LOCAL ] MCI: Deleted primary data file. [2025-10-20T19:48:35.108822] [INFO ] [MAIN ] MCI: Starting full primary refresh. [2025-10-20T19:48:35.453783] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-06. [2025-10-20T19:48:35.453706] [INFO ] [REMOTE ] MCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:35.897854] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T19:48:36.167398] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T19:48:36.251467] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-02. [2025-10-20T19:48:37.051374] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-07. [2025-10-20T19:48:37.851354] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-08. [2025-10-20T19:48:37.923241] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T19:48:38.109785] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T19:48:38.651391] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-01. [2025-10-20T19:48:39.149771] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T19:48:39.386178] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:48:39.451337] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-02. [2025-10-20T19:48:40.251392] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-08. [2025-10-20T19:48:40.271774] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T19:48:40.957984] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T19:48:41.051397] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-03. [2025-10-20T19:48:41.784571] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T19:48:41.851342] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-09. [2025-10-20T19:48:42.651374] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-03. [2025-10-20T19:48:43.371856] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:48:43.451415] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-10. [2025-10-20T19:48:44.251345] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-09. [2025-10-20T19:48:44.776017] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-03 (2.12 seconds). [2025-10-20T19:48:44.843996] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:48:45.051380] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-10. [2025-10-20T19:48:45.070904] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T19:48:45.078056] [INFO ] [REMOTE ] MBC: Full fetch complete (63127 records). [2025-10-20T19:48:45.078160] [DEBUG] [VALIDATOR ] MBC: Validating primary data. [2025-10-20T19:48:45.457330] [DEBUG] [VALIDATOR ] MBC: All expected trading days are present. [2025-10-20T19:48:45.457417] [DEBUG] [VALIDATOR ] MBC: Primary data validation passed. [2025-10-20T19:48:45.458315] [INFO ] [MAIN ] MBC: Fetched primary data is valid. [2025-10-20T19:48:45.458380] [DEBUG] [LOCAL ] MBC: Saving primary data. [2025-10-20T19:48:45.462925] [DEBUG] [LOCAL ] MBC: Trimmed 63127 records to 43967 (keeping 200 days of data) [2025-10-20T19:48:45.713605] [DEBUG] [LOCAL ] MBC: Primary data saved (43967 records). [2025-10-20T19:48:45.713749] [INFO ] [MAIN ] MBC: Full history saved. [2025-10-20T19:48:45.713828] [INFO ] [MAIN ] Update Progress: 1308/2367 symbols completed. MBC took 39.4s. Estimated remaining: 2h 16m 35s. Rate: 7.8 symbols/min [2025-10-20T19:48:45.860966] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-04. [2025-10-20T19:48:45.888405] [INFO ] [MAIN ] MCK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:45.926242] [DEBUG] [LOCAL ] MCK: Loaded 39064 existing records from primary storage. [2025-10-20T19:48:45.926548] [INFO ] [MAIN ] MCK: Loaded 39064 existing records. [2025-10-20T19:48:46.058383] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:48:46.101245] [INFO ] [REMOTE ] MCB: Full fetch complete (16641 records). [2025-10-20T19:48:46.117444] [DEBUG] [VALIDATOR ] MCB: Validating primary data. [2025-10-20T19:48:46.121214] [WARN ] [MAIN ] MCK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:46.121545] [INFO ] [MAIN ] MCK: Wiping existing primary data before full refresh. [2025-10-20T19:48:46.121683] [INFO ] [LOCAL ] MCK: Starting deletion of primary data. [2025-10-20T19:48:46.206561] [DEBUG] [LOCAL ] MCK: Deleted primary data file. [2025-10-20T19:48:46.229930] [INFO ] [MAIN ] MCK: Starting full primary refresh. [2025-10-20T19:48:46.669356] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-04. [2025-10-20T19:48:46.839079] [DEBUG] [VALIDATOR ] MCB: All expected trading days are present. [2025-10-20T19:48:46.869994] [DEBUG] [VALIDATOR ] MCB: Primary data validation passed. [2025-10-20T19:48:46.881216] [INFO ] [MAIN ] MCB: Fetched primary data is valid. [2025-10-20T19:48:46.894703] [DEBUG] [LOCAL ] MCB: Saving primary data. [2025-10-20T19:48:46.894615] [INFO ] [REMOTE ] MCK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:46.895472] [DEBUG] [LOCAL ] MCB: Trimmed 16641 records to 11359 (keeping 200 days of data) [2025-10-20T19:48:46.963736] [DEBUG] [LOCAL ] MCB: Primary data saved (11359 records). [2025-10-20T19:48:46.963882] [INFO ] [MAIN ] MCB: Full history saved. [2025-10-20T19:48:46.963963] [INFO ] [MAIN ] Update Progress: 1309/2367 symbols completed. MCB took 31.2s. Estimated remaining: 2h 16m 22s. Rate: 7.8 symbols/min [2025-10-20T19:48:47.140249] [INFO ] [MAIN ] MCN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:47.150419] [DEBUG] [LOCAL ] MCN: Loaded 8247 existing records from primary storage. [2025-10-20T19:48:47.150634] [INFO ] [MAIN ] MCN: Loaded 8247 existing records. [2025-10-20T19:48:47.338281] [WARN ] [MAIN ] MCN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:47.338584] [INFO ] [MAIN ] MCN: Wiping existing primary data before full refresh. [2025-10-20T19:48:47.338637] [INFO ] [LOCAL ] MCN: Starting deletion of primary data. [2025-10-20T19:48:47.338839] [DEBUG] [LOCAL ] MCN: Deleted primary data file. [2025-10-20T19:48:47.338896] [INFO ] [MAIN ] MCN: Starting full primary refresh. [2025-10-20T19:48:47.444115] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T19:48:47.454672] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-01. [2025-10-20T19:48:47.701477] [INFO ] [REMOTE ] MCN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:48.047996] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T19:48:48.251421] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-01. [2025-10-20T19:48:49.030152] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T19:48:49.051352] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-02. [2025-10-20T19:48:49.127318] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T19:48:49.760406] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T19:48:49.851429] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-05. [2025-10-20T19:48:50.651426] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-02. [2025-10-20T19:48:51.451351] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-05. [2025-10-20T19:48:51.845844] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T19:48:52.090757] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T19:48:52.163631] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T19:48:52.251334] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-03. [2025-10-20T19:48:52.973986] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T19:48:53.051396] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-06. [2025-10-20T19:48:53.851363] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-04. [2025-10-20T19:48:54.651416] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-03. [2025-10-20T19:48:54.678632] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T19:48:55.001834] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T19:48:55.451371] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-07. [2025-10-20T19:48:56.096959] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T19:48:56.251346] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-04. [2025-10-20T19:48:57.051409] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-05. [2025-10-20T19:48:57.794285] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-07 (2.34 seconds). [2025-10-20T19:48:57.848067] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T19:48:57.854178] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-06. [2025-10-20T19:48:57.865203] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T19:48:58.651405] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-06. [2025-10-20T19:48:58.669938] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T19:48:59.450564] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T19:48:59.453813] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-07. [2025-10-20T19:49:00.115319] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T19:49:00.251364] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-05. [2025-10-20T19:49:01.051439] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-07. [2025-10-20T19:49:01.851396] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-08. [2025-10-20T19:49:01.953541] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:49:02.358913] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-10-20T19:49:02.658449] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-09. [2025-10-20T19:49:02.665438] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-05 (2.41 seconds). [2025-10-20T19:49:03.374377] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T19:49:03.451412] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-10. [2025-10-20T19:49:04.183205] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T19:49:04.184661] [INFO ] [REMOTE ] MCI: Full fetch complete (9896 records). [2025-10-20T19:49:04.184752] [DEBUG] [VALIDATOR ] MCI: Validating primary data. [2025-10-20T19:49:04.256613] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-08. [2025-10-20T19:49:04.557794] [DEBUG] [VALIDATOR ] MCI: All expected trading days are present. [2025-10-20T19:49:04.557867] [DEBUG] [VALIDATOR ] MCI: Primary data validation passed. [2025-10-20T19:49:04.558116] [INFO ] [MAIN ] MCI: Fetched primary data is valid. [2025-10-20T19:49:04.558155] [DEBUG] [LOCAL ] MCI: Saving primary data. [2025-10-20T19:49:04.558593] [DEBUG] [LOCAL ] MCI: Trimmed 9896 records to 6235 (keeping 200 days of data) [2025-10-20T19:49:04.594836] [DEBUG] [LOCAL ] MCI: Primary data saved (6235 records). [2025-10-20T19:49:04.594921] [INFO ] [MAIN ] MCI: Full history saved. [2025-10-20T19:49:04.594993] [INFO ] [MAIN ] Update Progress: 1310/2367 symbols completed. MCI took 29.8s. Estimated remaining: 2h 16m 22s. Rate: 7.8 symbols/min [2025-10-20T19:49:04.764012] [INFO ] [MAIN ] MCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:04.801308] [DEBUG] [LOCAL ] MCO: Loaded 40089 existing records from primary storage. [2025-10-20T19:49:04.801608] [INFO ] [MAIN ] MCO: Loaded 40089 existing records. [2025-10-20T19:49:04.977151] [WARN ] [MAIN ] MCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:04.977365] [INFO ] [MAIN ] MCO: Wiping existing primary data before full refresh. [2025-10-20T19:49:04.977415] [INFO ] [LOCAL ] MCO: Starting deletion of primary data. [2025-10-20T19:49:04.977608] [DEBUG] [LOCAL ] MCO: Deleted primary data file. [2025-10-20T19:49:04.977702] [INFO ] [MAIN ] MCO: Starting full primary refresh. [2025-10-20T19:49:05.056561] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-08. [2025-10-20T19:49:05.357536] [INFO ] [REMOTE ] MCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:05.851375] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-06. [2025-10-20T19:49:05.880829] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T19:49:06.446488] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-08 (2.19 seconds). [2025-10-20T19:49:06.651611] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-01. [2025-10-20T19:49:07.304841] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T19:49:07.451405] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-09. [2025-10-20T19:49:08.251374] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-07. [2025-10-20T19:49:08.338765] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:49:09.051474] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-02. [2025-10-20T19:49:09.475091] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T19:49:09.488549] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T19:49:09.851416] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-09. [2025-10-20T19:49:10.305097] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T19:49:10.587805] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:49:10.651382] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-10. [2025-10-20T19:49:11.451339] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-03. [2025-10-20T19:49:11.621740] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T19:49:11.629447] [INFO ] [REMOTE ] MCD: Full fetch complete (77148 records). [2025-10-20T19:49:11.629565] [DEBUG] [VALIDATOR ] MCD: Validating primary data. [2025-10-20T19:49:11.821187] [WARN ] [VALIDATOR ] MCD: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:49:11.821950] [FATAL] [MAIN ] MCD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:49:11.822031] [WARN ] [MAIN ] MCD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:49:11.822100] [INFO ] [LOCAL ] MCD: Starting deletion of primary data. [2025-10-20T19:49:11.822162] [DEBUG] [LOCAL ] MCD: No primary data file found to delete. [2025-10-20T19:49:11.822196] [INFO ] [LOCAL ] MCD: Starting deletion of derivative data. [2025-10-20T19:49:11.822234] [DEBUG] [LOCAL ] MCD: No derivative data file found to delete. [2025-10-20T19:49:11.822289] [INFO ] [MAIN ] Update Progress: 1311/2367 symbols completed. MCD took 38.7s. Estimated remaining: 2h 16m 14s. Rate: 7.8 symbols/min [2025-10-20T19:49:11.989147] [INFO ] [MAIN ] MCR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:11.999344] [DEBUG] [LOCAL ] MCR: Loaded 9061 existing records from primary storage. [2025-10-20T19:49:11.999544] [INFO ] [MAIN ] MCR: Loaded 9061 existing records. [2025-10-20T19:49:12.177296] [WARN ] [MAIN ] MCR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:12.177480] [INFO ] [MAIN ] MCR: Wiping existing primary data before full refresh. [2025-10-20T19:49:12.177527] [INFO ] [LOCAL ] MCR: Starting deletion of primary data. [2025-10-20T19:49:12.177753] [DEBUG] [LOCAL ] MCR: Deleted primary data file. [2025-10-20T19:49:12.177812] [INFO ] [MAIN ] MCR: Starting full primary refresh. [2025-10-20T19:49:12.256631] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-10. [2025-10-20T19:49:12.547229] [INFO ] [REMOTE ] MCR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:13.051397] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-08. [2025-10-20T19:49:13.053880] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:49:13.068554] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:49:13.070078] [INFO ] [REMOTE ] MCN: Full fetch complete (11580 records). [2025-10-20T19:49:13.070149] [DEBUG] [VALIDATOR ] MCN: Validating primary data. [2025-10-20T19:49:13.430942] [DEBUG] [VALIDATOR ] MCN: All expected trading days are present. [2025-10-20T19:49:13.431017] [DEBUG] [VALIDATOR ] MCN: Primary data validation passed. [2025-10-20T19:49:13.431273] [INFO ] [MAIN ] MCN: Fetched primary data is valid. [2025-10-20T19:49:13.431310] [DEBUG] [LOCAL ] MCN: Saving primary data. [2025-10-20T19:49:13.431874] [DEBUG] [LOCAL ] MCN: Trimmed 11580 records to 8069 (keeping 200 days of data) [2025-10-20T19:49:13.478188] [DEBUG] [LOCAL ] MCN: Primary data saved (8069 records). [2025-10-20T19:49:13.478320] [INFO ] [MAIN ] MCN: Full history saved. [2025-10-20T19:49:13.478398] [INFO ] [MAIN ] Update Progress: 1312/2367 symbols completed. MCN took 26.5s. Estimated remaining: 2h 16m 1s. Rate: 7.8 symbols/min [2025-10-20T19:49:13.646319] [INFO ] [MAIN ] MCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:13.667398] [DEBUG] [LOCAL ] MCS: Loaded 21467 existing records from primary storage. [2025-10-20T19:49:13.667633] [INFO ] [MAIN ] MCS: Loaded 21467 existing records. [2025-10-20T19:49:13.846162] [WARN ] [MAIN ] MCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:13.846366] [INFO ] [MAIN ] MCS: Wiping existing primary data before full refresh. [2025-10-20T19:49:13.846411] [INFO ] [LOCAL ] MCS: Starting deletion of primary data. [2025-10-20T19:49:13.846607] [DEBUG] [LOCAL ] MCS: Deleted primary data file. [2025-10-20T19:49:13.846693] [INFO ] [MAIN ] MCS: Starting full primary refresh. [2025-10-20T19:49:13.857872] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-01. [2025-10-20T19:49:14.215700] [INFO ] [REMOTE ] MCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:14.651355] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-04. [2025-10-20T19:49:14.742079] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T19:49:14.802793] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T19:49:15.451440] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-02. [2025-10-20T19:49:16.192453] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T19:49:16.251344] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-03. [2025-10-20T19:49:16.428794] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T19:49:16.709593] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T19:49:17.051410] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-05. [2025-10-20T19:49:17.851367] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-09. [2025-10-20T19:49:18.591094] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T19:49:18.651377] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-01. [2025-10-20T19:49:19.396457] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T19:49:19.451353] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-10. [2025-10-20T19:49:19.713607] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T19:49:20.251394] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-06. [2025-10-20T19:49:20.496978] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T19:49:20.502189] [INFO ] [REMOTE ] MCK: Full fetch complete (56728 records). [2025-10-20T19:49:20.502335] [DEBUG] [VALIDATOR ] MCK: Validating primary data. [2025-10-20T19:49:20.894421] [DEBUG] [VALIDATOR ] MCK: All expected trading days are present. [2025-10-20T19:49:20.894512] [DEBUG] [VALIDATOR ] MCK: Primary data validation passed. [2025-10-20T19:49:20.895370] [INFO ] [MAIN ] MCK: Fetched primary data is valid. [2025-10-20T19:49:20.895435] [DEBUG] [LOCAL ] MCK: Saving primary data. [2025-10-20T19:49:20.896706] [DEBUG] [LOCAL ] MCK: Trimmed 56728 records to 38322 (keeping 200 days of data) [2025-10-20T19:49:21.071939] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-02. [2025-10-20T19:49:21.140196] [DEBUG] [LOCAL ] MCK: Primary data saved (38322 records). [2025-10-20T19:49:21.140343] [INFO ] [MAIN ] MCK: Full history saved. [2025-10-20T19:49:21.140426] [INFO ] [MAIN ] Update Progress: 1313/2367 symbols completed. MCK took 35.4s. Estimated remaining: 2h 15m 53s. Rate: 7.8 symbols/min [2025-10-20T19:49:21.307105] [INFO ] [MAIN ] MCY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:21.329549] [DEBUG] [LOCAL ] MCY: Loaded 24248 existing records from primary storage. [2025-10-20T19:49:21.329821] [INFO ] [MAIN ] MCY: Loaded 24248 existing records. [2025-10-20T19:49:21.511846] [WARN ] [MAIN ] MCY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:21.519349] [INFO ] [MAIN ] MCY: Wiping existing primary data before full refresh. [2025-10-20T19:49:21.519501] [INFO ] [LOCAL ] MCY: Starting deletion of primary data. [2025-10-20T19:49:21.532462] [DEBUG] [LOCAL ] MCY: Deleted primary data file. [2025-10-20T19:49:21.532926] [INFO ] [MAIN ] MCY: Starting full primary refresh. [2025-10-20T19:49:21.533538] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T19:49:21.854318] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-07. [2025-10-20T19:49:21.894884] [INFO ] [REMOTE ] MCY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:22.075540] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T19:49:22.651369] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-01. [2025-10-20T19:49:23.451358] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-03. [2025-10-20T19:49:23.478091] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T19:49:24.255323] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-04. [2025-10-20T19:49:24.278148] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:49:24.573850] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:49:24.777312] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-10-20T19:49:25.051371] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-02. [2025-10-20T19:49:25.851385] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-08. [2025-10-20T19:49:26.353041] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:49:26.651360] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-04. [2025-10-20T19:49:27.264379] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T19:49:27.451427] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-05. [2025-10-20T19:49:27.759099] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:49:28.155396] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T19:49:28.251369] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-06. [2025-10-20T19:49:28.938414] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T19:49:29.051373] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-07. [2025-10-20T19:49:29.768759] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T19:49:29.851399] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-03. [2025-10-20T19:49:30.651377] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-05. [2025-10-20T19:49:31.155510] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:49:31.451377] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-04. [2025-10-20T19:49:31.731048] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:49:32.251394] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-09. [2025-10-20T19:49:32.601201] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:49:33.051368] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-06. [2025-10-20T19:49:33.470584] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T19:49:33.851379] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-08. [2025-10-20T19:49:34.004587] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T19:49:34.651393] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-07. [2025-10-20T19:49:34.668995] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T19:49:35.451389] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-10. [2025-10-20T19:49:35.555751] [DEBUG] [REMOTE ] MCO: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T19:49:35.556041] [WARN ] [REMOTE ] MCO: No data returned for month 2025-10 [2025-10-20T19:49:35.556162] [FATAL] [REMOTE ] MCO: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:49:35.556257] [FATAL] [MAIN ] MCO: Failed to fetch primary data. [2025-10-20T19:49:35.556334] [WARN ] [MAIN ] MCO: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:49:35.556420] [INFO ] [LOCAL ] MCO: Starting deletion of primary data. [2025-10-20T19:49:35.556533] [DEBUG] [LOCAL ] MCO: No primary data file found to delete. [2025-10-20T19:49:35.556590] [INFO ] [LOCAL ] MCO: Starting deletion of derivative data. [2025-10-20T19:49:35.556842] [DEBUG] [LOCAL ] MCO: Deleted derivative data file. [2025-10-20T19:49:35.556941] [INFO ] [MAIN ] Update Progress: 1314/2367 symbols completed. MCO took 31.0s. Estimated remaining: 2h 15m 51s. Rate: 7.8 symbols/min [2025-10-20T19:49:35.748227] [INFO ] [MAIN ] MD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:35.782384] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T19:49:35.801229] [DEBUG] [LOCAL ] MD: Loaded 39102 existing records from primary storage. [2025-10-20T19:49:35.801527] [INFO ] [MAIN ] MD: Loaded 39102 existing records. [2025-10-20T19:49:35.974971] [WARN ] [MAIN ] MD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:35.975161] [INFO ] [MAIN ] MD: Wiping existing primary data before full refresh. [2025-10-20T19:49:35.975208] [INFO ] [LOCAL ] MD: Starting deletion of primary data. [2025-10-20T19:49:35.975430] [DEBUG] [LOCAL ] MD: Deleted primary data file. [2025-10-20T19:49:35.975485] [INFO ] [MAIN ] MD: Starting full primary refresh. [2025-10-20T19:49:36.256618] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-08. [2025-10-20T19:49:36.321665] [INFO ] [REMOTE ] MD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:37.051360] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-09. [2025-10-20T19:49:37.306207] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:49:37.637914] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:49:37.851385] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-01. [2025-10-20T19:49:38.651371] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-05. [2025-10-20T19:49:39.186858] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T19:49:39.451377] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-02. [2025-10-20T19:49:39.892499] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:49:40.251432] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-10. [2025-10-20T19:49:40.758970] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T19:49:40.943084] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T19:49:40.944935] [INFO ] [REMOTE ] MCR: Full fetch complete (13535 records). [2025-10-20T19:49:40.945005] [DEBUG] [VALIDATOR ] MCR: Validating primary data. [2025-10-20T19:49:41.056472] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-06. [2025-10-20T19:49:41.304068] [DEBUG] [VALIDATOR ] MCR: All expected trading days are present. [2025-10-20T19:49:41.304143] [DEBUG] [VALIDATOR ] MCR: Primary data validation passed. [2025-10-20T19:49:41.304462] [INFO ] [MAIN ] MCR: Fetched primary data is valid. [2025-10-20T19:49:41.304501] [DEBUG] [LOCAL ] MCR: Saving primary data. [2025-10-20T19:49:41.305018] [DEBUG] [LOCAL ] MCR: Trimmed 13535 records to 8879 (keeping 200 days of data) [2025-10-20T19:49:41.354993] [DEBUG] [LOCAL ] MCR: Primary data saved (8879 records). [2025-10-20T19:49:41.355120] [INFO ] [MAIN ] MCR: Full history saved. [2025-10-20T19:49:41.355192] [INFO ] [MAIN ] Update Progress: 1315/2367 symbols completed. MCR took 29.5s. Estimated remaining: 2h 15m 42s. Rate: 7.8 symbols/min [2025-10-20T19:49:41.520119] [INFO ] [MAIN ] MDST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:41.529021] [DEBUG] [LOCAL ] MDST: Loaded 6124 existing records from primary storage. [2025-10-20T19:49:41.529224] [INFO ] [MAIN ] MDST: Loaded 6124 existing records. [2025-10-20T19:49:41.703372] [WARN ] [MAIN ] MDST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:41.703595] [INFO ] [MAIN ] MDST: Wiping existing primary data before full refresh. [2025-10-20T19:49:41.703678] [INFO ] [LOCAL ] MDST: Starting deletion of primary data. [2025-10-20T19:49:41.703884] [DEBUG] [LOCAL ] MDST: Deleted primary data file. [2025-10-20T19:49:41.703939] [INFO ] [MAIN ] MDST: Starting full primary refresh. [2025-10-20T19:49:41.856457] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-03. [2025-10-20T19:49:42.072753] [INFO ] [REMOTE ] MDST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:42.115890] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T19:49:42.651361] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-01. [2025-10-20T19:49:43.048391] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T19:49:43.301049] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T19:49:43.451514] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-02. [2025-10-20T19:49:44.066239] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T19:49:44.251423] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-07. [2025-10-20T19:49:45.051406] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-04. [2025-10-20T19:49:45.389236] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:49:45.851393] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-03. [2025-10-20T19:49:46.472186] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T19:49:46.521715] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T19:49:46.651376] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-04. [2025-10-20T19:49:47.218829] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T19:49:47.451405] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-05. [2025-10-20T19:49:47.983136] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T19:49:48.251388] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-05. [2025-10-20T19:49:49.051417] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-09. [2025-10-20T19:49:49.766610] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T19:49:49.851383] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-06. [2025-10-20T19:49:50.102117] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T19:49:50.651352] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-10. [2025-10-20T19:49:51.287813] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T19:49:51.451384] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-07. [2025-10-20T19:49:51.865240] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T19:49:51.869077] [INFO ] [REMOTE ] MCS: Full fetch complete (30751 records). [2025-10-20T19:49:51.869155] [DEBUG] [VALIDATOR ] MCS: Validating primary data. [2025-10-20T19:49:52.231080] [DEBUG] [VALIDATOR ] MCS: All expected trading days are present. [2025-10-20T19:49:52.231157] [DEBUG] [VALIDATOR ] MCS: Primary data validation passed. [2025-10-20T19:49:52.231786] [INFO ] [MAIN ] MCS: Fetched primary data is valid. [2025-10-20T19:49:52.231847] [DEBUG] [LOCAL ] MCS: Saving primary data. [2025-10-20T19:49:52.232518] [DEBUG] [LOCAL ] MCS: Trimmed 30751 records to 21228 (keeping 200 days of data) [2025-10-20T19:49:52.282238] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-06. [2025-10-20T19:49:52.361020] [DEBUG] [LOCAL ] MCS: Primary data saved (21228 records). [2025-10-20T19:49:52.361136] [INFO ] [MAIN ] MCS: Full history saved. [2025-10-20T19:49:52.361209] [INFO ] [MAIN ] Update Progress: 1316/2367 symbols completed. MCS took 38.9s. Estimated remaining: 2h 15m 36s. Rate: 7.8 symbols/min [2025-10-20T19:49:52.532941] [INFO ] [MAIN ] MDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:52.556095] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T19:49:52.610848] [DEBUG] [LOCAL ] MDT: Loaded 54458 existing records from primary storage. [2025-10-20T19:49:52.611162] [INFO ] [MAIN ] MDT: Loaded 54458 existing records. [2025-10-20T19:49:52.794067] [WARN ] [MAIN ] MDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:52.794386] [INFO ] [MAIN ] MDT: Wiping existing primary data before full refresh. [2025-10-20T19:49:52.794444] [INFO ] [LOCAL ] MDT: Starting deletion of primary data. [2025-10-20T19:49:52.794731] [DEBUG] [LOCAL ] MDT: Deleted primary data file. [2025-10-20T19:49:52.794788] [INFO ] [MAIN ] MDT: Starting full primary refresh. [2025-10-20T19:49:52.975984] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T19:49:53.056569] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-08. [2025-10-20T19:49:53.167710] [INFO ] [REMOTE ] MDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:53.851353] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-07. [2025-10-20T19:49:54.159225] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T19:49:54.651407] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-08. [2025-10-20T19:49:54.716687] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T19:49:55.451386] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-08. [2025-10-20T19:49:56.088824] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T19:49:56.089253] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T19:49:56.251406] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-09. [2025-10-20T19:49:56.965306] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T19:49:57.051467] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-09. [2025-10-20T19:49:57.851350] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-01. [2025-10-20T19:49:58.038942] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T19:49:58.651377] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-09. [2025-10-20T19:49:59.451381] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-10. [2025-10-20T19:49:59.814896] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T19:50:00.146019] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:50:00.251492] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-02. [2025-10-20T19:50:00.668693] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T19:50:00.674117] [INFO ] [REMOTE ] MCY: Full fetch complete (40951 records). [2025-10-20T19:50:00.674292] [DEBUG] [VALIDATOR ] MCY: Validating primary data. [2025-10-20T19:50:01.056489] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-10. [2025-10-20T19:50:01.122518] [DEBUG] [VALIDATOR ] MCY: All expected trading days are present. [2025-10-20T19:50:01.122616] [DEBUG] [VALIDATOR ] MCY: Primary data validation passed. [2025-10-20T19:50:01.123393] [INFO ] [MAIN ] MCY: Fetched primary data is valid. [2025-10-20T19:50:01.123455] [DEBUG] [LOCAL ] MCY: Saving primary data. [2025-10-20T19:50:01.124427] [DEBUG] [LOCAL ] MCY: Trimmed 40951 records to 23809 (keeping 200 days of data) [2025-10-20T19:50:01.264054] [DEBUG] [LOCAL ] MCY: Primary data saved (23809 records). [2025-10-20T19:50:01.264177] [INFO ] [MAIN ] MCY: Full history saved. [2025-10-20T19:50:01.264251] [INFO ] [MAIN ] Update Progress: 1317/2367 symbols completed. MCY took 40.1s. Estimated remaining: 2h 15m 30s. Rate: 7.7 symbols/min [2025-10-20T19:50:01.851359] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-10. [2025-10-20T19:50:02.019781] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T19:50:02.021532] [INFO ] [REMOTE ] MDST: Full fetch complete (8828 records). [2025-10-20T19:50:02.021634] [DEBUG] [VALIDATOR ] MDST: Validating primary data. [2025-10-20T19:50:02.397322] [DEBUG] [VALIDATOR ] MDST: All expected trading days are present. [2025-10-20T19:50:02.397516] [DEBUG] [VALIDATOR ] MDST: Primary data validation passed. [2025-10-20T19:50:02.397776] [INFO ] [MAIN ] MDST: Fetched primary data is valid. [2025-10-20T19:50:02.397851] [DEBUG] [LOCAL ] MDST: Saving primary data. [2025-10-20T19:50:02.398384] [DEBUG] [LOCAL ] MDST: Trimmed 8828 records to 6125 (keeping 200 days of data) [2025-10-20T19:50:02.439441] [DEBUG] [LOCAL ] MDST: Primary data saved (6125 records). [2025-10-20T19:50:02.439563] [INFO ] [MAIN ] MDST: Full history saved. [2025-10-20T19:50:02.439638] [INFO ] [MAIN ] Update Progress: 1318/2367 symbols completed. MDST took 21.1s. Estimated remaining: 2h 15m 17s. Rate: 7.8 symbols/min [2025-10-20T19:50:02.557834] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-02 (2.31 seconds). [2025-10-20T19:50:02.651369] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-03. [2025-10-20T19:50:03.317290] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T19:50:03.323262] [INFO ] [REMOTE ] MD: Full fetch complete (55218 records). [2025-10-20T19:50:03.323362] [DEBUG] [VALIDATOR ] MD: Validating primary data. [2025-10-20T19:50:03.730792] [DEBUG] [VALIDATOR ] MD: All expected trading days are present. [2025-10-20T19:50:03.730888] [DEBUG] [VALIDATOR ] MD: Primary data validation passed. [2025-10-20T19:50:03.731862] [INFO ] [MAIN ] MD: Fetched primary data is valid. [2025-10-20T19:50:03.731929] [DEBUG] [LOCAL ] MD: Saving primary data. [2025-10-20T19:50:03.733221] [DEBUG] [LOCAL ] MD: Trimmed 55218 records to 38589 (keeping 200 days of data) [2025-10-20T19:50:03.959963] [DEBUG] [LOCAL ] MD: Primary data saved (38589 records). [2025-10-20T19:50:03.960113] [INFO ] [MAIN ] MD: Full history saved. [2025-10-20T19:50:03.960194] [INFO ] [MAIN ] Update Progress: 1319/2367 symbols completed. MD took 28.4s. Estimated remaining: 2h 15m 4s. Rate: 7.8 symbols/min [2025-10-20T19:50:04.735224] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T19:50:04.747021] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-04. [2025-10-20T19:50:06.657926] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T19:50:06.669108] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-05. [2025-10-20T19:50:08.553364] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T19:50:08.564563] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-06. [2025-10-20T19:50:10.813582] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-06 (2.25 seconds). [2025-10-20T19:50:10.824463] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-07. [2025-10-20T19:50:12.931222] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T19:50:12.942926] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-08. [2025-10-20T19:50:14.988769] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T19:50:15.000876] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-09. [2025-10-20T19:50:16.968022] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:50:16.979416] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-10. [2025-10-20T19:50:17.824985] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T19:50:17.832777] [INFO ] [REMOTE ] MDT: Full fetch complete (77857 records). [2025-10-20T19:50:17.832896] [DEBUG] [VALIDATOR ] MDT: Validating primary data. [2025-10-20T19:50:18.230938] [DEBUG] [VALIDATOR ] MDT: All expected trading days are present. [2025-10-20T19:50:18.231019] [DEBUG] [VALIDATOR ] MDT: Primary data validation passed. [2025-10-20T19:50:18.232000] [INFO ] [MAIN ] MDT: Fetched primary data is valid. [2025-10-20T19:50:18.232060] [DEBUG] [LOCAL ] MDT: Saving primary data. [2025-10-20T19:50:18.233414] [DEBUG] [LOCAL ] MDT: Trimmed 77857 records to 53678 (keeping 200 days of data) [2025-10-20T19:50:18.556695] [DEBUG] [LOCAL ] MDT: Primary data saved (53678 records). [2025-10-20T19:50:18.556844] [INFO ] [MAIN ] MDT: Full history saved. [2025-10-20T19:50:18.557139] [INFO ] [MAIN ] Update Progress: 1320/2367 symbols completed. MDT took 26.2s. Estimated remaining: 2h 15m 2s. Rate: 7.8 symbols/min [2025-10-20T19:50:19.129119] [INFO ] [MAIN ] MED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:19.258027] [INFO ] [MAIN ] MEC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:19.303252] [INFO ] [MAIN ] MDV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:19.311295] [INFO ] [MAIN ] MDU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:19.328484] [DEBUG] [LOCAL ] MDV: Loaded 8524 existing records from primary storage. [2025-10-20T19:50:19.328926] [INFO ] [MAIN ] MDV: Loaded 8524 existing records. [2025-10-20T19:50:19.381275] [DEBUG] [LOCAL ] MEC: Loaded 16672 existing records from primary storage. [2025-10-20T19:50:19.449714] [INFO ] [MAIN ] MEC: Loaded 16672 existing records. [2025-10-20T19:50:19.444098] [DEBUG] [LOCAL ] MED: Loaded 21503 existing records from primary storage. [2025-10-20T19:50:19.503898] [INFO ] [MAIN ] MED: Loaded 21503 existing records. [2025-10-20T19:50:19.762105] [WARN ] [MAIN ] MDV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:19.849011] [INFO ] [MAIN ] MDV: Wiping existing primary data before full refresh. [2025-10-20T19:50:19.850748] [DEBUG] [LOCAL ] MDU: Loaded 48794 existing records from primary storage. [2025-10-20T19:50:19.884166] [INFO ] [MAIN ] MDU: Loaded 48794 existing records. [2025-10-20T19:50:19.884027] [INFO ] [LOCAL ] MDV: Starting deletion of primary data. [2025-10-20T19:50:19.900721] [WARN ] [MAIN ] MEC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:19.916478] [INFO ] [MAIN ] MEC: Wiping existing primary data before full refresh. [2025-10-20T19:50:19.988548] [INFO ] [LOCAL ] MEC: Starting deletion of primary data. [2025-10-20T19:50:20.060987] [WARN ] [MAIN ] MED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:20.053230] [DEBUG] [LOCAL ] MEC: Deleted primary data file. [2025-10-20T19:50:20.011941] [DEBUG] [LOCAL ] MDV: Deleted primary data file. [2025-10-20T19:50:20.076827] [INFO ] [MAIN ] MDV: Starting full primary refresh. [2025-10-20T19:50:20.066451] [INFO ] [MAIN ] MEC: Starting full primary refresh. [2025-10-20T19:50:20.087293] [INFO ] [MAIN ] MED: Wiping existing primary data before full refresh. [2025-10-20T19:50:20.127538] [INFO ] [LOCAL ] MED: Starting deletion of primary data. [2025-10-20T19:50:20.166561] [DEBUG] [LOCAL ] MED: Deleted primary data file. [2025-10-20T19:50:20.185512] [INFO ] [MAIN ] MED: Starting full primary refresh. [2025-10-20T19:50:20.472256] [WARN ] [MAIN ] MDU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:20.586909] [INFO ] [MAIN ] MDU: Wiping existing primary data before full refresh. [2025-10-20T19:50:20.688929] [INFO ] [LOCAL ] MDU: Starting deletion of primary data. [2025-10-20T19:50:20.770226] [DEBUG] [LOCAL ] MDU: Deleted primary data file. [2025-10-20T19:50:20.841607] [INFO ] [MAIN ] MDU: Starting full primary refresh. [2025-10-20T19:50:21.233857] [INFO ] [REMOTE ] MEC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:21.255356] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-01. [2025-10-20T19:50:21.397583] [INFO ] [REMOTE ] MDV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:21.455109] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-01. [2025-10-20T19:50:21.454748] [INFO ] [REMOTE ] MED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:21.465242] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-01. [2025-10-20T19:50:21.665962] [INFO ] [REMOTE ] MDU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:21.666300] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-01. [2025-10-20T19:50:22.130665] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T19:50:22.132839] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-02. [2025-10-20T19:50:22.283202] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T19:50:22.286860] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-02. [2025-10-20T19:50:22.669134] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T19:50:22.674222] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-02. [2025-10-20T19:50:22.839465] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T19:50:22.841829] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-03. [2025-10-20T19:50:22.944612] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T19:50:22.954361] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-02. [2025-10-20T19:50:23.054364] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T19:50:23.056769] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-03. [2025-10-20T19:50:23.365464] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T19:50:23.368091] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-04. [2025-10-20T19:50:23.807321] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:50:23.812511] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-03. [2025-10-20T19:50:23.978127] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-10-20T19:50:23.983394] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-04. [2025-10-20T19:50:24.099284] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:50:24.108928] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-03. [2025-10-20T19:50:24.288032] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T19:50:24.290627] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-05. [2025-10-20T19:50:24.948804] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T19:50:24.954121] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-04. [2025-10-20T19:50:25.035531] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T19:50:25.041080] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-05. [2025-10-20T19:50:25.081887] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T19:50:25.084583] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-06. [2025-10-20T19:50:25.769113] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T19:50:25.774801] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-06. [2025-10-20T19:50:25.862537] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T19:50:25.868866] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T19:50:25.871776] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-05. [2025-10-20T19:50:25.873133] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-07. [2025-10-20T19:50:25.985468] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T19:50:25.995825] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-04. [2025-10-20T19:50:26.360205] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T19:50:26.427361] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-10-20T19:50:26.659178] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-07. [2025-10-20T19:50:26.660309] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T19:50:27.302364] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T19:50:27.451363] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-05. [2025-10-20T19:50:27.564416] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T19:50:28.251371] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-06. [2025-10-20T19:50:29.051366] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-08. [2025-10-20T19:50:29.155711] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:50:29.331528] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T19:50:29.851498] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-08. [2025-10-20T19:50:30.041531] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:50:30.586277] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T19:50:30.651432] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-07. [2025-10-20T19:50:31.451411] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-09. [2025-10-20T19:50:31.793590] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:50:32.020512] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T19:50:32.251407] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-06. [2025-10-20T19:50:33.051406] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-08. [2025-10-20T19:50:33.829282] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:50:33.851329] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-07. [2025-10-20T19:50:34.126625] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T19:50:34.651369] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-10. [2025-10-20T19:50:35.430964] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:50:35.451399] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-08. [2025-10-20T19:50:35.711835] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T19:50:35.714525] [INFO ] [REMOTE ] MEC: Full fetch complete (22759 records). [2025-10-20T19:50:35.714594] [DEBUG] [VALIDATOR ] MEC: Validating primary data. [2025-10-20T19:50:36.082037] [DEBUG] [VALIDATOR ] MEC: All expected trading days are present. [2025-10-20T19:50:36.082128] [DEBUG] [VALIDATOR ] MEC: Primary data validation passed. [2025-10-20T19:50:36.082576] [INFO ] [MAIN ] MEC: Fetched primary data is valid. [2025-10-20T19:50:36.082680] [DEBUG] [LOCAL ] MEC: Saving primary data. [2025-10-20T19:50:36.083300] [DEBUG] [LOCAL ] MEC: Trimmed 22759 records to 16516 (keeping 200 days of data) [2025-10-20T19:50:36.178802] [DEBUG] [LOCAL ] MEC: Primary data saved (16516 records). [2025-10-20T19:50:36.178953] [INFO ] [MAIN ] MEC: Full history saved. [2025-10-20T19:50:36.179038] [INFO ] [MAIN ] Update Progress: 1321/2367 symbols completed. MEC took 17.6s. Estimated remaining: 2h 15m 2s. Rate: 7.7 symbols/min [2025-10-20T19:50:36.256610] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-09. [2025-10-20T19:50:36.353613] [INFO ] [MAIN ] MEG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:36.384736] [DEBUG] [LOCAL ] MEG: Loaded 32094 existing records from primary storage. [2025-10-20T19:50:36.385000] [INFO ] [MAIN ] MEG: Loaded 32094 existing records. [2025-10-20T19:50:36.562416] [WARN ] [MAIN ] MEG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:36.562667] [INFO ] [MAIN ] MEG: Wiping existing primary data before full refresh. [2025-10-20T19:50:36.562729] [INFO ] [LOCAL ] MEG: Starting deletion of primary data. [2025-10-20T19:50:36.562912] [DEBUG] [LOCAL ] MEG: Deleted primary data file. [2025-10-20T19:50:36.562951] [INFO ] [MAIN ] MEG: Starting full primary refresh. [2025-10-20T19:50:36.927731] [INFO ] [REMOTE ] MEG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:36.970990] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T19:50:37.014330] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T19:50:37.051339] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-09. [2025-10-20T19:50:37.851415] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-10. [2025-10-20T19:50:38.651373] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-09. [2025-10-20T19:50:38.679535] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T19:50:38.681185] [INFO ] [REMOTE ] MDV: Full fetch complete (11685 records). [2025-10-20T19:50:38.681256] [DEBUG] [VALIDATOR ] MDV: Validating primary data. [2025-10-20T19:50:39.028762] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T19:50:39.118309] [DEBUG] [VALIDATOR ] MDV: All expected trading days are present. [2025-10-20T19:50:39.118383] [DEBUG] [VALIDATOR ] MDV: Primary data validation passed. [2025-10-20T19:50:39.118674] [INFO ] [MAIN ] MDV: Fetched primary data is valid. [2025-10-20T19:50:39.118732] [DEBUG] [LOCAL ] MDV: Saving primary data. [2025-10-20T19:50:39.119154] [DEBUG] [LOCAL ] MDV: Trimmed 11685 records to 8358 (keeping 200 days of data) [2025-10-20T19:50:39.168879] [DEBUG] [LOCAL ] MDV: Primary data saved (8358 records). [2025-10-20T19:50:39.169009] [INFO ] [MAIN ] MDV: Full history saved. [2025-10-20T19:50:39.169085] [INFO ] [MAIN ] Update Progress: 1322/2367 symbols completed. MDV took 20.6s. Estimated remaining: 2h 14m 50s. Rate: 7.8 symbols/min [2025-10-20T19:50:39.340779] [INFO ] [MAIN ] MEGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:39.361596] [DEBUG] [LOCAL ] MEGI: Loaded 20099 existing records from primary storage. [2025-10-20T19:50:39.361870] [INFO ] [MAIN ] MEGI: Loaded 20099 existing records. [2025-10-20T19:50:39.460512] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-01. [2025-10-20T19:50:39.545008] [WARN ] [MAIN ] MEGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:39.545283] [INFO ] [MAIN ] MEGI: Wiping existing primary data before full refresh. [2025-10-20T19:50:39.545348] [INFO ] [LOCAL ] MEGI: Starting deletion of primary data. [2025-10-20T19:50:39.545535] [DEBUG] [LOCAL ] MEGI: Deleted primary data file. [2025-10-20T19:50:39.545592] [INFO ] [MAIN ] MEGI: Starting full primary refresh. [2025-10-20T19:50:39.835871] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T19:50:39.919217] [INFO ] [REMOTE ] MEGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:40.251412] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-01. [2025-10-20T19:50:40.951431] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:50:41.051342] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-02. [2025-10-20T19:50:41.347208] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:50:41.851398] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-10. [2025-10-20T19:50:42.308519] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T19:50:42.651377] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-10. [2025-10-20T19:50:43.451414] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-02. [2025-10-20T19:50:43.529343] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:50:43.544102] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T19:50:43.547219] [INFO ] [REMOTE ] MDU: Full fetch complete (70681 records). [2025-10-20T19:50:43.547401] [DEBUG] [VALIDATOR ] MDU: Validating primary data. [2025-10-20T19:50:43.547575] [INFO ] [REMOTE ] MED: Full fetch complete (31931 records). [2025-10-20T19:50:43.547679] [DEBUG] [VALIDATOR ] MED: Validating primary data. [2025-10-20T19:50:44.253948] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-03. [2025-10-20T19:50:44.369134] [DEBUG] [VALIDATOR ] MDU: All expected trading days are present. [2025-10-20T19:50:44.369718] [DEBUG] [VALIDATOR ] MDU: Primary data validation passed. [2025-10-20T19:50:44.370600] [INFO ] [MAIN ] MDU: Fetched primary data is valid. [2025-10-20T19:50:44.370895] [DEBUG] [VALIDATOR ] MED: All expected trading days are present. [2025-10-20T19:50:44.371066] [DEBUG] [LOCAL ] MDU: Saving primary data. [2025-10-20T19:50:44.371166] [DEBUG] [VALIDATOR ] MED: Primary data validation passed. [2025-10-20T19:50:44.372155] [INFO ] [MAIN ] MED: Fetched primary data is valid. [2025-10-20T19:50:44.372495] [DEBUG] [LOCAL ] MED: Saving primary data. [2025-10-20T19:50:44.374147] [DEBUG] [LOCAL ] MED: Trimmed 31931 records to 21123 (keeping 200 days of data) [2025-10-20T19:50:44.374037] [DEBUG] [LOCAL ] MDU: Trimmed 70681 records to 48046 (keeping 200 days of data) [2025-10-20T19:50:44.599073] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T19:50:44.739741] [DEBUG] [LOCAL ] MED: Primary data saved (21123 records). [2025-10-20T19:50:44.741895] [INFO ] [MAIN ] MED: Full history saved. [2025-10-20T19:50:44.742761] [INFO ] [MAIN ] Update Progress: 1323/2367 symbols completed. MED took 26.2s. Estimated remaining: 2h 14m 41s. Rate: 7.8 symbols/min [2025-10-20T19:50:44.985619] [DEBUG] [LOCAL ] MDU: Primary data saved (48046 records). [2025-10-20T19:50:44.985811] [INFO ] [MAIN ] MDU: Full history saved. [2025-10-20T19:50:44.989330] [INFO ] [MAIN ] Update Progress: 1324/2367 symbols completed. MDU took 26.4s. Estimated remaining: 2h 14m 27s. Rate: 7.8 symbols/min [2025-10-20T19:50:44.989118] [INFO ] [MAIN ] MEI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:45.054951] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-03. [2025-10-20T19:50:45.165480] [INFO ] [MAIN ] MET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:45.187607] [DEBUG] [LOCAL ] MEI: Loaded 32083 existing records from primary storage. [2025-10-20T19:50:45.188012] [INFO ] [MAIN ] MEI: Loaded 32083 existing records. [2025-10-20T19:50:45.385234] [WARN ] [MAIN ] MEI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:45.385467] [INFO ] [MAIN ] MEI: Wiping existing primary data before full refresh. [2025-10-20T19:50:45.385539] [INFO ] [LOCAL ] MEI: Starting deletion of primary data. [2025-10-20T19:50:45.385767] [DEBUG] [LOCAL ] MEI: Deleted primary data file. [2025-10-20T19:50:45.385825] [INFO ] [MAIN ] MEI: Starting full primary refresh. [2025-10-20T19:50:45.515896] [DEBUG] [LOCAL ] MET: Loaded 54096 existing records from primary storage. [2025-10-20T19:50:45.537203] [INFO ] [MAIN ] MET: Loaded 54096 existing records. [2025-10-20T19:50:45.964475] [INFO ] [REMOTE ] MEI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:45.997850] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-01. [2025-10-20T19:50:46.009535] [WARN ] [MAIN ] MET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:46.012974] [INFO ] [MAIN ] MET: Wiping existing primary data before full refresh. [2025-10-20T19:50:46.013417] [INFO ] [LOCAL ] MET: Starting deletion of primary data. [2025-10-20T19:50:46.015442] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T19:50:46.020870] [DEBUG] [LOCAL ] MET: Deleted primary data file. [2025-10-20T19:50:46.041059] [INFO ] [MAIN ] MET: Starting full primary refresh. [2025-10-20T19:50:46.042403] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T19:50:46.410299] [INFO ] [REMOTE ] MET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:46.651373] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-01. [2025-10-20T19:50:47.290712] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T19:50:47.451371] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-04. [2025-10-20T19:50:48.251369] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-04. [2025-10-20T19:50:48.375981] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T19:50:49.015191] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T19:50:49.051431] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-05. [2025-10-20T19:50:49.265748] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T19:50:49.851425] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-05. [2025-10-20T19:50:49.979797] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T19:50:50.651449] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-02. [2025-10-20T19:50:50.876559] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T19:50:51.451380] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-06. [2025-10-20T19:50:51.717468] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T19:50:52.251374] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-06. [2025-10-20T19:50:52.467778] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T19:50:53.051355] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-07. [2025-10-20T19:50:53.569478] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T19:50:53.851461] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-02. [2025-10-20T19:50:54.237760] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T19:50:54.651431] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-07. [2025-10-20T19:50:55.456528] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-03. [2025-10-20T19:50:55.486320] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T19:50:55.754799] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T19:50:56.251452] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-08. [2025-10-20T19:50:56.695434] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T19:50:57.051351] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-04. [2025-10-20T19:50:57.276950] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T19:50:57.851394] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-08. [2025-10-20T19:50:58.462713] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:50:58.651402] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-09. [2025-10-20T19:50:58.988816] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T19:50:59.451422] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-09. [2025-10-20T19:50:59.975822] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T19:51:00.251382] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-03. [2025-10-20T19:51:00.427715] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T19:51:01.051419] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-10. [2025-10-20T19:51:01.851433] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-05. [2025-10-20T19:51:02.227607] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T19:51:02.230539] [INFO ] [REMOTE ] MEGI: Full fetch complete (29178 records). [2025-10-20T19:51:02.230690] [DEBUG] [VALIDATOR ] MEGI: Validating primary data. [2025-10-20T19:51:02.612942] [DEBUG] [VALIDATOR ] MEGI: All expected trading days are present. [2025-10-20T19:51:02.613115] [DEBUG] [VALIDATOR ] MEGI: Primary data validation passed. [2025-10-20T19:51:02.613821] [INFO ] [MAIN ] MEGI: Fetched primary data is valid. [2025-10-20T19:51:02.613884] [DEBUG] [LOCAL ] MEGI: Saving primary data. [2025-10-20T19:51:02.614758] [DEBUG] [LOCAL ] MEGI: Trimmed 29178 records to 19796 (keeping 200 days of data) [2025-10-20T19:51:02.664551] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-10. [2025-10-20T19:51:02.738423] [DEBUG] [LOCAL ] MEGI: Primary data saved (19796 records). [2025-10-20T19:51:02.738588] [INFO ] [MAIN ] MEGI: Full history saved. [2025-10-20T19:51:02.738691] [INFO ] [MAIN ] Update Progress: 1325/2367 symbols completed. MEGI took 23.6s. Estimated remaining: 2h 14m 27s. Rate: 7.7 symbols/min [2025-10-20T19:51:02.916587] [INFO ] [MAIN ] MFA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:02.952270] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-03 (2.70 seconds). [2025-10-20T19:51:03.001691] [DEBUG] [LOCAL ] MFA: Loaded 46378 existing records from primary storage. [2025-10-20T19:51:03.002062] [INFO ] [MAIN ] MFA: Loaded 46378 existing records. [2025-10-20T19:51:03.190083] [WARN ] [MAIN ] MFA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:03.205473] [INFO ] [MAIN ] MFA: Wiping existing primary data before full refresh. [2025-10-20T19:51:03.206285] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T19:51:03.206376] [INFO ] [LOCAL ] MFA: Starting deletion of primary data. [2025-10-20T19:51:03.211761] [DEBUG] [LOCAL ] MFA: Deleted primary data file. [2025-10-20T19:51:03.212385] [INFO ] [MAIN ] MFA: Starting full primary refresh. [2025-10-20T19:51:03.457610] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-04. [2025-10-20T19:51:03.564399] [INFO ] [REMOTE ] MFA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:04.251389] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-06. [2025-10-20T19:51:04.282508] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T19:51:04.288236] [INFO ] [REMOTE ] MEG: Full fetch complete (47500 records). [2025-10-20T19:51:04.288355] [DEBUG] [VALIDATOR ] MEG: Validating primary data. [2025-10-20T19:51:04.692098] [DEBUG] [VALIDATOR ] MEG: All expected trading days are present. [2025-10-20T19:51:04.692186] [DEBUG] [VALIDATOR ] MEG: Primary data validation passed. [2025-10-20T19:51:04.692987] [INFO ] [MAIN ] MEG: Fetched primary data is valid. [2025-10-20T19:51:04.693053] [DEBUG] [LOCAL ] MEG: Saving primary data. [2025-10-20T19:51:04.694171] [DEBUG] [LOCAL ] MEG: Trimmed 47500 records to 31609 (keeping 200 days of data) [2025-10-20T19:51:04.876405] [DEBUG] [LOCAL ] MEG: Primary data saved (31609 records). [2025-10-20T19:51:04.876621] [INFO ] [MAIN ] MEG: Full history saved. [2025-10-20T19:51:04.876848] [INFO ] [MAIN ] Update Progress: 1326/2367 symbols completed. MEG took 28.7s. Estimated remaining: 2h 14m 15s. Rate: 7.8 symbols/min [2025-10-20T19:51:05.047592] [INFO ] [MAIN ] MFAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:05.059268] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-01. [2025-10-20T19:51:05.081296] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:51:05.096154] [DEBUG] [LOCAL ] MFAN: Loaded 4421 existing records from primary storage. [2025-10-20T19:51:05.096754] [INFO ] [MAIN ] MFAN: Loaded 4421 existing records. [2025-10-20T19:51:05.298586] [WARN ] [MAIN ] MFAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:05.298943] [INFO ] [MAIN ] MFAN: Wiping existing primary data before full refresh. [2025-10-20T19:51:05.299053] [INFO ] [LOCAL ] MFAN: Starting deletion of primary data. [2025-10-20T19:51:05.304512] [DEBUG] [LOCAL ] MFAN: Deleted primary data file. [2025-10-20T19:51:05.313613] [INFO ] [MAIN ] MFAN: Starting full primary refresh. [2025-10-20T19:51:05.320138] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T19:51:05.671058] [INFO ] [REMOTE ] MFAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:05.851377] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-01. [2025-10-20T19:51:06.401692] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T19:51:06.643921] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:51:06.652055] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-02. [2025-10-20T19:51:07.123263] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T19:51:07.451369] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-03. [2025-10-20T19:51:08.046292] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T19:51:08.251384] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-04. [2025-10-20T19:51:08.862301] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T19:51:09.051384] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-07. [2025-10-20T19:51:09.851416] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-02. [2025-10-20T19:51:10.557700] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T19:51:10.651395] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-08. [2025-10-20T19:51:11.422180] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:51:11.451337] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-03. [2025-10-20T19:51:11.959215] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:51:12.251371] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-09. [2025-10-20T19:51:13.051365] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-05. [2025-10-20T19:51:13.141417] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:51:13.602401] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:51:13.851378] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-10. [2025-10-20T19:51:14.651359] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-05. [2025-10-20T19:51:15.048275] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T19:51:15.131197] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T19:51:15.135804] [INFO ] [REMOTE ] MEI: Full fetch complete (45365 records). [2025-10-20T19:51:15.135900] [DEBUG] [VALIDATOR ] MEI: Validating primary data. [2025-10-20T19:51:15.334412] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T19:51:15.456608] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-06. [2025-10-20T19:51:15.541365] [DEBUG] [VALIDATOR ] MEI: All expected trading days are present. [2025-10-20T19:51:15.541443] [DEBUG] [VALIDATOR ] MEI: Primary data validation passed. [2025-10-20T19:51:15.542328] [INFO ] [MAIN ] MEI: Fetched primary data is valid. [2025-10-20T19:51:15.542389] [DEBUG] [LOCAL ] MEI: Saving primary data. [2025-10-20T19:51:15.543376] [DEBUG] [LOCAL ] MEI: Trimmed 45365 records to 31501 (keeping 200 days of data) [2025-10-20T19:51:15.717760] [DEBUG] [LOCAL ] MEI: Primary data saved (31501 records). [2025-10-20T19:51:15.717889] [INFO ] [MAIN ] MEI: Full history saved. [2025-10-20T19:51:15.718243] [INFO ] [MAIN ] Update Progress: 1327/2367 symbols completed. MEI took 31.0s. Estimated remaining: 2h 14m 10s. Rate: 7.8 symbols/min [2025-10-20T19:51:15.882089] [INFO ] [MAIN ] MFAO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:15.889687] [DEBUG] [LOCAL ] MFAO: Loaded 4021 existing records from primary storage. [2025-10-20T19:51:15.889907] [INFO ] [MAIN ] MFAO: Loaded 4021 existing records. [2025-10-20T19:51:16.067360] [WARN ] [MAIN ] MFAO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:16.067633] [INFO ] [MAIN ] MFAO: Wiping existing primary data before full refresh. [2025-10-20T19:51:16.067720] [INFO ] [LOCAL ] MFAO: Starting deletion of primary data. [2025-10-20T19:51:16.067886] [DEBUG] [LOCAL ] MFAO: Deleted primary data file. [2025-10-20T19:51:16.067942] [INFO ] [MAIN ] MFAO: Starting full primary refresh. [2025-10-20T19:51:16.171321] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T19:51:16.259325] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-04. [2025-10-20T19:51:16.421739] [INFO ] [REMOTE ] MFAO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:17.051369] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-01. [2025-10-20T19:51:17.612563] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:51:17.851351] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-07. [2025-10-20T19:51:17.886467] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T19:51:18.427505] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T19:51:18.651384] [INFO ] [RE